How to Deploy and Manage AI Agents

Welcome to this edition of Balanced AI Insights, part of our ongoing series focused on AI agents in finance. In earlier issues, we examined their capabilities, possible applications, and the future of hybrid human-AI collaborations. Today, we take a closer look at a vital stage: the effective deployment and scaling of AI agents. 

Let’s begin!

The Balanced View: The Step-by-Step Guide to Building AI Teams

AI is revolutionizing finance teams. We can envision a future where finance leaders oversee a blend of human workers and AI agents. Nevertheless, the deployment of AI agents necessitates meticulous implementation. Understanding how to initiate, expand, and oversee these tools is vital for achieving success. 

AI Agents Work in Teams

A single AI agent rarely handles complex financial processes. The technology is not yet advanced enough for AI agents to manage complex, multi-step tasks independently. They perform much better when tasks are divided into smaller, specific pieces, with each agent trained to handle one task effectively.

To ensure smooth operation, “AI managers” or orchestrator agents oversee these tasks, verifying outputs and managing the workflow across the team of AI agents.

Example: Budget Update Automation

For example, an AI agent designed to handle budget updates would operate as a team of agents working in tandem. Here’s how it could function:

1. Data Retrieval Agent: Gathers actual financial data from accounting systems or relevant sources.

2. Variance Analysis Agent: Compares actuals to budgeted figures, identifies variances, and analyzes the underlying causes.

3. Budget Adjustment Agent: Suggests corrections to future budget periods, considering trends and historical data.

An orchestrator agent would oversee these steps, ensuring accuracy, consistency, and alignment across the agents. This approach ensures that each step is handled by a specialized agent, leading to more precise and actionable outputs.

Step-by-Step Guide to Building AI Teams

Step 1: Break Down the Process
Identify the process or task you aim to optimize with AI. Divide it into smaller, manageable components or subtasks. Ensure each task is clearly defined with measurable outcomes and potential risks noted.

Development Phase

Step 2: Map the Workflow
Create a process map with control checkpoints. Identify critical workflow points for validations, approvals, or reviews to ensure accuracy and compliance.

Step 3: Select the Right Platform
Choose a platform that aligns with your needs and integrates smoothly with your current technology stack.

Step 4: Define Success Metrics and Testing Parameters
Set clear success criteria for each AI agent. Define testing parameters for subtasks and document possible failures. Incorporate QA procedures to ensure the AI agent's accuracy and functionality during development.

Step 5: Build and Test AI Agents for Each Task
Create AI agents for specific tasks with controls to validate inputs, outputs, and decisions. Test comprehensively in diverse scenarios to ensure proper performance.

Deployment Phase

Step 6: Deploy Specialized AI Agents
Introduce AI agents gradually, establishing automated checkpoints or manual reviews to ensure output accuracy before continuing.

Step 7: Integrate and Validate the Workflow
Integrate AI agents into the workflow and conduct end-to-end testing. Verify the system respects controls and that agent dependencies function smoothly.

Step 8: Introduce a Manager Agent
Deploy a manager agent to oversee the AI team. Equip this agent with oversight capabilities, such as monitoring agent activities, flagging anomalies, and ensuring adherence to control points.

Post-Deployment Phase

Step 9: Monitor Performance and Maintain Logs
Continuously monitor the AI team’s performance using dashboards and logs. Regularly review these to identify discrepancies or patterns needing adjustments.

Step 10: Perform Regular Audits and Updates
Conduct audits to ensure AI compliance with controls and regulations. Update algorithms and processes as needed for new data or changes.

Step 11: Establish a Feedback Loop
Establish a feedback system for stakeholders to report issues, suggest improvements, or raise concerns, using their input to improve system reliability over time.

Step 12: Prepare for Incident Response
Create an incident response plan for unexpected failures. This plan must detail how to identify, isolate, and resolve issues while minimizing disruptions.

While experimenting with AI, tech-savvy finance leaders might successfully set up basic custom GPT models without external help. However, building an interconnected system of AI agents capable of handling dynamic inputs, making decisions, and adapting as they process tasks requires more profound technical expertise. 

Here's why:

• The AI Agent Landscape Is Complex: The sheer variety of AI agents and platforms can be overwhelming. For an idea of the scope, see the AI Agents Directory landscape.

• Navigating Alone Is Nearly Impossible: Identifying the right tools, ensuring compatibility, and integrating them into cohesive systems requires specialized knowledge.

• Finance Processes Are High-Stakes: Financial workflows are highly sensitive, where even small mistakes can have significant repercussions. From compliance risks to financial inaccuracies, ensuring precision is paramount when implementing AI in finance.

I created multiple custom GPTs for myself and my clients with minimal technical assistance. However, AI agents are a significantly more complicated area; therefore, I’ve realized that developing AI agent systems requires cooperation with technical specialists. Depending on the project's intricacy and the client’s technology stack, I collaborate with dedicated AI implementation agencies to access the necessary expertise.

I’m excited about the future of AI agents. I have a roadmap for automating processes with AI agents for both my clients and myself. However, it’s crucial to acknowledge that we are still in the early stages. Until AI agents become dependable partners, there will be much experimentation ahead.

In the next section, I want to share how I am transforming one of my favorite Custom GPTs into a system of AI agents to further automate quarterly investor report preparation.

A Sneak Peek: Transforming a Custom GPT into an AI Agent System

I am in the process of upgrading a custom GPT model I created last year for automating quarterly investor reports, transforming it into a complete system of AI agents.

Previously, the custom GPT generated a draft investor report based on the financial and market data I uploaded. After the close of each quarter, I would collect data from various sources: financial reports from QuickBooks, market updates from industry tools, and company updates from meeting minutes or relevant emails. I had to upload all the data sources to my custom GPT and prompt it to create a draft of the report based on the format that I shared.

The result was a well-formatted draft with visualizations, insights, and summaries. This process reduced preparation time from several days to just a few hours. It is one of my favorite and frequently used GPTs.

The next step is to refine and expand this system by adding specialized AI agents to automate the data retrieval and synthesis process further, eliminating even more manual steps.

Here's how I am building this streamlined, multi-agent system:

1. Data Retrieval Agent automates the collection of quarterly financials directly from QuickBooks.

2. Research Agent focuses on external research, gathering updates on industry trends, competitor activities, and market insights.

3. Company Updates Agent monitors internal sources, such as leadership meeting minutes, to identify relevant updates, such as new product launches, leadership changes, or significant milestones.

4. Report Drafting Agent (an upgraded existing custom GPT) compiles the collected data and insights into a structured investor report and ensures consistent formatting,

5. Orchestrator Agent is the central coordinator, managing the workflow across all agents. It triggers each step, validates outputs, and ensures seamless integration of inputs before notifying that the report is ready for review.

I normally don’t share work in progress until I validate and test the solution, but in this case, I am confident that this project will be implemented successfully.

Contact me if you’re looking to achieve similar automation. I‘ll build a custom GPT tailored to your needs, capable of automating 80% of your reporting task within a few days.

Plus, once my system of AI agents is tested, you’ll get early access and an accelerated implementation since one of the agents will already be up and running. This solution isn’t limited to quarterly investor reports; it works seamlessly for any regular report you’re preparing.

Closing Thoughts

Implementing AI agents is a more strategic endeavor than using generic AI models like ChatGPT or Claude; plus, it requires greater technical expertise.

The key to success lies in starting thoughtfully. Begin small, with a clear process or workflow in mind, and ensure every AI agent has a specific purpose. Modular designs allow you to scale without overwhelming your team, while embedded controls ensure that accuracy and compliance are never compromised.

AI agents are the next step in AI automation, enabling finance leaders to eliminate even more manual tasks.

We Want Your Feedback!

This newsletter is for you, and we want to make it as valuable as possible. Please reply to this email with your questions, comments, or topics you'd like to see covered in future issues. Your input shapes our content!

Want to dive deeper into balanced AI adoption for your finance team? Or do you want to hire an AI-powered CFO? Book a consultation! 

Did you find this newsletter helpful? Forward it to a colleague who might benefit!

Until next Tuesday, keep balancing!

Anna Tiomina 
AI-Powered CFO