Before expanding AI automation across a private equity portfolio, measure the cost of a result that the business actually accepts. Include human review, corrections, integration support, and failed attempts alongside model charges. A low price per AI request does not establish a low cost per completed business task.
This question is timely. In its October 22, 2026 guide to simplifying agent systems, Anthropic describes enterprise deployments that benefited from consolidating specialist agents, while also explaining situations where separate agents remain useful. For an operating partner, the practical response is to ask for a measured comparison rather than assume either a larger or smaller agent system will deliver better economics.
The following framework is a suggested way to evaluate a pilot. It is not a prediction of investment returns. The numerical example is hypothetical.
Define the business result before counting activity
Choose a unit of work that operations can recognize: a correctly classified service request, an approved product description, or a reviewed contract summary. Generated words, tool calls, and completed agent runs are activity measures. They do not establish that the business received a usable result.
Write acceptance conditions before comparing alternatives. For a document summary, specify which sections must be covered, how statements should reference the original, and what happens when information is missing. Identify mistakes that require correction and mistakes that make the result unacceptable.
Count a case as accepted only after it meets those conditions. If an employee must substantially rewrite the output, capture that effort even when the automation reports that its run completed successfully.
Establish a representative baseline
Observe the current workflow before the pilot begins. Record incoming volume, completed cases, hands-on effort, waiting time, and common exceptions. Separate ordinary cases from difficult ones so the comparison does not reward the pilot for handling only easy work.
Waiting time and employee effort are different measures. An automation may shorten a queue without reducing much hands-on work. Another may save employee time while leaving an approval bottleneck untouched. Both can be useful findings, but they imply different improvements.
Document conditions that could distort the comparison: a staffing shortage, seasonal demand, a policy change, or unusually clean sample data. A baseline with visible limitations is more useful than a precise-looking result with hidden assumptions.
Include the surrounding operating costs
Ask the project owner to identify the costs required to keep the workflow working:
- Model usage and other consumption-based services.
- Software subscriptions and minimum commitments.
- Hosting, storage, and monitoring.
- Employee review and correction time.
- Integration support and incident investigation.
- Training and ongoing maintenance.
Show initial implementation spending separately from recurring operations. Also distinguish a planning value for employee time from an actual reduction in expense. Time released by automation can provide capacity for other work without reducing payroll.
Ask which costs grow with volume and which rise in steps. A pilot supported by a developer manually repairing failures may not represent the operating model proposed for the wider rollout. Include that intervention in the evaluation rather than treating it as free background effort.
Calculate cost per accepted task
A practical comparison divides the in-scope operating cost by the number of accepted tasks in the same period. Use the same cost boundaries for the existing process and proposed alternatives.
Consider this hypothetical monthly pilot:
- The system attempts 1,000 cases, of which 800 meet the acceptance conditions.
- Software and service charges total $400.
- Review and correction take 20 hours, valued for planning at $40 per hour.
- Support takes five hours, valued at $60 per hour.
The modeled operating cost is $1,500: $400 plus $800 plus $300. Dividing by 800 accepted cases gives approximately $1.88 per accepted case. Reporting only $0.40 in service charges per attempted case answers a different question.
These assumptions are illustrative, not market prices or promised results. The labor values are not necessarily incremental cash expenses. Implementation spending and the consequences of unsuccessful cases are excluded from this calculation and must be considered separately before approving a larger investment.
Make architectural complexity justify itself
When a supplier proposes multiple specialist agents, ask what each component contributes and what evidence supports that design. A role in a system diagram is not a business benefit.
Request a comparison against a simpler implementation using the same cases and acceptance conditions. Review the failures, not only the average score. A design that performs better overall may still fail a critical category the company cannot accept.
The current debate about simplifying agents is a reason to test alternatives, not an instruction to rebuild every existing deployment. A functioning system should have a credible improvement case before management accepts migration cost and disruption.
Ask who will maintain the result. Lower runtime costs may be offset by fragile integrations or a configuration only its original developer understands. Include diagnosis, ownership, and supportability in the decision.
Measure quality and turnaround alongside cost
A cost metric can reward incomplete work if it stands alone. Pair it with acceptance rate, correction effort, turnaround time, and the categories of errors observed.
Define severe mistakes separately. Assigning a summary to the wrong customer is different from an awkward sentence. Management should be able to see whether better performance on ordinary cases conceals unacceptable failures elsewhere.
A concise review should answer four questions: did the task succeed, what did it cost, how long did it take, and what happened when it failed? Keep those answers visible rather than compressing everything into a single score.
Permissions and approval controls remain a separate readiness requirement. Blake Technologies’ AI agent permissions guide covers that evaluation. Favorable economics do not justify removing controls a workflow needs.
Validate transferability across portfolio companies
Do not extrapolate one company’s results to the entire portfolio without examining task mix, data quality, integrations, and local operating procedures. Processes that share a name may require different work.
For each proposed rollout, identify what can be reused and what must be revalidated. A common evaluation method may transfer more easily than a complete application. Keep company-specific records and access appropriately separated.
Ask local management to confirm the baseline and acceptance conditions. This gives the portfolio team comparable evidence without pretending all companies have identical economics.
If the underlying operational problem is still unclear, start there. The guide to overlooked portfolio value explains how to investigate handoffs and exceptions before selecting a solution. AI may fit a narrow task within an improvement rather than the whole process.
Agree on the decision before the pilot ends
Set a review date and define the evidence needed to expand, revise, or stop. Include the operational owner, finance reviewer, and technical support owner.
Possible decisions include retaining the current process, automating only a subset of cases, simplifying the implementation, or expanding gradually. A pilot that prevents an unsupported larger commitment can still be valuable.
Preserve the evaluation assumptions and observed failures within the company’s normal project records. They provide a baseline when models, service charges, or business needs change. Revisit the comparison when a material assumption changes rather than treating the first result as permanent.
Invest in demonstrated outcomes
The number of agents deployed is not an operating result. Focus on a task the business needs, an outcome it can accept, and a complete account of the work required to produce it.
Start with one representative pilot. Compare quality, cost, turnaround, and supportability, then expand where the evidence supports the next step. That creates a clearer basis for allocating implementation resources across a portfolio.
Illustrative photography from Unsplash: a working meeting, paperwork and a calculator, and office buildings. These images do not depict Blake Technologies clients or portfolio companies.