Autonomy is the real test for AI agents

"All vendors want to call their capabilities, their applications, an AI agent. It's just catchy," Jean Sullivan says on the latest WealthTech Today podcast, where she and Craig Iskowitz try to put the term on firmer footing. Their test is autonomy: whether the software can carry out an assignment unaided or needs a human to finish it, and an offering that depends on a person to execute, Sullivan argues, is non-agentic regardless of the label attached.

Ezra Group has given that test a catalog, and the firm's AI Agents Directory—which this publication covered in August when it listed nearly 50 AI agents—now tallies 69 agents and MCP servers, scoring each on a four-to-five-level autonomy scale built with input from the firm's Head of AI Labs and running from no autonomy to fully autonomous. The directory separates live production tools from prototypes and evaluates governance and fiduciary oversight, so buyers can compare real capabilities instead of marketing language.

The episode also raises a question that belongs in any purchase review—what the tool costs to run—because, the podcast says, token expenses have 98% of C-suite leaders reconsidering their AI plans yet only two-thirds of firms have any method for metering usage, leaving the rest, in the episode's words, flying blind. Pricing models are responding: traditional per-seat, per-account, and AUM pricing is giving way to usage- and outcome-based contracts that tie fees to work completed.

The growth case is the reason to take the autonomy test seriously: Sullivan says wealth managers are applying AI less as a cost-saver and more as a growth engine, particularly for serving the mass affluent and moving advisors to deeper client work. Some of the largest RIAs are already exploring AI-driven staffing models; one described plans for 500 virtual employees, a scale at which the purchase decision stops being a tool pick and becomes an operating-model choice.

Before signing, ask the vendor what the software does when no human is in the loop. If it waits for a click, an approval, or a typed instruction, it is an assistant, not an agent. An assistant can still be a fine purchase—priced like an assistant, scoped like an assistant, and measured on whether it actually cuts steps for the team.