AI Agents vs Data Analysts: The Truth from Amplitude PM

Evans points to data quality and workflow as core constraints: useful analytics depends on reliable instrumentation, and actions such as launching experiments need to fit existing tools and processes. He says larger SaaS companies may also hesitate to launch early versions of AI products because expectations are high.

Evans frames the main analytics bottleneck as time. Many teams lack people who can spend hours exploring data, checking replays, and applying best practices. He describes Amplitude agents as AI users of the product that can carry out this work in the background.

Amplitude uses goal-specific agent templates, such as improving web conversion, rather than one agent responsible for every metric. In the demo, an agent finds dead clicks on the pricing page, uses session replay clips to support the finding, and proposes an A/B test to make plan headers clickable. A person reviews and approves the proposed action.

Evans says recommendations need to connect clearly to both the evidence and the template’s best practices so users can assess them. Amplitude looks at whether customers consider experiments reasonable enough to approve, while exploring ways to ask later whether an insight created value. He notes that judging insight quality remains difficult.

Evans expects agents to help teams run experiments they currently lack time for, rather than replace analysts or PMs outright. He calls the approach an Iron Man suit: agents can handle repetitive work while people retain feedback and decision-making roles. For AI product builders, he recommends prototyping coherent products, testing them with users, and adapting as models improve.