The Amplitude AI line-up, per the documentation
- Global Agent. Built into every page of Amplitude. Ask a product question in plain language and it queries the data, builds and edits charts, creates cohorts and runs multi-step investigations; it is the entry point for ad-hoc analysis. Admins control whether it is on and can monitor activity through AI Controls.
- Specialized Agents. Four purpose-built agents that each watch one domain on a schedule and push findings to an Agent Inbox, email or Slack: Dashboard Agent, Session Replay Agent, Website Conversion Agent and Customer Feedback Agent. They do not act on end users without explicit human approval, admins can audit all input and output, and they inherit the organization's data access controls.
- AI Assistant. A different product: an in-product helper for your own users, answering from the docs and help content you connect, not a general analytics agent. It is listed here so the two are not confused in a question set.
- Data Assistant. Described by Amplitude as a way to spot taxonomy issues and keep the tracking plan clean at scale. Relevant to the audit because the taxonomy is what every other agent reads.
- Ask Amplitude appears in the documentation as an AI-powered feature, reachable in Slack through an MCP integration and subject to the same data access restrictions as the rest of the product. The pages we read do not state how it relates to the Global Agent.
Amplitude states that it orchestrates models from OpenAI, Claude through AWS Bedrock, and Gemini behind its own memory and tooling.
What the Global Agent does with a question
Amplitude documents three response modes.
| Mode | Time | Tool calls | Taxonomy exploration |
|---|---|---|---|
| Fast | 2 to 10 seconds | 2 to 4 | Minimal; simple lookups. |
| Default | 30 to 60 seconds | 4 to 6 | Explores the taxonomy; searches existing work before creating new charts. The recommended mode. |
| Investigate | 4 to 5 minutes | 10 or more | Extensive exploration and hypothesis testing. |
Every response is a short answer, 2 to 5 sentences, with links to the charts or analyses that back it, and optional follow-up suggestions. Amplitude describes the agent as reporting what the data shows and linking every claim to data, and its own guidance tells users to review and validate AI-created analyses before using them in important decisions.
The agent prefers existing work: it searches charts, dashboards, notebooks and cohorts and prioritizes content marked Official and highly viewed. That is a strength and a grading point. If the Official chart carries a wrong definition, the agent will find it first and answer from it.
Why the taxonomy is the whole story
Amplitude's own overview says the more complete the Amplitude setup, the more context and action recommendations the agents have to work with. The inverse is the audit's business.
An agent that explores the taxonomy finds the events and properties by name and by past use. It cannot see whether the event fires once or twice, whether the property is a constant, or whether a page fires the event inside a frame the SDK never reached. Those facts live on the site.
On one national retailer's site, a checkout progress event fired six times in one pass and begin checkout fired twice. Closing the mandatory cookie notice left every consent category on, so the consent field in the history is a constant. That site runs a different analytics stack, but the fault pattern is platform-agnostic: an event that double-fires into Amplitude is a doubled funnel step in every chart the Global Agent builds, and a property that never varies is a segment that cannot exist. The agent will still draw it.
How we grade an Amplitude agent answer
The row: the question, the answer with its linked charts and the mode it ran in, the measured truth, the evidence, one of six classes. Rubric on /how-we-grade.
Mode is recorded because Amplitude documents that modes differ in taxonomy exploration. The same question can land in different classes in Fast and Investigate.
Evidence is followed, not trusted. The chart link tells us which chart to open; the measured site tells us whether the chart's events hold what their names claim. Method on /tracking-audit.
For Specialized Agents, the scheduled output is the row: the Dashboard Agent's stated trend, the Website Conversion Agent's stated drop-off, each graded against the measured flow. Amplitude documents that these agents diagnose root causes and propose actions such as launching experiments; a root cause found in a doubled event is the most useful failure to catch, because a human would have approved the experiment.
The tracking plan is the fix
When a row fails on taxonomy, the fix is in the plan: one event per step, one property per fact, names that mean one thing. Amplitude's Data Assistant is documented as a tool for spotting taxonomy issues at scale. Our tracking-plan audit, the Amplitude implementation work behind it, and the vendor terms around Amplitude live on ampl.webclat.com. This page grades the answers; that site fixes the plan they come from.
The engagement is on /answer-audit; the method for the category on /ai-agent-evaluation; the one case we can show on /cases.