Agent Analytics, Monitoring and Cost

The surfaces

Settings carries several agent tabs: Agent and Agent Config for behaviour and model, Agent Tools for custom webhook tools, Agent Analytics, and Agent Monitor.

Analytics is the aggregate: how much the agent is being used, by whom, for what.

Monitor is the operational view: what is running, what has run, and what went wrong. When someone reports the agent failing, this is where you look before asking them to reproduce it.

Credits and plan live on the billing tabs, covered properly in Module 11. What matters here is the connection between the two: agent activity is what consumes credits, and these tabs are how you attribute it.

What drives cost

Three things, in rough order.

How much context a turn carries. A conversation with a lot of history, in a workspace with a large tool registry, costs more per turn than a fresh short one. This is the main reason to start new threads for new topics rather than continuing a three-week conversation.

How many tool calls a request needs. 'Summarize this task' is one lookup. 'Review every open deal and draft follow-ups' is many, and it costs accordingly. That is fine — it is also doing much more — but it is worth knowing that the second is a different order of spend.

And the model in use. Personas can specify their own model; a persona doing simple triage does not need the most capable one, and choosing deliberately is a real lever.

The two patterns that grow unnoticed

Recurring runs. A daily schedule is roughly 250 runs a year, every one billed, and its cost never changes week to week, which is exactly why nobody notices it. This is the single most common source of surprise consumption. The defence is the quarterly review from Lesson 8.5.

Exec meetings. A weekly boardroom of five execs is a large number of model calls per meeting, on a cadence. Same shape, larger unit cost. Set a daily budget on the AI-spend category.

Both share the property that they were configured once, by someone who has since moved on to other things, and they will keep running until someone deliberately looks. Put that look on a calendar.

Guardrails worth having

A daily budget on the AI-spend action category. This is the direct control: it caps how many credit-spending actions the agent may start in a day, and it is what turns a misconfigured loop from an incident into a capped annoyance.

A deliberate model choice per persona. Specialists doing narrow work rarely need the largest model.

A quarterly schedule review. Every recurring run and every armed boardroom, one question each: what did the last one say, and did anyone act on it?

And an alert you actually receive when something goes wrong with billing or charging. This matters more than it sounds: the failure mode where usage is not billed correctly is one where the problem compounds silently rather than stopping, so a failed charge is something you want to hear about rather than discover.

Keeping cost proportionate

The honest framing is a ratio rather than a number. An agent that costs a meaningful amount and saves several hours a week is cheap. One that costs a small amount and produces nothing anyone reads is expensive.

So the review question is never 'can we reduce spend'. It is 'which of these things is anyone acting on'. In practice that usually finds one or two recurring items to pause and leaves the conversational usage — which people actively choose to do, and therefore self-regulates — alone.