Module 7 — Agentic Project Management · Lesson 7.3
Encoding Recurring Work — Runbooks, Goals, and Schedules
Turning the management jobs you redo every week into things the agent runs, with checkpoints where judgment belongs
~13 min
What you'll learn
- Tell a workflow apart from an agent, and pick the cheaper one deliberately
- Write a runbook whose checkpoints sit where judgment actually is
- Define a goal the agent can measure rather than a slogan it cannot
- Decide what deserves a scheduled run and what should stay on demand
Setting an autonomy level tells the agent how much it may do. It does not tell it what to do. Left there, the result is a well-governed agent that sits idle until someone thinks to ask it something — which is most agentic deployments, and the reason so many of them quietly lapse. The work worth handing over is the work you redo. The quarterly close. The weekly triage sweep. The new-client onboarding sequence you have run eleven times and still improvise. Kavanah has three surfaces for encoding that, and they answer three different questions: runbooks say how, goals say whether it is working, and schedules say when.
Workflow or agent — pick the cheap one on purpose
There is a distinction worth borrowing before you build anything. In Anthropic's engineering guidance, workflows are systems where models and tools are orchestrated through predefined code paths, while agents are systems where the model dynamically directs its own process and tool use. The same guidance is blunt about the trade: the autonomous nature of agents means higher costs and the potential for compounding errors, and it recommends bounded testing and guardrails (Anthropic 2024).
The management translation: if you can write the steps down, write the steps down. A process whose shape you already know does not benefit from being rediscovered by a model on every run — it just costs more and varies more. Reach for open-ended agency when the path genuinely cannot be specified in advance, not as a default.
Most recurring management work is a workflow wearing an agent's clothes. Your quarterly close has an order. Your onboarding sequence has an order. Encode the order; let the model do the parts inside each step that actually need judgment. That is what a runbook is.
Runbooks — the SOP with checkpoints
A runbook is an ordered list of steps the agent executes one at a time. Each step carries an instruction (what to do), a category — action, review, notify, or wait — and a flag for whether it is a checkpoint. Review steps are checkpoints by default.
A checkpoint pauses the run for explicit human confirmation. The agent surfaces the step, waits, and only advances when you give the go-ahead. Because a run tracks its own cursor and per-step log in the database, a paused runbook survives across sessions — you can approve a checkpoint on Thursday for a run that started on Tuesday. This is what makes a runbook different from a long chat: the state is durable and outside the conversation.
The craft is entirely in where you put the checkpoints, and there is a reliable failure mode in both directions. Too many, and the runbook is a wizard with extra steps — you are doing the work and the agent is doing the clicking, and the team stops running it. Too few, and the checkpoint stops being a decision and becomes a speed bump you tap through, which is the automation-bias trap from the last lesson arriving by a different road.
The heuristic: put a checkpoint exactly where the cost of being wrong changes. Not at every step — at the step after which an error gets expensive or hard to reverse. In a quarterly close, that is before anything is filed, not before each account is read. In client onboarding, that is before the first message reaches the client, not before the internal project is created. A runbook with three well-placed checkpoints beats one with eleven.
Goals — a number, a target, and a threshold
A goal in Kavanah is deliberately narrower than the word suggests. It is a metric source, a comparator, a target value, and a variance threshold. The metric source is one of a fixed set the system can actually evaluate — task completion rate, open task count, overdue task count, tasks completed in the last seven days, a finance number read live from your books, or manual, where you supply the value yourself. The comparator is at least, at most, or exactly. The threshold is the fraction of target inside which the goal reads as at risk rather than off track. A scheduled check evaluates each goal and flags the variance.
The narrowness is the feature. Most stated goals cannot be evaluated by anything — 'improve delivery quality' has no number, so nothing can ever tell you it slipped, which means it is a mood rather than a goal. Being forced to name a metric source and a comparator is the same forcing function the Vision axis applies in KVN, pointed at measurement instead of intent.
The honest limitation, which you should know before you lean on this: the fixed metric sources are mostly about task flow. If the outcome you care about is not expressible as one of them, your options are the manual source — where the agent tracks and reminds but you supply the number — or accepting that this particular goal lives outside the system. Do not contort a real objective into completion rate because completion rate is what is on the menu. That produces a workspace optimizing a proxy, which is the false-friend failure from lesson 5.2 with a dashboard attached.
Schedules — the difference between standing work and nagging
A scheduled run is the agent doing something on a cadence without being asked: a morning digest, a weekly triage sweep, a variance check on your goals.
The rule for what deserves a schedule is narrower than people expect. Schedule a run when the value is in the regularity — when the thing being surfaced is only useful if it arrives before you would have thought to ask. A Monday digest of what slipped over the weekend qualifies: by the time you think to ask, you have already started the week wrong. A monthly report nobody reads does not qualify, no matter how professional it feels to have one.
Everything else should stay on demand. A scheduled run that produces output nobody acts on is worse than nothing: it costs model calls, it trains the team to ignore a channel, and — because a run that fires reliably looks like a system that is working — it can mask the fact that the underlying discipline lapsed months ago.
A practical test before you create one: name the decision the output changes. If you cannot, the run is a habit, not an instrument.
Memory — the part that makes the rest compound
The agent can record durable facts about the workspace: decisions taken and why, commitments made, preferences it should respect, standing context. These persist across conversations, which is what separates an agent from a chatbot that meets you fresh each morning.
The management value is that memory is where a runbook's exceptions accumulate. The first time you run the quarterly close, three things surprise you. If those three things go into memory, the second run starts from eleven months of institutional knowledge rather than from the document. This is the compounding part of agentic management and it is almost entirely un-exploited by most teams, because remembering to record a fact is exactly the kind of thing people do not do.
The discipline: when you override the agent, say why in a sentence it can keep. 'We do not do X for this client because Y.' That sentence is worth more than the override itself, because the override fixes one run and the sentence fixes every future one.
Encode one recurring job
- 1
Pick the management job you have improvised more than three times
Quarterly close, client onboarding, incident postmortem, weekly triage. If you have done it four times and it still feels ad hoc, it is a runbook.
- 2
Ask the agent to create a runbook and write the steps with it
Give it the steps in order. For each, say whether it is an action, a review, a notify, or a wait — and mark as checkpoints only the steps after which an error gets expensive.
- 3
Define one goal with a real metric source
Pick a metric the system can actually evaluate, a comparator, a target, and a warn threshold. If your real objective does not fit the menu, use the manual source rather than substituting a proxy.
- 4
Run the runbook once, end to end, and fix the checkpoints
The first run tells you which checkpoints were theatre and which were load-bearing. Then record what surprised you as a memory, so run two starts smarter.
Standing-work health
- Runbook completion rate
- Fraction of started runbook runs that reach the final step rather than being abandoned mid-run.
- Healthy signal: Our recommended read, not a measured finding: abandonment usually means too many checkpoints, not a lazy team. Look at which step runs die on — that is the checkpoint to remove.
- Checkpoint dwell time
- Median time a run sits paused at a checkpoint waiting for a human.
- Healthy signal: Long dwell means the checkpoint is in the wrong place — it is asking someone unavailable, or asking for a decision they do not feel qualified to make.
- Goal evaluability
- Fraction of your stated goals that are backed by a real metric source rather than the manual source.
- Healthy signal: Not a target to maximise. A high manual share is honest if your objectives genuinely do not fit the menu; it is a warning only if you chose manual to avoid defining the number.
- Scheduled-run action rate
- Fraction of scheduled runs whose output produced a visible action — a task, a reply, a decision — within a week.
- Healthy signal: A run with a near-zero action rate should be deleted, not tuned. It is costing model calls to train your team to ignore a channel.
Key takeaways
- ·If you can write the steps down, write them down. Open-ended agency costs more and varies more; spend it only where the path genuinely cannot be specified.
- ·Put checkpoints where the cost of being wrong changes — not at every step. Three well-placed beats eleven.
- ·A goal is a metric source, a comparator, a target, and a threshold. If it has no number, it is a mood.
- ·Do not bend a real objective to fit the metric menu. That is a false friend with a dashboard.
- ·Schedule only what is useless if it arrives late. Everything else is on demand.
- ·Memory is what makes runbooks compound: when you override the agent, record why in a sentence it can keep.
You now have levels, policies, and standing work. What you do not yet have is a way to tell whether any of it is set correctly — and the answer is not to think harder about it, but to read what your own workspace has already recorded. That is the last lesson.
Sources
- 1.Building Effective AI Agents
Anthropic engineering blog (unbylined) · 2024
The workflow-vs-agent distinction quoted in section one — workflows orchestrate through predefined code paths; agents dynamically direct their own process — along with the warning that agent autonomy means higher costs and compounding errors, and should be bounded by testing and guardrails.
- 2.Ironies of Automation
Lisanne Bainbridge · Automatica 19(6), 775–779 · 1983
Cited here for the checkpoint-placement argument: the human is left with the exceptions the designer could not automate, so the checkpoint belongs where judgment actually is rather than spread evenly across steps.
- 3.Artificial Intelligence Risk Management Framework (AI RMF 1.0)
National Institute of Standards and Technology · NIST AI 100-1, U.S. Department of Commerce · 2023
A voluntary framework whose GOVERN function is explicitly cross-cutting rather than a one-time step, and whose GOVERN 3.2 subcategory asks for policies defining roles and responsibilities for human-AI configurations and oversight — the standards-world version of 'decide where the checkpoints go before you run it'.