Autonomy levels on the board
- L1SuggestRecommends. A person does the work.
- L2DraftPrepares the work. A person sends it.
- L3Act with approvalActs once a person approves the step.
- L4Act within limitsActs alone inside set limits, escalates outside them.
An illustrative mission board for “Your company”. Six missions sit in four columns: queued, agent working, needs approval and done. Each card shows the agent, the person who owns the work, its autonomy level, spend against a budget and progress through its steps. One billing dispute waits for the Service lead to approve a credit over the limit.
Capability 03 / 10Where AI earns its place
Decide where AI fits. Then put agents to work on it.
AI Strategy & Agents decides where AI fits in your business, what to build first and how to measure it, then builds the custom agents and copilots that take on real work, with guardrails, evaluations and a person in the loop.
01Where AI stalls
Most AI pilots stall between the demo and daily use.
The model is rarely the problem. Pilots stop because nobody owns the outcome, the data is not ready, the risk cannot be explained or the running cost was never worked out. Strategy is deciding those four things before building.
Pilot funnel · per 100 ideas
Illustrative, not a statistic- Ideas raisedEvery team has a list 100
- 01 Stalls on: No owner, no measure −58
- Prototypes builtA demo on sample data 42
- 02 Stalls on: Data not ready −24
- Pilots with real usersReal data, a handful of people 18
- 03 Stalls on: Risk unclear, so legal says no −10
- Approved for rolloutOwner, risk and cost signed off 8
- 04 Stalls on: Unit cost unknown −3
- In daily useMeasured every week 5
Decided before any build starts
- A named owner for the work
- A baseline and a target measure
- Data readiness, scored
- A risk tier and an autonomy level
- A cost-per-task ceiling
-
01
No owner, no measure
The demo impressed, but nobody owns the process it changes or the number it should move.
How it is closedEvery use case in the portfolio carries a named owner, a baseline and a target before a line of code is written.
The portfolio -
02
Data not ready
The prototype ran on a clean sample. Production data is scattered, stale or locked in a system with no API.
How it is closedData readiness is scored per use case in the workshop, and every gap goes on the roadmap with a cost and an owner.
The workshop -
03
Risk unclear, so legal says no
Nobody can say what the agent may touch, who approves its actions or which rules apply, so the safe answer is no.
How it is closedA risk tier, an autonomy level, approval points and an audit log are designed in, mapped to NIST AI RMF and the EU AI Act.
Governance -
04
Unit cost unknown
The pilot bill was small. At full volume nobody knows the cost per task, or whether it beats the manual process.
How it is closedCost per task is an eval metric from day ten, compared with the manual baseline and right-sized model by model.
The eval harness
02Use-case portfolio
Twelve ideas. Three worth building first.
Every candidate is scored on value, feasibility, risk and time to value with the people who run the work. The output is a ranked roadmap and a business case for the first three, not a slide of possibilities.
Value
Hours returned or revenue affected, from volumes and handling times the teams supply.
Feasibility
Data readiness and integration effort: where the data lives, how clean it is, whether there is an API.
Risk
EU AI Act tier, personal data involved and the harm if the agent is wrong.
Time to value
Weeks until a measured pilot, including the integration and data work it needs.
A chart of twelve use cases by value and feasibility. The three in the high-value, high-feasibility corner are marked to build first: reconcile vendor invoices, triage inbound RFPs and draft renewal quotes. The ranked list that follows holds the same information.
Minimal Limited · transparency duties High risk · Annex III
Bubble size = hours of work a monthIllustrative portfolio
What the portfolio hands over
- Ranked roadmapAll twelve, sequenced by quarter with dependencies
- Business case × 3Baseline, target, build and run cost, payback
- Owners and measuresOne accountable owner and one number per use case
Selected · rank 01 · score 7.8
Reconcile vendor invoices
Finance · owner Finance lead · 4–6 wks to a measured pilot
- Value
- 360 hours a month of three-way matching across about 2,900 invoices.
- Feasibility
- Invoices and purchase orders already sit in the ERP. About 18% arrive as PDFs and need extraction.
- Risk Minimal risk
- Minimal tier. Supplier data only, and payment release stays with a person.
DecisionBuild firstL4 · Act within limits
03Try it · the autonomy dial
You choose how much the agent may do. Every step is traced.
Pick a task, set the autonomy level and the limits, then run the agent. The same plan runs every time; the level decides what happens at the write. Push a limit below the agent’s proposal and watch it escalate instead of acting.
Interactive demonstration of an agent run. Choose one of three tasks, choose an autonomy level from suggest to act within limits, adjust the spend, record and approval limits, then press Run. The trace lists each step with its tool, input, output, latency, tokens and cost. Depending on the level, the run ends with a suggestion, an editable draft, an approval request, an action taken within limits, an escalation or a halted run, and each run adds a row to the audit log. Outcomes are pre-written for illustration; no live model is called.
run_7f3a21 · renewals-agent · prompt renewals v14
Renew a customer contract
- Latency
- 3.9 s
- Tokens
- 4.6k → 800
- Records
- 3
- Spend
- $0.020
Run complete · approved by Sales ops lead · quote sent · audit row written
-
Plan
plannerrouter-smallinGoal: renew account 20-418 before 14 Oct
out4 sub-tasks · tools kb, crm, calc
640 ms1.2k → 180$0.0003
-
Retrieve policy
kb.searchreadembedin“renewal discount policy”
outPricing policy v4 §3.2 · discounts up to the approval limit
410 ms40 → 0$0.0000
-
Read system
crm.get_accountreadinaccount_id = 20-418
outARR $48,000 · 3 seats added · usage +18%
280 ms——
-
Calculate
calc.renewalinARR, seats, 24-month term
outPrice $51,840 · discount 8%
35 ms——
-
Draft
llm.draftreasoner-largeinQuote and cover email in the account’s tone
outQuote Q-7731 · email of 146 words
2100 ms3.4k → 620$0.020
-
Policy check
policy.checkindiscount, records, spend vs limits
outDiscount 8% ≤ 10% · records 3 ≤ 25 · spend $0.020 ≤ $0.25 · cites §3.2
90 ms——
-
Write action
crm.update_quotewriteinQ-7731 → status “sent”
outApproved by Sales ops lead · Q-7731 sent
320 ms——
Ready
Press Run agent to start. Change a limit first to see a guard step in.
Suggestion · no write tools at L1
Renew account 20-418 for 24 months at $51,840 with an 8% discount. Sources: pricing policy v4 §3.2 and CRM usage.
Sent to Sales ops lead. A person does the work.
Draft saved · crm.save_draft
Editable. Nothing leaves until Sales ops lead sends it.
Approval needed · Sales ops lead
Send quote Q-7731 to the customer?
Evidence: pricing policy v4 §3.2 · CRM usage +18% · 3 records affected
Executed after approval
Quote Q-7731 sent. Approved by Sales ops lead in 14 s.
Run stopped by a guard
The run passed its limit and stopped before acting.
Handed to Sales ops lead with the trace so far.
Audit log · append-only
| Time | Run | Task | Level | Models | Prompt | Tools · scope | Records | Spend | Outcome | Approver |
|---|---|---|---|---|---|---|---|---|---|---|
| 10:42:07 | run_7f3a21 | Renew a customer contract | L3 | router-small · reasoner-large | renewals v14 | kb.search:r · crm.get_account:r · crm.update_quote:w | 3 | $0.020 | Executed after approval | Sales ops lead |
Tool allow-list
The agent can only call tools granted to its identity, per level.
Read and write scopes
Reads are broad; writes are narrow, named and short-lived.
Spend and record limits
A run stops when it passes its budget or touches too many records.
Policy check, cited
Every write is checked against the policy it quotes.
Human approval
Consequential actions wait for a named person.
Full audit log
Model, prompt version, tool inputs, spend and approver, per run.
LLM06 · Excessive Agency. The controls above are the OWASP mitigations made concrete: minimal tools, minimal permissions, human approval for high-impact actions and complete mediation of every call. Prompt injection (LLM01) is tested against the same run in the eval suite.
04Agent anatomy · the stack
What a production agent is actually made of.
A demo is a prompt and a model. A production agent is a loop with eight parts around it, each one replaceable, tested and watched. These are the technologies we work with for each part, chosen per client rather than by habit.
-
Models
planChosen per step: a small model to route, a larger one only where reasoning pays for itself.
- OpenAI
-
Orchestration
planThe agent as a graph with checkpoints, retries and runs that resume after a failure.
-
Memory & knowledge
planSession state in Redis; knowledge in a vector index that keeps a link to every source.
- pgvector
- LlamaIndex
-
Identity & permissions
actEach agent has its own identity, with short-lived, least-privilege credentials.
A diagram of an agent loop: plan, act, observe, repeated until the task is done. Around it sit eight parts: models, orchestration, memory and knowledge, and identity and permissions on one side; tools and APIs, guardrails, evals and observability on the other. Each part lists the technologies used for it.
- Planrouter-small picks the next tool from the six on its allow-list
- Actcrm.get_account called through an MCP server · read scope · agent identity
- Observeresult checked, trace span written, cost added, next step decided
-
Tools & APIs
actMCP servers wrap your systems once, so any agent can use them with scoped permissions.
- Salesforce
-
Guardrails
actInput and output filters, tool allow-lists and spend limits enforced outside the model.
-
Evals
observeGolden sets and calibrated LLM-as-judge scoring on every change, gated in CI.
-
Observability
observeEvery step traced: latency, tokens, cost and tool inputs, in the tools your SREs use.
Model-agnostic by design
A model swap is a change to one route, re-run through the eval suite before it ships.
Your data stays yours
Enterprise API terms or self-hosted open-weight models. Nothing trains on your data.
Region where required
Inference and storage pinned to the region your policy or the DPDP Act and GDPR require.
Let’s build what happens next.
Tell us what you’re building. We’ll answer straight.