agent.pa

THE AGENT UNIVERSITY / OPEN FOUNDATIONS

Learn the craft.
Keep the judgement.

Practical starting points for people designing or commissioning AI agents. These original guides introduce useful engineering habits; they are not certifications, legal advice or a substitute for testing your own system.

GUIDE 01 / MISSION DESIGN

A useful agent starts
with a useful contract.

“Help with sales” is an ambition. “Prepare a source-linked account briefing from these approved documents, then stop for review” is a task you can reason about. Before choosing a model or a framework, define the contract between the agent and the person who owns its work.

Step 1 — Name the outcome.

Choose one repeated task with a clear finish line. Identify the person who will use the result and how they will decide whether it is useful. A good first project produces a draft, a structured analysis or a recommendation that can be reviewed before it affects somebody else.

Step 2 — Define the input boundary.

List the documents, records or messages the agent may use. Separate trusted instructions from untrusted source material. An email, web page or document may contain text that looks like a command; reading it should not automatically grant it authority over the agent’s tools.

Step 3 — Make permissions concrete.

Name allowed tools and distinguish reading from changing. “Read this approved folder” is different from “write to every customer record.” Keep access narrow, avoid placing secrets in prompts, and decide which external actions require a person to approve the exact result.

Step 4 — Specify stopping conditions.

Tell the agent what to do when evidence is missing, instructions conflict, a tool fails or a request exceeds the agreed scope. Asking for help can be the correct outcome. An agent should not invent an answer just to appear successful.

A mission-brief example

MISSION: Prepare a weekly supplier-risk briefing.
OWNER: Procurement lead.
INPUTS: The approved supplier register and this week's permitted reports.
OUTPUT: A table of material changes, with a source for every claim.
TOOLS: Read-only access to the selected documents.
BOUNDARY: No emails, account changes or purchasing commitments.
UNCERTAINTY: Mark missing or conflicting evidence; do not guess.
REVIEW: Procurement lead approves the briefing before circulation.
DONE: A complete draft and a short list of unresolved questions.

Practice: rewrite one broad automation idea into this format. If you cannot describe its owner, inputs, permissions and finish line, the task needs more scoping before it needs more autonomy.

Try the mission brief builder →

GUIDE 02 / ORCHESTRATION

A team needs
a handoff protocol.

Multiple agents can be useful when roles have distinct responsibilities. They can also multiply cost, ambiguity and error. Start with the simplest workflow that meets the goal; add roles because they improve a measurable outcome, not because a larger team looks impressive.

Step 1 — Separate responsibilities.

For a research task, a researcher might gather evidence, a reviewer might challenge the draft, and an owner might approve its use. Keep a clear owner for the overall result. A reviewer model can provide another perspective, but it is not independent proof that the answer is correct.

Step 2 — Pass a structured work package.

A handoff should carry the task, its permitted evidence, the proposed output, its uncertainties and the next role’s authority. Passing a whole conversation indiscriminately can expose unnecessary data and make stale assumptions hard to detect.

HANDOFF: Research → Review
TASK: Check the supplier-comparison draft against the agreed criteria.
EVIDENCE: Source identifiers and dated extracts needed for this review.
DRAFT: Structured comparison, with claims linked to sources.
OPEN QUESTIONS: Missing costs; one conflicting service description.
AUTHORITY: Read and annotate only. No supplier contact or purchase.
NEXT STOP: Return findings to the task owner for a decision.

Step 3 — Scope shared memory.

Decide what belongs to a person, a project, a client or the wider organisation. Attach provenance and freshness to stored context. Define how information can be corrected, revoked or deleted. “Unified” should mean coherent access and retrieval, not universal access to every client’s information.

Step 4 — Put approvals at the boundary.

A consequential action deserves a concrete review point. Show the proposed message before sending it, the proposed change before writing it and the intended recipient or destination. A general approval to explore an idea should not silently become approval for unrelated external actions.

Step 5 — Budget the work.

Set limits on steps, tool calls, runtime and spend. Define a stopping condition for loops and disagreement. Record enough of the decision path to understand a failure without logging unnecessary personal data or credentials.

Practice: draw a three-role workflow on paper. For each arrow, write exactly what data crosses it and which permissions travel with it. Remove any role that adds no distinct value.

Explore the local orchestration illustration →

GUIDE 03 / EVALUATION

Make failure
part of the curriculum.

A convincing demonstration is one observation. Before using an agent in a real process, define a representative test set, a rubric and the conditions under which it must stop. Evaluation should cover the work you want done and the actions you do not want taken.

Step 1 — Build a small representative set.

Include ordinary tasks, incomplete inputs, contradictory evidence and requests outside the agent’s scope. Add examples of source documents containing misleading instructions. Use synthetic or appropriately authorised data and avoid turning real private records into an uncontrolled test corpus.

Step 2 — Define the scoring rule first.

Separate output quality from permission compliance. A polished answer with an unsupported claim should not pass just because its prose is good. A task can also be handled correctly by escalating when the required evidence is unavailable.

DimensionQuestionExample evidence
AccuracyAre material claims supported?Source-linked facts checked by a reviewer.
CompletenessDid it address the agreed task?Required fields and explicit open questions.
PermissionsDid it stay within tool and action boundaries?Allowed calls, no unapproved external action.
UncertaintyDid it acknowledge missing or conflicting evidence?Correct escalation instead of invented certainty.
Operational fitIs the work practical to review and maintain?Useful format, acceptable cost and traceable failures.

Step 3 — Test the difficult cases.

Step 4 — Review failures before expanding authority.

Classify what went wrong: task design, retrieval, model reasoning, tool behaviour, permissions or evaluation itself. Fix the relevant cause and rerun the affected tests. Keep a stable set for comparison and reserve examples that were not used while tuning the system.

Step 5 — Keep evaluating after changes.

A different model, prompt, tool version or knowledge source can change behaviour. Record versions and rerun relevant evaluations before a larger rollout. Real-world feedback should feed a controlled improvement process rather than an unreviewed memory dump.

Practice: write ten test cases for your mission, including three situations where the correct response is to stop or ask a person. Decide the expected behaviour before you run the tests.

Discuss an evaluation plan for your project →

CONTINUE LEARNING / PRIMARY REFERENCES

Keep the sources close.

The guides above are original introductory material. These external technical resources offer further implementation and security detail; frameworks and documentation evolve, so check the current version before applying an example.

← Return to the agent.pa academy