All chapters
  1. Start hereThe model proposes a call. Your code decides whether it runs
  2. Foundations · 1Functions: arguments, return values and side effects
  3. Foundations · 2Tool calling: how a model asks, and the loop that answers
  4. Foundations · 3Tool schemas: names, descriptions and argument constraints
  5. Foundations · 4Types and annotations: one source for the schema and the handler
  6. Foundations · 5Result contracts: success, error, partial result, reference
  7. Protocols · 6Discovery and selection: how the right tool gets chosen
  8. Protocols · 7MCP architecture: host, client, server, tools, resources, prompts
  9. Protocols · 8MCP on the wire: transports, lifecycle, authorization, notifications
  10. Protocols · 9Retrieval: a tool the model calls, or knowledge it is given
  11. Production · 10Implementing tools: API wrappers, database queries, files, code execution
  12. Production · 11Safety: least privilege, validation, idempotency, sandboxing
  13. Production · 12Reliability: timeouts, retries, rate limits, circuit breakers
  14. Production · 13Testing tools: contracts, stand-ins, faults and traces
  15. Reference · 14Frameworks and standards: what each one does for a tool
  16. Reference · 15Advanced: composition, pagination, streaming, binary content, observability
  17. Reference · 16The insurance lab: one submission, five tools, end to end
  18. ReferenceGlossary
  19. AboutAbout this site and its sources
Reference · 15

Advanced: composition, pagination, streaming, binary content, observability

Tools that call tools, results too large for one answer, results that arrive in pieces, images and files, and the spans and counters that show what a tool really did.

What you will understand
Why one call, one small answer stops being true, and the five shapes a tool takes when it does.
What you can do
Chain tools, page a long result, report progress, return a reference instead of a file, and version a tool without breaking callers.
What you can prove
What a tool really did, from a trace: one span per call, one child span per attempt.

Every page so far has assumed a simple tool. The model writes one request, the application runs one function, and one small result goes back. The model proposes a call; your code decides whether it runs. For the five tools of the insurance example that is mostly true.

Then the underwriting agent meets case UW-T07, one of this site's own cases. It searches the underwriting guidance for "roof age wind zone C eligibility office sprinklers". Fourteen passages match. The tool returns five. The roof report that get_roof_assessment points to is a full document with observations and, in a real system, an aerial image. And case UW-006 shows a call to the roof-data provider that takes 4,221 ms of simulated time before it gives up. During those four seconds the model, the person watching the cockpit and the on-call engineer all see nothing.

None of these is a new kind of tool. Each is a familiar tool meeting a limit: the context window is finite, a result is not always ready at once, a file is not text, and a tool changes after people depend on it. This page takes the limits one at a time, on the same submission SUB-1004, and ends with the record that shows what happened.

Plain English. A result is what a tool sends back. Until now it was small, complete and immediate. This page is about results that are none of those three, and about tools whose work spans several calls.

Five shapes a tool can take#

A pre-AI backend engineer knows all five shapes from HTTP APIs. A REST endpoint returns one object. A list endpoint returns a page and a link to the next page. A report endpoint returns 202 Accepted and a job id. A file endpoint returns a URL to download. A service that calls three other services composes them. The difference here is the reader of the result. A browser or a Java client reads every byte at no cost. A model reads every byte into its context window, the limited amount of text it can see at one time, and each byte crowds out something else.

Diagram: Five shapes of a tool result · from one small answer to a reference

Read left to right. The first column is the simple tool of the earlier pages. Each later column is the same kind of tool meeting one limit: the next call needs this answer, the answer is long, the answer takes time, or the answer is not text. Click a box to see where it appears in the example.

  1. 1 · One call
    1. One result: The model writes one request, the application runs one function, one small result goes back.
  2. 2 · A chain
    1. Composition: The result of one call is used as input to the next call.
  3. 3 · A page
    1. Page + cursor: The tool returns at most five passages, the total count and an opaque cursor for the next page.
  4. 4 · A stream
    1. Progress: The server sends progress notifications while the work runs, then the final result. Described from the MCP specification; not run in the example.
  5. 5 · A reference
    1. Summary + link: The result carries a short summary and an address; the full report or image is read only when needed.
ShapeWhat the model receivesWhere it appears in the example
One callA complete result in one messagecheck_eligibility
A chainSeveral results, one after another, each feeding the next requestroof lookup, then the eligibility check
A pagePart of the result and a cursor to ask for moresearch_guidance in UW-T07
A streamProgress while the work runs, then the resultdescribed from the MCP specification; not run here
A referenceA short summary and an address for the full contentthe roof report uw://roof-assessments/SUB-1004

Composition: when one tool's answer is the next tool's argument#

Start with the first shape that is not one call. The eligibility rules need two roof ages: the one the broker stated and the one the provider assessed. check_eligibility cannot ask the provider itself. It receives a submission and returns an outcome. So the agent first asks for the roof assessment, reads roof_age_years: 22, and then sends the submission to check_eligibility with provider_roof_age_years: 22 added.

That is tool composition: using the result of one tool call as input to another. Here the model composes the tools, turn by turn. It reads the first result and writes the second request.

There are three places the composition can live, and they differ in who is trusted with the order.

Where it livesWho decides the orderExample
In the model, across turnsThe model, one turn at a timeThe agent reads the roof result, then asks for the eligibility check
In your code, behind one toolYour code, fixed in advanceA single assess_submission tool that calls the provider and the rules
In code the model writesThe model writes a program; a sandbox runs itAnthropic documents a feature it calls programmatic tool calling, described below

The first is flexible and expensive: every intermediate result passes through the context window. The second is cheap and rigid: the order is code, and code is tested. The insurance example deliberately keeps the eligibility check separate, because a person reviewing the trace should see the provider figure and the rule outcome as two pieces of evidence. A Spring engineer will recognize the trade-off: an orchestration service that calls three services in fixed order, against a client that calls each service and decides what to do next.

In our insurance agent. In UW-004 the stated roof age is 9 years and the provider says 22. The composition matters because the gap of 13 years is what fires R-ROOF-03 and makes the recommendation escalate. If the agent had called check_eligibility before the roof lookup, the rules would only see 9 years and find nothing wrong.

A composed chain fails in a new way: the second call can be valid and still wrong, because the first result was misread. Validation catches a number where a string belongs. It does not catch the model copying 2 instead of 22. That is why the eligibility check in the example reads the provider figure from the evidence, and the recommendation cites EV-ROOF-0412: a reviewer can compare the number with its source.

What this is not. Choosing what runs next across many agents and steps, with conditions and retries, is the control plane. That belongs to https://orchestrate.sammuthu.com. Composition here means one agent using one tool's answer as the next tool's argument.

Pagination: a long result in pieces#

Back to UW-T07. The search matches 14 passages. Returning all 14 would put a few thousand words of guidance commentary into the model's context, most of it irrelevant. Returning the five best and a way to ask for more lets the model decide whether it needs the rest.

This is pagination: splitting a long result into pages, with a cursor that says where the next page starts. A cursor is an opaque value. The model passes it back unchanged; it does not compute it. In the example the cursor is cur-5, which means "the next passage is at position 5", but the model is not told that and must not rely on it.

Here is the run, in a scratch copy of the example. A successful run shows that the agent read exactly two pages and that the case still matches its expected recommendation.

cd examples/uw-tools
/opt/homebrew/bin/python3 run_cases.py --case UW-T07 --verbose
case     result recommendation calls errors                             rows
UW-T07   PASS   escalate           6 -                                  1
    ok  recommendation: expected escalate, actual escalate
    ok  citations: expected ["R-ROOF-03"], actual ["R-ROOF-03"]
    ok  total_matches: expected 14, actual 14
    ok  pages_read: expected 2, actual 2
    ok  requires_human: expected true, actual true

(Output trimmed to the lines discussed; the full run prints ten checks.) The model here is a scripted stand-in, so these results show how the tool layer works, not how good any model is.

The envelope of the first page carries two fields that a complete result does not have. This is the result envelope, this site's own design for every tool result, from Result contracts:

{"ok": true, "partial": true, "next_cursor": "cur-5",
 "meta": {"tool": "search_guidance", "attempt_number": 1, "duration_ms": 240, "evidence_ids": []}}

The data field is left out above; it holds five passages and total_matches: 14. The stand-in model reads the first page, finds commentary that ranks high but cites no rule, and asks for the next page with "cursor": "cur-5". The second page holds R-ROOF-01, R-ROOF-02 and R-ROOF-03. It stops there, although the second result also says next_cursor: "cur-10". It has what it needs.

Two design choices sit inside this small exchange. First, the tool caps the page size in code: search_guidance never returns more than five passages, whatever top_k the model asks for. A model that asks for 500 gets five. Second, total_matches tells the model how much it is not seeing, so it can decide whether one more page is worth the context.

Common confusion. Pagination of a tool's result is different from pagination of the tool list. The Model Context Protocol also pages tools/list, with a cursor parameter and a nextCursor field, as documented on 2026-11-17. That pages the catalog for the application. This section pages one tool's answer for the model.
In a coding agent. A search_code tool that matches 900 lines does the same thing: it returns the first 50 with file and line, says how many more there are, and accepts a cursor. A coding agent that reads all 900 lines has less room left for the file it was asked to change.

Streaming and progress: a result that is not ready yet#

UW-006 is the roof lookup that times out. The adapter waits 2,000 ms for attempt 1, waits 221 ms, waits 2,000 ms for attempt 2, then gives up after 4,221 ms of simulated time and reports timeout. That is correct behavior, covered on Reliability. The problem here is different: for four seconds, nobody outside the adapter knows anything.

For a slow but healthy tool, the answer is to report progress while the work runs. A streaming result is a result that arrives in pieces over time instead of in one message. The pieces can be progress updates ("2 of 5 sources read") or parts of the content itself.

The Model Context Protocol (MCP, the standard for offering tools to applications; see MCP architecture) describes progress like this, as documented on 2026-11-17 for revision 2026-07-28. The client puts a progressToken in the _meta of its request. The server may then send notifications/progress messages with the same token, a progress value that must increase, an optional total and an optional message. Over the Streamable HTTP transport, the server can answer a request with an event stream that carries these notifications before the final response. The documentation also says that implementations should set timeouts on requests, may reset a timeout when progress arrives, and should still enforce a maximum.

That last rule is the important one for an engineer. Progress keeps a slow call alive. Without a maximum, it also keeps a stuck call alive forever.

Production pitfall. A provider that sends progress every second and never finishes defeats a timeout that resets on progress. Keep two clocks: an idle timeout that progress resets, and a deadline that nothing resets. The example adapter has a 5,000 ms deadline per call for this reason.

Progress is for people and for the application. Whether a model sees partial content while a tool runs depends on the host application; the MCP documentation read here describes the messages, not what a host shows the model. The example does not implement progress or streaming. Its README says so, and this section describes them from the specification only.

Binary content and references: a result that is not text#

The roof-data provider in a real system would return an aerial image. A model can read images, and MCP defines content types for them. As documented on 2026-11-17, a tool result's content array may hold text, image (base64 data and a mimeType), audio, a resource_link (a uri, a name, an optional mimeType) or an embedded resource.

Binary content means data that is not text: an image, a PDF, a sound file. Sending it inline as base64 makes it about a third larger and puts all of it into the conversation, whether the model needs it or not.

The alternative is a resource reference: a short summary in the result, plus an address where the full content can be read if someone needs it. The example does this for the roof report. Here is the tools/call result that the example's MCP-style server sent, captured in out/mcp_transcript.json (message 6, trimmed to the content array):

{"content": [
  {"type": "text", "text": "{\"ok\": true, \"data\": {\"submission_id\": \"SUB-1004\", \"roof_age_years\": 22, \"evidence_id\": \"EV-ROOF-0412\"}}"},
  {"type": "resource_link", "uri": "uw://roof-assessments/SUB-1004", "name": "SUB-1004",
   "mimeType": "application/json", "description": "The full roof report"}
]}

The model reads the summary: 22 years, evidence EV-ROOF-0412. That is all the rules need. The full report, 351 bytes in this example, stays behind the address. When the application or a reviewer wants it, the client asks with resources/read and the URI.

Who decides to read the reference matters. A resource in MCP is application-controlled: the host application chooses what to attach. So the underwriter cockpit can show the full report and the image to the underwriter, while the model works from the summary. The person sees the evidence; the model sees enough to cite it.

Under the hood. The example also puts the full envelope in structuredContent, MCP's field for a machine-readable result, and repeats it as JSON text. The specification asks for that text copy for backwards compatibility, as documented on 2026-11-17.

Versioning a tool: changing it after people depend on it#

The last limit is time. Suppose the insurer's provider adds a second figure, roof_age_confidence, and the team wants get_roof_assessment to return it. Or suppose they decide submission_id should accept the new format SUB-2026-1004.

Tool versioning is changing a tool's definition or behavior in a controlled way so that callers that depend on the old one keep working. Two kinds of change behave very differently:

ChangeSafe for existing callers?Why
Add an optional result fieldYes, usuallyOld readers ignore it; a model may start to use it
Add an optional argumentYes, if the default keeps the old behaviorOld requests still validate
Make an argument requiredNoOld requests now fail validation
Narrow a pattern, for example the submission_id formatNoRequests that passed now fail
Change what a field meansNo, and the worst kindNothing fails; results are silently wrong

A Kafka engineer knows this table as schema compatibility for a topic. The extra difficulty with a model as caller is that the caller learns the tool from its description each time. A model reading a changed description adapts at once, but a recorded trace, a cached catalog or a test fixture still says the old thing. The MCP specification now lets a server attach ttlMs and cacheScope to tools/list results, so a client knows how long it may cache the catalog, as documented on 2026-11-17.

Observability: what did the tool really do?#

The four limits above share one consequence. Once a call can be retried, paged, streamed or composed, the transcript of messages no longer tells the whole story. UW-006 shows one get_roof_assessment request in the transcript and two attempts inside the adapter. The model never saw the second attempt. Neither would an engineer reading the messages.

A trace is the record of one run as a tree of timed steps. Each step is a span: a named piece of work with a start, a duration, a status and attributes. On this site there is one trace per run; each tool call is a span; each attempt is a child span with attempt_number.

Diagram: One run, one trace · the roof lookup of UW-006 as spans

Each row is a span. The tool call is one span; each attempt is a child span with its attempt number. The model only ever saw one request and one timeout result. The trace shows the two attempts and the wait between them. Times are on the simulated clock.

  1. run run-471a: 0 ms · lasts 4627 ms · ok
  2. tool get_roof_assessment: call_01 · lasts 4221 ms · error timeout
  3. attempt 1: 0 ms · lasts 2000 ms · timeout
  4. attempt 2: 2221 ms · lasts 2000 ms · timeout
  5. tool search_guidance: call_02 · 4221 ms · lasts 240 ms · ok

Here is the part of the trace of UW-006 that covers the roof lookup, from out/UW-006/trace.json in a scratch copy of the example. A successful run proves that the two attempts are visible as children of one call, with the wait between them:

/opt/homebrew/bin/python3 run_cases.py --case UW-006
run run-471a                       0 ms   4627 ms  ok      trace_id 83bd43fb7abe969aa315cbaf63e178d4
  tool get_roof_assessment         0 ms   4221 ms  error   call_01  error.code timeout
    event retry_wait            2000 ms             after_attempt 1, wait_ms 221
    attempt 1                      0 ms   2000 ms  error   outcome timeout  breaker.state closed
    attempt 2                   2221 ms   2000 ms  error   outcome timeout  breaker.state closed
  tool search_guidance          4221 ms    240 ms  ok      call_02

(The JSON spans are shown here as an indented list with the same values; times are on the simulated clock.) The trace answers questions the messages cannot: how long the agent waited, how many times the provider was asked, and whether the circuit breaker was involved.

OpenTelemetry, the open standard for traces, metrics and logs, has conventions for exactly this, as documented on 2026-11-17. A tool execution span is named execute_tool {gen_ai.tool.name}, with gen_ai.operation.name set to execute_tool, plus gen_ai.tool.name and, when the model supplied one, gen_ai.tool.call.id. These conventions now live in a separate repository, open-telemetry/semantic-conventions-genai, and are marked as in development. The example uses its own attribute names (tool.name, tool.call_id) and would map them onto those names before export.

What this is not. Logs, metrics and traces as a discipline, with sampling, storage and alerting, belong to https://observe.sammuthu.com. This section only says what a tool call must record so that the discipline has something to work with.
Who may compose: model, code, or model-written codeThe trust boundary moves with each design, and so do testing and cost

Composition by the model across turns keeps every intermediate result in the context window. That is expensive in tokens, but it has one property that matters in regulated work: each step is a separate, validated, authorized tool call with its own span. A reviewer can replay the chain.

Composition in your code behind a single tool is the cheapest and the most testable. Its cost is rigidity. Every new combination is a new tool, and the catalog grows. It also hides the intermediate evidence from the model, which is good for context and bad when the model needs to explain its reasons.

Composition in code the model writes is the third design. Anthropic's article on advanced tool use, read on 2026-11-17, describes a feature it calls programmatic tool calling: the model writes code that calls tools inside a sandbox, and only the final output returns to the context. The article reports its own internal measurements of token savings; this site does not repeat them. The engineering consequences are clear without numbers. Each tool call inside that program must still pass through your execution adapter, with validation and authorization, because the program was written by a model that may have read untrusted text. The sandbox becomes part of your security boundary. And your traces must show the calls made from inside the program, or the trace check of Testing loses its subject.

A rule that holds for all three: a write tool such as record_recommendation is never called from inside an unreviewed chain without the same authorization and idempotency key it would need on its own. Composition changes who orders the calls. It never changes who is allowed to make them.

Cursors, stability and the page the model never asked forOpaque cursors, consistent snapshots, and why total counts are a context budget

An opaque cursor protects you from the model. If the cursor were an offset the model could compute, a model would sooner or later write cursor: "cur-500" and skip what it did not want to read. Make the cursor meaningless outside your server, and reject a cursor you did not issue. The MCP documentation read on 2026-11-17 says an invalid cursor should give error -32602.

Offsets also break when the data changes between pages. If a new passage is indexed between page 1 and page 2, an offset cursor shows one passage twice or skips one. A cursor that encodes a snapshot (an index version, or the last sort key seen) avoids that. For guidance with a policy_version, the version itself is a natural snapshot: a cursor issued under 2026.2 should not silently continue under a later version.

The harder question is what the model should do with pages. A careful agent reads until it has what it needs. A model that always reads every page has turned pagination into a slow full result. A model that never reads page 2 misses R-ROOF-03 in UW-T07. Both are testable: count pages_read in the trace, as the example's case file does, and treat "read every page" and "read only page 1 when the cited rule was on page 2" as failures.

Finally, rank before you page. Pagination does not fix a bad ranking. In UW-T07 the commentary outranks the rules because the stand-in scores by shared words. A real retrieval tool would rank the policy rules higher. Retrieval quality belongs to https://context.sammuthu.com.

Versioning tools that a model readsNew names versus new versions, deprecation windows, and traces that outlive the definition

There are three common ways to change a tool that callers depend on.

Add, never change. New optional fields and arguments only. This works for a long time and fails when a meaning must change.

A new name for a breaking change, for example get_roof_assessment_v2, kept side by side with the old tool for a deprecation window. This is explicit and costs catalog space; with many tools it makes selection harder, which is the problem of Discovery.

A version in the server, not in the tool name: the MCP protocol does this for itself, with dated revisions and a deprecation policy of at least twelve months for a feature, as documented on 2026-11-17. For your own tools, the equivalent is to version the server and let a host choose which server to connect.

Two things outlive any definition and must carry a version: the trace and the stored recommendation. A recommendation recorded under policy_version 2026.2 with a roof figure from a tool that later changed its method is still evidence, but only with the tool version next to it. Record the tool version as a span attribute and in the stored row. Then case UW-007, a past decision under an older policy, has a twin for tools: a past result produced by an older tool.

Contract tests catch the breaking rows of the table above before release. A test that validates last month's recorded requests against today's schema finds the narrowed pattern on the day it is introduced. That test is described on Testing.

What to record on a tool span, and what never to recordAttributes for operations, arguments as a privacy risk, and the trace context across MCP

A useful tool span carries: the tool name and the call id; the caller and the scope that authorized it; the attempt number on each child span; the outcome code from the envelope (timeout, not_authorized); the breaker state; the evidence ids in the result; and for a write, the idempotency key and whether the result was a replay. With those, an engineer can answer the operational questions without opening any payload: which provider is slow, which caller is being refused, how many writes were replays.

Arguments and results are another matter. A submission carries business data; in a real system it may carry personal data. Record the shape (sizes, field names, the violations of a failed validation) by default, and the content only where the data policy allows it. The example records no argument values on its tool spans; the run span carries only the submission_id.

Across an MCP connection the trace should continue. The MCP specification reserves _meta keys traceparent, tracestate and baggage for W3C trace context, and the OpenTelemetry MCP conventions say instrumentations should inject trace context there, both as documented on 2026-11-17. The OpenTelemetry MCP conventions read today still describe sessions and the initialize handshake, which MCP revision 2026-07-28 removed. Expect the two documents to be reconciled; until then, map the attributes that still exist and leave the session attributes empty.

Counters complete the picture: calls per tool and outcome, attempts per call, time per tool, breaker openings, replayed writes. OpenTelemetry documents a histogram gen_ai.execute_tool.duration in seconds for the duration of one tool execution.

Checklist for Monday#

CheckWhy
Every list-like tool caps its page size in code and returns total_matches and a cursorThe model cannot ask for an unbounded result, and knows what it is not seeing
Cursors are opaque and rejected if you did not issue themThe model cannot skip ahead or forge a position
Large or binary content goes behind a reference; the result carries a short summary and the evidence idThe context window holds what the reasoning needs, and the person can still see the full evidence
Slow tools report progress, and a deadline that progress cannot reset still appliesA stuck call cannot run forever
A composed chain cites the evidence of each stepA reviewer can check that the second call used the first result correctly
Breaking changes get a new tool name or a new server version, and old recorded requests are validated against the new schemaCallers fail in a test, not in production
Each tool call is a span; each attempt is a child span with attempt_number; spans record outcomes, not personal dataThe trace shows what the messages hide

In one breath#

A tool stops being one small call when results grow, take time, are not text, or change. Page long results with an opaque cursor and a cap in code; send a reference instead of a large file; report progress but keep a deadline; and version a tool like any public interface. Whatever the shape, record each call as a span and each attempt as a child span, because the messages alone no longer show what happened.

You can now design the less tidy tools that real systems need. What remains is to see all five tools of the example work together, from the broker's email to the underwriter's decision. That is The insurance lab.