Case UW-003 is a submission whose application leaves the roof age empty. The underwriting agent asks for get_roof_assessment on SUB-1003, and the roof-data provider has no record of it either. Inside functions.py, that is an ordinary failure: the function raises RecordNotFound. A Python caller would catch it by type and move on.
The caller here is not Python. It is a model, and a model cannot catch anything. It reads text and decides what to write next. Consider what it could be shown. A crashed loop shows it nothing, and the run ends without a recommendation. A pasted exception shows it something like Traceback (most recent call last): File "functions.py", line 181, in roof_report ... RecordNotFound (illustrative), which names a file, a line and a class, and still does not say whether trying again could help. An empty answer, {}, invites the worst reading of all: that the roof has no problems.
In the example, the model reads this:
{"ok": false, "error": {"code": "not_found", "message": "The roof-data provider has no record for SUB-1003.", "retryable": false}, "meta": {"tool": "get_roof_assessment", "attempt_number": 1, "duration_ms": 180, "evidence_ids": []}}
Every field answers a question the model has. Did it work? No. What kind of failure? not_found, a fact about the world, not a fault in the system. In words? The provider has no record. Should I ask again? No. With that, the model treats the roof age as unknown and recommends request_info under rule R-ROOF-02 of the policy fixture. The model recommends; a person decides, and the underwriter sees a request for the missing roof age instead of an invented one.
The model proposes a call. Your code decides whether it runs, and, as this page shows, your code also decides what the model learns from it. Types made one typed source describe what goes into a tool. This page designs what comes out.
The reader is a model#
A tool result is what the application sends back to the model for one call, matched to the call by its id, as Tool calling showed. Its reader has three properties that a compiler or a calling service does not have.
It reads everything literally and acts on it. A result is not parsed by a type checker; it is read, as text, by a system that will base its next request on what it understood. An ambiguous result gets an ambiguous next step.
It cannot ask a follow-up question of the function. It can only ask the tool again, possibly with different arguments, or give up. So a result must say whether asking again could help, and if the arguments were wrong, which one.
It repeats what it reads. Whatever reaches the model can reach the final answer, the underwriter's screen, a log or another agent. A result is a disclosure.
Before AI. HTTP APIs went through the same lesson. A mature API returns a JSON body with a stable error code and a message written for the client, keeps the status code meaningful (a 404 is not a 500), and never returns a Java stack trace in production. A client library then branches on the code, not on the text. A tool result is that kind of response, with one difference: the client branching on it is a model, so the message must also make sense as plain language.
One envelope for every outcome#
The example answers those needs with a single shape for every result, which this site calls the result envelope. It is this site's design, not a standard, and the example's README is authoritative for its field names. Two forms, success and error, quoted from that README:
{"ok": true, "data": {...}, "meta": {"tool": "get_roof_assessment", "attempt_number": 1, "duration_ms": 180, "evidence_ids": ["EV-ROOF-0412"]}}
{"ok": false, "error": {"code": "timeout", "message": "The roof-data provider did not answer in 2.0 s.", "retryable": true}, "meta": {...}}
The design choice is that the first field settles the question every reader asks first. ok is true or false, never absent. Success puts the function's return value under data. Failure puts an error result under error: a code from a closed list, a message written by the adapter, and retryable. Both carry meta, which records which tool answered, which attempt produced this result, how long the whole call took, and which evidence ids the result contains.
The adapter builds both forms in two small functions, quoted from executor.py:
def envelope_ok(data: dict[str, Any], meta: dict[str, Any]) -> dict[str, Any]:
"""Success. Lifts `resource` and `next_cursor` out of the data into the envelope."""
data = dict(data)
out: dict[str, Any] = {"ok": True}
resource = data.pop("resource", None)
cursor = data.pop("next_cursor", None) if "next_cursor" in data else None
out["data"] = data
if cursor:
out["partial"] = True
out["next_cursor"] = cursor
if resource:
out["resource"] = resource
out["meta"] = meta
return out
def envelope_error(code: str, message: str, meta: dict[str, Any], final: bool = False, **extra: Any) -> dict[str, Any]:
"""Failure. `retryable` tells the model whether asking again, now, may help; `final` forces it false
(the circuit breaker is open: asking again now will fail the same way)."""
assert code in ERROR_CODES, code
retryable = code in RETRYABLE and not final
return {"ok": False, "error": {"code": code, "message": message, "retryable": retryable, **extra}, "meta": meta}
Three decisions are visible in these lines. The error code must be one of nine (ERROR_CODES), so a function cannot invent a tenth. retryable is derived from the code by the adapter, not chosen by each function. And two pieces of a successful result, the resource and the next_cursor, are lifted out of data into the envelope, because they tell the model what to do next rather than what was found.
The fields, now that each has a reason:
| Field | When | Meaning |
|---|---|---|
ok | always | true or false |
data | ok true | The function's return value, without resource and next_cursor |
error.code | ok false | One of invalid_arguments, unknown_tool, not_authorized, not_found, timeout, rate_limited, provider_unavailable, conflict, internal |
error.message | ok false | Written by the adapter per code; never a stack trace, class name or file path |
error.retryable | ok false | Whether asking again, now, may help |
error.violations | invalid_arguments | Each failing path, keyword and message |
error.retry_after_ms | rate_limited | What the provider asked for |
partial, next_cursor | a page of a longer result | true and an opaque cursor such as cur-5 |
resource | a reference instead of content | uri, mime_type, bytes |
meta.tool, meta.attempt_number | always | Which tool; which attempt produced this result (0 if nothing ran) |
meta.duration_ms, meta.evidence_ids | always | Simulated time of the whole call; evidence ids in this result |
meta.idempotent_replay | only when true | The stored result of an earlier attempt of this write was returned |
Diagram: From a function's outcome to the model's next step
Read left to right. The function returns a value or raises one of a small family of exceptions. The adapter is the only place that turns either into an envelope. The model reads the envelope and nothing else; what happens next depends on ok, the code and retryable. Details of a failure go to the trace, for people.
- 1 · Function outcome
- Return value: The function's answer, in Python terms.
- Exception: RecordNotFound, ProviderTimeout and the rest carry a code; any other exception is a bug.
- 2 · Adapter shapes
- Result envelope: One shape for every outcome; the adapter writes every message per code.
- 3 · Model reads
- ok: use data: Continue with the data; ask for the next page only if needed.
- Error: act on code: invalid_arguments: fix the path. not_found: reason with the gap. timeout: continue without it.
- 4 · For people only
- Trace event: The exception type and timing of each attempt are recorded for operators.
The diagram follows one call from the function's outcome to the model's next step. The function returns a value or raises one of a small family of exceptions. The adapter is the only place that turns either into an envelope. The model reads the envelope and nothing else, and what the loop does next depends on ok, the code and retryable.
Errors are data the model can act on#
A function in the example reports a failure by raising an exception; Functions showed that family. The adapter catches each kind and writes its own message. Here is the heart of that translation, quoted from Executor._attempts in executor.py (the excerpt starts at the call to the function and leaves out the timeout's clock arithmetic and the success path):
try:
data = tool.function(world, **arguments) if isinstance(arguments, dict) else tool.function(world, arguments)
breaker.record_success(clock.now_ms)
except fn.ProviderTimeout:
...
error = ("timeout", msg, {})
except fn.ProviderRateLimited as e:
error = ("rate_limited", f"{tool.provider_label} is rate limited; it asks to retry after "
f"{e.retry_after_ms / 1000:.1f} s.", {"retry_after_ms": e.retry_after_ms})
except fn.ProviderUnavailable:
breaker.record_failure(clock.now_ms)
error = ("provider_unavailable", f"{tool.provider_label} is unavailable (it answered with a server "
"error).", {})
except fn.RecordNotFound as e:
breaker.record_success(clock.now_ms) # the provider answered: no record is an answer
error = ("not_found", str(e), {})
except fn.ToolFailure as e:
error = (e.code if e.code in ERROR_CODES else "internal", "The tool failed. The failure was logged.", {})
except Exception as e: # a bug or a bad argument that validation would have caught
tracer.event(a_span, "exception", type=type(e).__name__)
error = ("internal", "The tool failed. The failure was logged for the operators.", {})
Read the comment on RecordNotFound: "the provider answered: no record is an answer". That one line separates two kinds of failure that look alike from outside. A timeout means the system did not answer; a not_found means it did, and the answer was "nothing". The circuit breaker counts the first and not the second, and the model must treat them differently too: a missing record is a fact to reason about, a timeout is a gap to work around.
The script below was written for this page and run in a scratch copy of the example. It sends four calls through the real adapter, each on the world of a named case, and prints the envelope the model would read. A successful run proves that UW-003's missing record and UW-006's timeout produce different codes and different retryable values, that UW-T07's long result arrives as a page with a cursor, and that a bug inside the function reaches the model as a generic internal error while its type is recorded only in the trace. The model here is a scripted stand-in, so these results show how the tool layer works, not how good any model is.
import json
from executor import AdapterOptions, Executor
from functions import World
from run_cases import load_cases
from tracing import Tracer
CASES = {c["id"]: c for c in load_cases()}
def one(case_id, name, arguments, label, options=None):
world = World(CASES[case_id])
tracer = Tracer("run-demo", world.clock)
record = Executor(options=options).execute_detailed(
world, "underwriting-agent", name, arguments, "call_01", tracer, None, "run-demo")
envelope = record.envelope
if envelope.get("ok") and "passages" in envelope["data"]:
# print passage ids instead of the full passages, to keep the line short
envelope = {**envelope, "data": {**envelope["data"],
"passages": [p["passage_id"] for p in envelope["data"]["passages"]]}}
print(f"{label} ({case_id})")
print(json.dumps(envelope))
exceptions = [e for s in tracer.spans for e in s["events"] if e["name"] == "exception"]
if exceptions:
print("trace event for people:", exceptions[0])
print()
one("UW-003", "get_roof_assessment", {"submission_id": "SUB-1003"}, "not found")
one("UW-006", "get_roof_assessment", {"submission_id": "SUB-1006"}, "timeout")
one("UW-T07", "search_guidance", {"query": "roof age wind zone C eligibility office sprinklers", "top_k": 5}, "partial")
one("UW-004", "get_roof_assessment", {"submission_id": "SUB-1004", "detail": "full"},
"internal (validation switched off)", AdapterOptions(validate=False))
cd examples/uw-tools
/opt/homebrew/bin/python3 show_envelopes.py
not found (UW-003)
{"ok": false, "error": {"code": "not_found", "message": "The roof-data provider has no record for SUB-1003.", "retryable": false}, "meta": {"tool": "get_roof_assessment", "attempt_number": 1, "duration_ms": 180, "evidence_ids": []}}
timeout (UW-006)
{"ok": false, "error": {"code": "timeout", "message": "The roof-data provider did not answer in 2.0 s.", "retryable": true}, "meta": {"tool": "get_roof_assessment", "attempt_number": 2, "duration_ms": 4280, "evidence_ids": []}}
partial (UW-T07)
{"ok": true, "data": {"passages": ["P-COMM-01", "P-COMM-02", "P-COMM-03", "P-COMM-04", "P-COMM-05"], "total_matches": 14}, "partial": true, "next_cursor": "cur-5", "meta": {"tool": "search_guidance", "attempt_number": 1, "duration_ms": 240, "evidence_ids": []}}
internal (validation switched off) (UW-004)
{"ok": false, "error": {"code": "internal", "message": "The tool failed. The failure was logged for the operators.", "retryable": false}, "meta": {"tool": "get_roof_assessment", "attempt_number": 1, "duration_ms": 0, "evidence_ids": []}}
trace event for people: {'name': 'exception', 'at_ms': 0, 'type': 'TypeError'}
(For UW-T07 the script prints only the passage ids; the real result carries each passage's text, rule id, policy version, similarity and trust: "untrusted".) Two results deserve a closer look.
The UW-006 timeout says attempt_number 2 and duration_ms 4280. The adapter already retried once, after a backoff, and both attempts timed out on the simulated clock. So retryable: true means something precise: the failure is of a kind that may pass, but the application has spent its own retry budget for this call. The stand-in model reads that, does not ask again in this run, continues without the roof, and recommends request_info with missing set to roof_assessment. How many times to retry, and when, belongs to Reliability.
The last result is a deliberate bug. With validation switched off, an unknown field detail reaches the Python function, which raises TypeError because it has no such parameter. The model reads a code and a sentence; the exception's type appears only in the trace, which is for people. The test test_internal_error_hides_the_cause in test_tools.py checks exactly this, and the run passed:
/opt/homebrew/bin/python3 -m unittest test_tools.ErrorCodeTests.test_internal_error_hides_the_cause -v
test_internal_error_hides_the_cause (test_tools.ErrorCodeTests.test_internal_error_hides_the_cause) ... ok
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
The test serializes the envelope and asserts that none of a list of words appears in it: Traceback, Exception, TypeError, RecordNotFound, ProviderTimeout, .py, and line . It is a small test with a large job. It turns "never leak internals" from a guideline into a check that fails the build.
Retryable: asking again, now, may help#
A retryable error is one where the same call, sent again, could succeed: the provider was slow (timeout), busy (rate_limited) or failing (provider_unavailable). Every other code is not retryable, because the same call would get the same answer: the arguments are still invalid, the tool still does not exist, the caller still lacks the scope, the record is still missing.
The flag is computed by the adapter from the code, with one exception the docstring of envelope_error names. When the circuit breaker for a provider is open, the adapter answers provider_unavailable with retryable: false, because calls to that provider are paused and asking again now will fail the same way. That is the case this site's own UW-T06 shows: after five failures in a row, later calls fail fast until the cool-down ends.
The flag is a hint for the model, not an instruction to the loop. In the example the adapter does its own retries before the model sees anything, so the model normally reads a final answer for the call. The flag then tells the model whether a later request in the same run is reasonable, for example after other work. Two layers that both retry multiply each other's attempts; Reliability treats that interaction in depth.
A page of a result, and a reference instead of a result#
Two more shapes complete the envelope. Both are about size. A model's context is finite, every result is paid for in input on every following turn, and Anthropic's article on writing tools for agents, read on 2026-11-17, recommends pagination, filtering and truncation with sensible defaults for that reason.
A partial result is one page of a longer answer. In this site's own case UW-T07, search_guidance matches 14 passages and the page limit is 5. The envelope above says partial: true and next_cursor: "cur-5", and total_matches in the data says how much more exists. The model asks for the next page, by passing the cursor back as cursor, only if it needs it. Advanced covers pagination in depth.
A resource reference is a pointer to content that is not in the result. get_roof_assessment in UW-004 returns a short summary (roof age 22, EV-ROOF-0412) and, beside it, "resource": {"uri": "uw://roof-assessments/SUB-1004", "mime_type": "application/json", "bytes": 351}. The full report stays behind the uri unless the application decides to attach it. The model gets what it needs to reason; the bulk stays out of the conversation. Advanced covers references, binary content and large results.
The lab below shows each outcome: the envelope, the text the model reads, five contract checks, and what the loop does next. For an error outcome, a switch shows a bad design, with the exception text in the message, and the "no internals" check fails. The table after it holds the same information.
| Outcome | Case | Envelope | What the loop does next |
|---|---|---|---|
| success | UW-004 | ok true, data with decisions and current_policy_version | The model reads the data and continues |
| resource reference | UW-004 | ok true, short summary in data, resource with uri uw://roof-assessments/SUB-1004 | The full report stays behind the uri unless the application attaches it |
| not found | UW-003 | not_found, retryable false | Not an exception: the model treats the roof as unknown and recommends request_info (R-ROOF-02) |
| invalid arguments | UW-T01 | invalid_arguments with violations | The model reads the path, corrects the argument and asks again |
| timeout | UW-006 | timeout, retryable true, attempt 2 | The adapter already retried once; the model continues without the roof and lists it in missing |
| partial | UW-T07 | ok true, partial true, next_cursor cur-5 | The model asks for the next page only if it needs it |
When the result is an exception#
Two library defaults, as documented on 2026-11-17, show how easily internals reach a model. LangChain4j's default tool execution error handler sends the exception message to the model, and its page warns that this may leak sensitive data. Spring AI's default ToolExecutionExceptionProcessor sends the message of any RuntimeException back to the model, and rethrows checked exceptions. Neither default is wrong for a prototype. Both mean that an exception message written for a developer, such as the invented "connection refused to db-replica-3:5432 for user uw_svc", becomes text in the conversation, then possibly text in a recommendation an underwriter reads.
The fix is the one the example uses: a translation point that knows every code, writes every message, and keeps the original exception for the trace. In Spring AI that point is a custom ToolExecutionExceptionProcessor; in LangChain4j it is a custom error handler. In both, the test is the same as test_internal_error_hides_the_cause: provoke a failure and assert what the model does not see.
Production pitfall. "Never leak internals" also covers success. A result that returns a whole database row returns its internal columns too: an internal id, a cost figure, a note written by staff. Shapedatadeliberately, field by field, as the example'sstored_summarydoes for a stored recommendation: three fields, not the row. Anthropic's "Define tools" page, read on 2026-11-17, gives similar advice: return only high-signal information and stable semantic identifiers rather than opaque internal references.
How MCP expresses the same idea#
MCP, the Model Context Protocol, standardizes how a client reaches tools that another program offers. Its revision 2026-07-28, read on 2026-11-17, defines the result of tools/call with three fields that map closely onto the envelope. content is a list of content blocks the client can pass to the model: text, images, audio, a resource_link, or an embedded resource. Structured content is a JSON value in the field structuredContent; when the tool declares an outputSchema, the server MUST return structured results that conform to it and clients SHOULD validate them. And isError, which defaults to false, marks a failed call. The specification adds that, for backwards compatibility, a tool returning structured content SHOULD also return the serialized JSON in a text block, and notes that structuredContent is unrelated to the "structured outputs" feature of model APIs.
The same page separates two error channels. Protocol errors are problems with the request itself: an unknown tool, a malformed request, a server error. They are JSON-RPC errors, such as code -32602 with the message Unknown tool: invalid_tool_name. Tool execution errors are failures the model can learn from: API failures, input validation errors, business logic errors. They are normal results with isError: true, and the specification says clients SHOULD pass them to the model so it can self-correct.
The example's MCP-style server follows that split. It is a teaching subset of MCP revision 2026-07-28, written from the specification; it is not the official SDK and is not conformance tested. The script below was written for this page; it reads the transcript that mcp_client.py recorded in the scratch copy and prints the tools/call exchanges on one line each. A successful run proves that a valid call returns both a text block and a resource_link, that UW-T01's bad argument is a result with isError true carrying the envelope as structuredContent, and that an unknown tool is a JSON-RPC error, not a result.
import json
from pathlib import Path
transcript = json.loads(Path("out/mcp_transcript.json").read_text())
for m in transcript["messages"]:
msg = m["message"]
if m["method"] != "tools/call":
continue
if m["direction"] == "client_to_server":
print(f"request id {msg['id']}: {msg['params']['name']} {json.dumps(msg['params']['arguments'])}")
continue
if "error" in msg:
print(f"response id {msg['id']}: JSON-RPC error {json.dumps(msg['error'])}")
continue
r = msg["result"]
kinds = [c["type"] for c in r["content"]]
sc = r.get("structuredContent", {})
code = sc.get("error", {}).get("code") if isinstance(sc, dict) and not sc.get("ok", True) else None
print(f"response id {msg['id']}: resultType {r.get('resultType')}, isError {r.get('isError')}, "
f"content {kinds}, structuredContent ok={sc.get('ok')}" + (f" code={code}" if code else ""))
for c in r["content"]:
if c["type"] == "resource_link":
print(f" resource_link {json.dumps(c)}")
/opt/homebrew/bin/python3 mcp_client.py > /dev/null
/opt/homebrew/bin/python3 show_mcp_calls.py
request id 3: get_roof_assessment {"submission_id": "SUB-1004"}
response id 3: resultType complete, isError False, content ['text', 'resource_link'], structuredContent ok=True
resource_link {"type": "resource_link", "uri": "uw://roof-assessments/SUB-1004", "name": "SUB-1004", "mimeType": "application/json", "description": "The full roof report"}
request id 4: get_roof_assessment {"submission_id": 1004}
response id 4: resultType complete, isError True, content ['text'], structuredContent ok=False code=invalid_arguments
request id 5: get_roof_report {"submission_id": "SUB-1004"}
response id 5: JSON-RPC error {"code": -32602, "message": "Unknown tool: get_roof_report"}
Diagram: MCP: a tool execution error is a result; an unknown tool is a protocol error
Time runs down the page, over the example's MCP-style server (a teaching subset of revision 2026-07-28). A bad argument comes back as a normal result with isError true, so the model can read it. An unknown tool is a JSON-RPC error, which a client may or may not show the model.
- MCP client (in the host) to MCP server: tools/call get_roof_assessment SUB-1004. A valid call.
- MCP server to MCP client (in the host): result: text + resource_link, isError false. structuredContent holds the envelope; the resource_link points to uw://roof-assessments/SUB-1004.
- MCP client (in the host) to MCP server: tools/call with submission_id 1004. UW-T01's wrong type.
- MCP server to MCP client (in the host): result: isError true, invalid_arguments. A tool execution error: the model reads the path and corrects the call.
- MCP client (in the host) to MCP server: tools/call get_roof_report. UW-T02's unknown tool name.
- MCP server to MCP client (in the host): JSON-RPC error -32602. A protocol error, Unknown tool: get_roof_report; no result object at all.
Note the one place where the example's two front doors differ. Inside the loop, an unknown tool such as UW-T02's get_roof_report is answered with an unknown_tool envelope the model reads. Over MCP, the same request is a protocol error, -32602, as the specification lists it. The specification allows a client to pass protocol errors to the model and notes they are less likely to lead to recovery. A host that wants the model to recover from a wrong name has to turn that JSON-RPC error into something the model reads, which is exactly what the loop's envelope does. MCP architecture covers the protocol's roles and messages.
Provider APIs mark an error in their own way, as documented on 2026-11-17. Anthropic's tool_result block takes an optional "is_error": true, and its documentation advises instructive messages that say what went wrong and what to try next; for an invalid call it says the model typically retries two to three times with corrections. OpenAI's function_call_output carries an output that the guide says is usually a string, such as JSON, text or an error code; the guide read here describes no separate error flag. Gemini's function_result step carries a result list of content blocks; the pages read do not describe a separate error flag either. On those two, the envelope's ok and error.code are the error signal, which is one more reason to put them first. These are the documented shapes; not run here.
What this is not. This page designs the result one tool returns. Typed contracts for data in general, including parsing and repairing what a model writes, belong to Schema Engineering. Recording results and errors as traces and metrics for operators is covered briefly in Advanced; the runtime that decides what the agent does after a failed run belongs to Harness Engineering.
In a coding agent. A coding agent'srun_testsproduces thousands of lines of output. Returned whole, it fills the context and buries the one assertion that failed. A good result is an envelope:okfalse, a code such astests_failed, a shortdatawith the count and the first failing test name and message, and a reference to the full log file that the agent can read withread_fileif it needs more. A missing test command is a different code, not retryable, and says so. The model then fixes the code, not the test runner.
The message field is a channel: who writes it, and what may flow through itAdapter-written messages, echoed arguments, and provider text the model will trust
The example's adapter writes every message per code, with one exception worth noticing. For not_found it uses str(e), the text of the RecordNotFound the function raised: "The roof-data provider has no record for SUB-1003." The function author wrote it, and it echoes an argument the model sent. That is fine here, because the function's messages are reviewed code and submission_id passed a strict pattern before the function ran. It becomes a problem when either condition fails: a message built from a provider's error body, or an argument that was not validated. The provider's text is then untrusted content inside a field the model treats as the application's own voice. Retrieved text gets a trust: "untrusted" marker in the example, as UW-005 shows; an error message has no such marker.
A defensible rule: messages come from a fixed template per code, filled only with values that passed validation (ids, numbers, names from the registry). Anything from outside the application goes into data, marked as untrusted where that matters, or into the trace. Anthropic's handle-tool-calls page, read on 2026-11-17, makes the related point that tool results often carry untrusted content and should stay inside tool_result blocks, not be moved into system prompts or plain user text.
There is also the question of how much to say. "The tool failed" helps nobody fix anything; "the roof-data provider did not answer in 2.0 s" tells the model the failure was outside its control. Aim for the cause at the level the model can act on, and stop there: which dependency, which argument, whether to retry. Never which host, which query, which line.
Mapping the envelope onto MCP and provider fieldsWhich channel carries an error, and why invalid arguments are a result, not a protocol error
The envelope travels inside other protocols, so each of its outcomes needs a mapping. The example's MCP-style server uses this one: every envelope becomes a tools/call result with the envelope as structuredContent, its JSON as a text block, isError equal to not ok, and a resource_link block when there is a resource. Only an unknown tool, a missing _meta and an unsupported version become JSON-RPC errors.
The tempting alternative is to map invalid_arguments to -32602 Invalid params, since JSON-RPC defines that code for exactly this. The MCP specification read today says otherwise: input validation errors are tool execution errors, reported with isError: true, because the model can fix them. A protocol error is for a request the client should never have sent in that form. Keep the line where the specification draws it, and the model gets the violations with their paths instead of a transport error a client may not show it.
On the provider side the mapping is shorter. With Anthropic, set is_error from ok and put the envelope's JSON in content. With OpenAI and Gemini, as documented on 2026-11-17, the pages read name no error flag, so the envelope's JSON is the whole signal. Either way, keep the envelope's shape identical across front doors, so that a contract test can check one shape and the model learns one convention. If a tool declares an MCP outputSchema, that schema should describe the envelope, and a client that follows the SHOULD will validate results before passing them on.
Testing the result contractShape, closed codes, retryable consistency, no internals, size: five checks per tool
The envelope is a contract, so it gets contract tests. The example has the pieces: test_envelope_shape asserts the exact key order of a success (ok, data, resource, meta), the keys of meta, and the keys of an error (code, message, retryable); the error-code tests provoke each of the nine codes; test_internal_error_hides_the_cause and its siblings assert that no internal word appears.
Five checks generalize to any tool. The shape: every result has ok, and exactly one of data and error. The vocabulary: every error.code is in the closed list, so a new failure mode forces a deliberate decision about its code. Consistency: retryable is true only for the retryable codes, and never when the breaker is open. Hygiene: no stack-trace words, file paths, host names or class names, checked on a serialized result. Size: a result stays under a byte budget, and a larger one must be a page or a reference.
Run them on recorded results as well as on fresh ones. A trace replayed from last month is the cheapest way to learn that a refactoring changed a code from not_found to internal, which would quietly change the model's behavior on every UW-003-like case: an unknown roof becomes a system failure, and the recommendation changes with it. Testing builds these into a suite.
Partial results and replays: what "ok" meansok with partial true is complete for this page; a replayed write is ok but did not run
Two envelopes say ok: true while something else is also true. A page with partial: true is a complete answer to the request "give me the first five", and an incomplete answer to "what does the guidance say". The model must not treat the first page as everything: UW-T07's total_matches of 14 is there so that it cannot. A cursor must be opaque (cur-5 means nothing to the model and should not be constructed by it) and stable for as long as the model may reasonably use it; an expired cursor is an error with its own code, not an empty page.
A write answered with meta.idempotent_replay: true is also ok, but the function did not run this time: the adapter returned the stored result of an earlier attempt with the same idempotency key, as in this site's own case UW-T04. For the model the difference rarely matters; the recommendation is stored once, as it should be. For operations it matters a great deal, because a high replay rate means replies are being lost somewhere. That is why the flag is in meta, beside the attempt number, and not hidden. Safety explains idempotency keys in depth.
Checklist for Monday#
| Check | Why |
|---|---|
Every tool returns one envelope shape, with ok first | The model and the tests learn one convention |
Error codes come from a closed list; retryable is derived from the code | A missing record and a timeout need different next steps |
| The adapter writes every error message from a template per code | No stack trace, class name, host or file path reaches the model |
| A test provokes an internal error and asserts the serialized result is clean | test_internal_error_hides_the_cause; library defaults may send exception text |
invalid_arguments lists each failing path | The model fixes the right field in one turn (UW-T01) |
| Long results are pages with an opaque cursor and a total | UW-T07: 14 matches, 5 per page |
Large content is a reference, with a short summary in data | The roof report stays behind uw://roof-assessments/SUB-1004 |
Over MCP, execution errors are results with isError true; over provider APIs, the error flag is set where one exists | The model can only correct what it is shown |
In one breath#
A tool's result is read by a model, so every outcome goes back in one envelope: ok, then either data or an error with a code from a closed list, a message the adapter writes, and whether asking again could help. A long answer comes back as a page with a cursor, and a large one as a reference with a short summary. Exceptions stop at the adapter, which keeps their details for the trace and gives the model a sentence it can act on; a test makes sure of it.
You can now design what comes back from a tool, map it onto MCP and provider fields, and prove it leaks nothing. Each tool now has a clear contract in both directions. What is still open is the choice between tools: with five tools the model rarely picks the wrong one, but with forty, names, descriptions and the size of the catalog decide whether it finds get_roof_assessment at all. That is Discovery.