LLM tool schema design explained

Schema structure basics

Most providers accept JSON Schema (draft 2020-12 or a subset) wrapped in a tool definition. Treat each tool as a small API endpoint the model can discover.

A minimal pattern:

{
  "name": "get_order",
  "description": "Fetch order details by ID. Use when the customer asks about status, shipping, or refunds. Do not call if no order ID is known.",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": {
        "type": "string",
        "description": "Order ID exactly as shown in confirmation email, format ORD- followed by 8 digits.",
        "pattern": "^ORD-[0-9]{8}$"
      }
    },
    "required": ["order_id"],
    "additionalProperties": false
  }
}

The pattern is enforced server-side even if the model ignores it; the description teaches the model what valid IDs look like before validation fires.

Parameter description patterns

Models read parameter descriptions at inference time. Vague schemas produce vague calls.

• Name for semantics. refund_reason_code beats reason; amount_cents beats amount.
• Description = example + constraint. “ISO 8601 date in UTC, e.g. 2026-06-11. Must not be in the future.”
• Enum entries documented inline. List each value and when to pick it: damaged_item (product arrived broken), not_as_described (listing mismatch).
• Negative space. “Do not pass customer email here; use lookup_customer_by_email instead.”
• Cross-parameter hints. “Required when refund_type is partial; omit for full refunds.”

Descriptions are prompt tokens. Keep them dense, not essay-length. One to three sentences per property is usually enough if names are good.

Granularity taxonomy

How many tools and how wide each schema should be is the highest-leverage design choice.

Default to atomic reads and writes. Introduce composite tools only when latency dominates and you can validate the full parameter bundle deterministically. Meta tools belong behind guardrails and narrow allowlists, not as the first design.

Validation and coercion pipeline

Never trust model output. Your handler should run a strict pipeline before side effects:

1. Parse JSON. Reject malformed payloads with a structured error the model can read on retry (see tool error handling).
2. JSON Schema validate. Types, enums, patterns, min/max.
3. Business rules. Order exists, user owns it, refund within policy window.
4. Coerce cautiously. Trim strings and parse integers; do not guess missing required fields.
5. Authorize. Map tool args to principal; reject cross-tenant access.
6. Execute idempotently. Dedupe keys on writes.

Return validation errors as observations, not HTTP 500s. The model often self-corrects on the next turn when errors cite the exact field and allowed values.

Harbor Support refactor

Before: one update_order tool with twelve optional parameters covering status changes, notes, refunds, and address edits. The model frequently mixed intents & mdash; setting status: refunded without calling payment APIs.

After:

• Split into get_order, get_shipment_status, check_refund_eligibility, create_refund, add_order_note.
• Added pattern on order_id and enum on refund_reason_code with per-value descriptions.
• Moved free-text customer quotes to a separate customer_message field on a read-only logging tool, not on write schemas.
• Registered tool list filtered by intent routing so refund flows never saw address-edit tools.
• Versioned schemas in a registry tied to prompt versions for A/B tests.

Bad tool call rate: 31% to 4%. Median turns to resolution dropped from 3.8 to 2.4 because the model stopped recovering from self-inflicted validation loops.

Technique decision table

Common pitfalls

• Overloaded tools. One function with many optional params hides required combos from the model.
• Generic parameter names. id, type, data collide across tools in the same context.
• Missing format hints. Models guess date formats; always specify.
• Enum without documentation. Raw ["a","b","c"] forces hallucinated mappings.
• Too many tools in one prompt. Past ~15–20 tools, routing accuracy falls; use intent-based subsets.
• Schema drift without versioning. Changing enums breaks in-flight sessions; version and deprecate.
• Descriptions that document implementation. “Calls Postgres table orders_v2” does not help the model pick arguments.
• No server validation. Trusting model JSON is how refunds hit wrong accounts.

Production checklist

• Every tool has a verb_noun name and a when-to-call description.
• additionalProperties: false on all parameter objects.
• Enums documented per value; patterns on IDs and codes.
• Atomic writes; composite tools justified and tested.
• JSON Schema validation before any side effect.
• Structured validation errors returned as tool observations.
• Tool list filtered by intent or route where catalog exceeds ~15 tools.
• Schemas versioned alongside prompts in a registry.
• Golden eval set of user utterances to expected tool + args.
• Metrics: tool call rate, validation failure rate, retry count per tool.
• Idempotency keys on all mutating tools.
• Runbook for rolling back schema changes under incident.

Key takeaways

• Schemas are prompts. Parameter descriptions teach the model what to pass.
• Atomic tools beat Swiss Army functions. Split overloaded endpoints.
• Validate everything server-side. Models are probabilistic; APIs are not.
• Route tools by intent. Smaller catalogs route more accurately.
• Measure validation failures. They predict production pain before users do.

Related reading

• LLM function calling explained — provider APIs and runtime registration
• LLM structured outputs explained — JSON mode vs tool calls for extraction
• LLM tool error handling explained — validation errors as observations
• LLM output parsing and validation explained — post-model parsing pipelines