Skip to main content

Tool Discovery

Every upstream added to MCP Gateway grows the agent's tool list. Past a few dozen tools, tools/list itself becomes the problem: agents burn context budget on definitions they'll never call in a given conversation, and providers start rationing how many tools they list by default. Tool Discovery keeps tools/list short without hiding anything permanently.

Eager and Cold Tools

options.cache.tools.eager partitions the listing cache's tools into two sets, applied after per-session scope filtering:

north_mcp_proxy:
  type: mcp
  kind: proxy
  options:
    cache:
      store: cache
      ttl: PT5M
      tools:
        eager:
          - type: explicit
            match:
              - everything__echo
              - github__create_pr
              - petstore__list_pets
  • Eager tools matching match stay listed in every tools/list response, same as before Tool Discovery existed.
  • Cold tools, everything else, are omitted from tools/list entirely once a search toolkit is configured, rather than crowding it out or being flagged defer_loading: true.

A toolkit that registers a dozen demo tools can stay eager for the one or two an agent actually reaches for, while the rest stay out of the way until asked for by name or by search.

tools.eager accepts a list, so multiple policies can run in sequence, each narrowing the eager set the next one starts from — a single object, as above, is shorthand for a one-element list.

Ranking-Based Eager Policies

★Zilla Plus explicit requires maintaining a name list by hand. Two additional policies rank candidates instead:
tools:
  eager:
    - type: by-size
      priority:
        - github__create_pr
      tokens: 2000

by-size admits priority-listed tools first, in the given order, then fills the rest of a tokens byte budget in scan order — the first tool that would exceed the budget, and everything after it, is left cold.

tools:
  eager:
    - type: by-usage
      store: tool_usage
      identity: my_guard
      half-life: PT12H

by-usage keeps every tool eager for callers who actually use it: each tools/call boosts the called tool's score, and every score decays toward zero over time, so the eager set tracks recent, frequent calls rather than a fixed list. store persists scores across restarts; an optional identity guard keys them per caller instead of sharing one global ranking. half-life (default PT24H) controls how quickly usage from a past call stops counting.

Search, Describe, Execute

options.cache.tools.search synthesizes three fixed-purpose tools under a toolkit prefix, covering discovery, schema resolution, and invocation without ever requiring a cold tool's full definition up front:

options:
  cache:
    tools:
      search:
        toolkit: zilla
ToolPurpose
zilla__search_toolsKeyword search over every tool the caller is authorized for, cold or eager. Matches are schema-free, name and description only, so scanning many candidates never costs more than a digest.
zilla__describe_toolResolves a tool's full definition by name, the same shape tools/list would show, schema included, still enforcing the per-tool scope a caller would need to see it listed at all.
zilla__execute_toolInvokes a tool by name, through the identical route-resolution and authorization path a direct tools/call would take. Its result is the target tool's own result, passed through unchanged.

Cold does not mean inaccessible. Nothing about eager touches tools/call routing, only what tools/list reports: calling a cold tool directly by name, or through zilla__execute_tool, succeeds identically to calling an eager one.

Tuning the Ranking

The default keyword search ranks on tool name and description alone. search.fields extends ranking to output-schema as well, and search.weights biases which field matters more for a given deployment's tool names:

options:
  cache:
    tools:
      search:
        toolkit: zilla
        limit: 10
        fields: [ name, description, output-schema ]
        weights:
          name: 2
          description: 1

search.limit caps how many matches zilla__search_tools returns by default, further narrowed by any max_results argument in the request itself. search.type: keyword (the default shorthand) and search.index are mutually exclusive: index configures one or more ranking backends explicitly, fusing their results by reciprocal rank when more than one is listed, for deployments that need more than a single BM25 pass over the cache.

Semantic Ranking ★Zilla Plus

Keyword ranking misses a query that shares no token with a tool's name or description ("increment total by adding" for a tool named get-sum). Zilla Plus's binding-mcp-ext module contributes a semantic backend to search.index: it embeds each tool's configured fields at index time and the query text at search time, through a referenced embeddings: entry, then ranks by cosine similarity. Listing keyword and semantic together fuses both rankings by reciprocal rank automatically, no extra configuration beyond index itself:

embeddings:
  my_embeddings:
    type: glove

options:
  cache:
    tools:
      search:
        toolkit: zilla
        index:
          - type: keyword
          - type: semantic
            embedding: my_embeddings

embedding names a top-level embeddings.<name> entry, resolved through zilla's pluggable embedding provider SPI -- see glove for the local, dependency-free option shown above, or ibm-watsonx-ai for a hosted model backed by IBM watsonx.ai.

A query only semantic can resolve still surfaces the right tool; a query with shared keywords still gets the fast, literal keyword match — deployments that configure both get the benefit of each without choosing one over the other.

Editions

ComponentCommunityPlus
Eager/cold partitioning, explicit policy✓—
by-size / by-usage ranking policies—✓
Keyword search (zilla__search_tools, describe, execute)✓—
Semantic search ranking backend—✓

When to Use It

Reach for Tool Discovery once an MCP Gateway deployment aggregates enough toolkits that tools/list itself becomes noisy, commonly a reference server or SDK wrapper that registers many demo or rarely used tools alongside the few an agent actually needs. Search and describe give an agent (or a human driving one) a way to find and inspect the long tail without every upstream's full tool surface being listed by default.

Get Started

Try the example

The mcp.proxy example configures options.cache.tools.eager and options.cache.tools.search on north_mcp_proxy, and includes a tools-list-client helper for calling zilla__search_tools, zilla__describe_tool, and zilla__execute_tool directly.

Try semantic ranking ★Zilla Plus

Extend the same north_mcp_proxy config with type: semantic alongside type: keyword in search.index, backed by a configured embeddings: entry (e.g. glove), to also resolve a paraphrase query that shares no keyword with the target tool's name or description.