tool-use
Tool use — the third component of the agent anatomy: equipping the LLM to call external APIs for what its weights don’t contain — current information, code execution, proprietary data sources, actions in the world (Weng 2023). “Equipping LLMs with external tools can significantly extend the model capabilities.”
The load-bearing insight, from the MRKL arithmetic experiment: knowing when and how to call a tool is the hard part, and it’s bounded by the LLM’s capability — not by the tool. This is the seed of the modern ACI design discipline.
The lineage Weng surveys
| System | Year | Idea |
|---|---|---|
| MRKL (Karpas et al.) | 2022 | ”Modular Reasoning, Knowledge and Language” — a neuro-symbolic router; the LLM dispatches queries to expert modules (neural or symbolic: calculator, currency converter, weather API). Experiment: a 7B model failed to reliably extract arguments for basic arithmetic — verbal math is harder than explicit math. |
| TALM / Toolformer (Parisi et al. / Schick et al.) | 2022–23 | Fine-tune the LM to use APIs; expand the training set only when an API-call annotation actually improves the output (self-supervised tool learning). |
| ChatGPT Plugins / function calling (OpenAI) | 2023 | The productized form — tools provided by third parties (Plugins) or self-defined (function calls). The pattern this wiki’s 2026 sources take as baseline. |
| HuggingGPT (Shen et al.) | 2023 | ChatGPT as a controller orchestrating other models — see below. |
| API-Bank (Li et al.) | 2023 | A benchmark for tool-augmented LLMs — see below. |
HuggingGPT — the LLM as controller
A four-stage loop where ChatGPT plans and routes work to HuggingFace models:
- Task planning — parse the user request into tasks, each with
task type, id, dependencies, args(few-shot prompted). Dependencies let one task consume another’s output. - Model selection — frame model choice as a multiple-choice question over a (filtered) candidate list; the LLM outputs a model id + reason in strict JSON.
- Task execution — expert models run and log results.
- Response generation — the LLM summarizes execution results for the user.
Weng flags three real-world frictions: efficiency (many LLM rounds + model calls), reliance on a long context window, and stability of LLM outputs + external services. This is an early, explicit instance of orchestrator-workers.
API-Bank — evaluating tool use at three levels
A benchmark: 53 tools, 264 dialogues, 568 API calls. Because there are many APIs, the agent first searches an API search engine, then reads docs to make the call (the pseudocode loops: summarize → search → if found, gen_api_call → execute → refine). Three difficulty levels:
- Level 1 — call: given an API description, decide whether to call it, call it correctly, and respond to its return.
- Level 2 — retrieve: search for candidate APIs and learn to use them from documentation.
- Level 3 — plan: for unclear requests (book a trip, schedule a meeting), chain multiple calls. (Maps onto task-decomposition.)
API-Bank is one of the wiki’s earliest agent-evaluation references — decision-by-decision scoring of a tool-use trajectory.
The 2026 descendants
- agent-computer-interface (ACI) — Weng’s “knowing when & how” turned into a design discipline: tools designed for the agent’s goals (not raw API endpoints), descriptions that say when to use them, structured errors. The modern answer to the MRKL argument-extraction failure.
- model-context-protocol (MCP) — the standardized plug for external systems into agent harnesses; the productized successor to ad-hoc Plugins.
- agent-evaluation — API-Bank’s level structure prefigures runtime tool-use evals.
Cross-link
- 2026-06-04-llm-powered-autonomous-agents — the source.
- llm-agent — the anatomy this sits inside.
- agent-computer-interface, model-context-protocol — engineered descendants.
- agent-evaluation — where tool-use benchmarks live.
- multi-agent-orchestration — HuggingGPT as an early orchestrator-workers system.