Five patterns I took from Google's multi-agent ADK lab

I spent a couple of hours on Google Cloud's two-part ADK hands-on lab, the one where two presenters live-code an "Agent Wars" battle system: three elemental familiars summoned by an orchestrator, fighting a boss across the network. The RPG skin is doing a lot of work to keep a codelab watchable, and it mostly succeeds. But that's not why it's worth two hours.

Underneath the theme it's a tour of five layers — tools, coordination, cross-service calls, cross-cutting hooks, and shared context — and the lab is unusually disciplined about keeping them separate. Most multi-agent material I read collapses at least two of these into "the orchestrator prompt." Here's what each layer is for, and which parts I think survive contact with a stack that isn't Google's.

Everything below is from the lab, not from a system I've run in production. The opinions at the end are mine.

## 1. MCP, and the part I'd have hand-rolled

The framing is the usual one and still the correct one: a model is a strong reasoner with no access to the present. It can't tell you today's exchange rate and it is a genuinely bad calculator. A tool is how you hand it a deterministic capability instead of asking it to guess, and MCP is the standard wrapper — an MCP client on the agent side, a server exposing list_tools / call_tools, and framework-agnosticism as the whole point. Build the tool once, wrap it, and any agent stack can call it. (I've written separately about what makes an MCP server a model will actually use correctly — that's the layer above this one.)

The lab builds three flavors, each deployed as its own Cloud Run service so tools can change without redeploying agents:

• A custom API server — hand-written handlers, each wrapping one HTTP call.
• A general-function server — the same handler pattern around plain local Python functions.
• MCP Toolbox for Databases — no server code at all.

The third one is the reason I'm writing this section. My reflex for "give an agent read access to Postgres" has always been to write a small MCP server around a query layer. The Toolbox says: don't. Declare each tool as a parameterized SQL statement in YAML, point it at a source, deploy the prebuilt image.

sources:
  familiar-db:
    kind: cloud-sql-postgres
    project: <project-id>

tools:
  lookup_ability:
    kind: postgres-sql
    source: familiar-db
    statement: SELECT damage FROM abilities WHERE name = $1

toolsets:
  familiar_tools: [lookup_ability, apply_damage]

It normalizes configuration across engines, so the same mental model covers Cloud SQL today and Spanner later. On the agent side, wiring it up is one line: an MCPToolset pointed at the Toolbox's Cloud Run URL over SSE. And adk run against a single agent gives you a terminal chat, which is the right place to confirm tool calls actually resolve before you build a graph on top of them.

## 2. Workflow agents: coordination as code, not as prose

This is the section I'd hand to anyone whose orchestrator prompt contains the phrase "do these in order."

The loose option is a single coordinator LLM prompted to route to sub-agents. It's flexible and it is non-deterministic: nothing guarantees it calls A before B, or that "do these at the same time" produces actual concurrency rather than two sequential calls and a confident summary. ADK's answer is a small set of workflow agent classes that make coordination structural:

• Sequential — sub-agents run in the given order, always. The fire familiar is a Scout agent (look up base damage via the Toolbox) feeding an Amplify agent. Order matters, so it's a class, not a hope.
• Parallel — sub-agents run concurrently, and the whole thing can nest inside a sequential agent. The water familiar fans out two gathering branches and merges them in a following step. Real concurrency, real latency win.
• Loop — reruns sub-agents up to max_iterations, or until one escalates to break out early.

The loop is the one with the most reach beyond the demo. The lab uses it for "charge until a threshold is met," but the same shape is the standard producer-critique loop: one agent drafts, a second critiques, the first revises, repeat until the critique passes or the cap trips. That's essay drafting, code review, schema refinement — anything where the quality bar is easier to check than to hit first try.

The moment a prompt says "do these in order," that's the signal to stop prompting and start composing. Sequence is a control-flow problem wearing a prompt's clothing.

— the actual takeaway from part one

And they compose: parallel inside sequential, sequential inside loop. You build the shape of the pipeline out of primitives instead of describing the shape in English and hoping.

## 3. A2A: what happens when agents get their own deploys

Local sub-agents work right up until everything doesn't live in one process. The lab's second half pushes each familiar onto its own Cloud Run service, at which point the orchestrator needs a standard way to find and call agents it doesn't share a runtime with. That's A2A — an open wire protocol for agent-to-agent calls, so an ADK agent, a CrewAI agent, and a LangGraph agent can discover and invoke each other identically, wherever they're deployed.

The mechanism is the agent card: a JSON manifest auto-published at /.well-known/agent.json, generated from the same name, description, and instructions you'd give any agent. It lists skills, capabilities, and the security scheme for who's allowed to call it.

# server side — expose a local agent over A2A
from google.adk.a2a import to_a2a
app = to_a2a(root_agent)         # + uvicorn.run(app, port=8080)

# client side — consume it from another agent
from google.adk.a2a import RemoteA2aAgent
fire_familiar = RemoteA2aAgent(name="fire_familiar", url=FIRE_URL)
#   the .well-known path resolves automatically — pass the base URL

orchestrator = LlmAgent(
    name="summoner", model="gemini-2.5-flash",
    instruction="...",
    sub_agents=[fire_familiar, water_familiar, earth_familiar],
)

Once wrapped, a remote familiar slots into sub_agents exactly like a local one. The orchestrator still picks by matching the situation against each agent's description — it's just doing that over agents living on other services.

Which is the detail I keep coming back to. The description isn't documentation anymore; it's the interface contract another agent reasons over when deciding whom to delegate to. A2A formalizes something a lot of teams already do badly over ad hoc REST, and being forced to publish a capability manifest at a well-known path is good discipline even for agents that never leave your own infrastructure.

A2A and MCP are not competitors. MCP is agent → tool: stateless, call it, get a result. A2A is agent → agent: stateful delegation, with built-in support for polling long-running responses. Production systems use both, at different layers.

## 4. Callbacks and plugins — the most portable idea here

Every agent run has a lifecycle: before the agent starts, before and after each model call, before and after each tool call, after the agent finishes. ADK exposes a hook at every one of those points — before_agent_callback, before_model_callback, after_tool_callback, and friends — so you can inject logic without touching the agent's prompt or control flow.

The lab's example is a cooldown guard: a before_agent_callback on the earth familiar that reads a last-invoked timestamp out of state and, inside a 60-second window, short-circuits the turn — "exhausted, must recover" — before the model runs at all. Call it early and it visibly blocks; wait it out and it proceeds. Small, but it's the same throttle you want in front of any user-triggered action prone to spam.

Then the good part: callbacks are per-agent, but plugins are the global version. Same callback logic, registered once on the ADK Runner, applied to every agent under that runtime — including remote A2A agents, since the runner still mediates those calls. The lab promotes its earth-only cooldown into a CooldownPlugin on the runner and all three familiars start enforcing it, with zero edits to any agent definition.

The security case is the identical mechanism pointed at a different problem. Model Armor — Google's input/output safety-filtering API — wires in as a before_agent_callback to scan for prompt injection before the model sees input, and/or an after_agent_callback to catch leakage on the way out. Rate limiting and content safety turn out to be the same seam.

This is the piece I'd port first, and it doesn't require ADK at all. A clean interception point for throttling, logging, and safety that never touches agent logic is just good architecture, and most homegrown agent loops I've seen don't have one.

## 5. State vs. memory, which are not the same word

Two terms used almost interchangeably in agent writing, and ADK draws a line I found genuinely clarifying:

State is a structured key/value scratchpad scoped to a run. Explicit facts you deliberately stash — last_summon = "fire_familiar" — flowing tool → agent, agent → agent, and agent → tool through a callback_context / tool_context handle.

Memory is the semantic record of a conversation or relationship over time. Closer to a running summary than a dictionary. Storage spans plain in-memory (gone on restart) through a database up to Vertex AI Memory Bank, which persists long enough to retrieve prior context across sessions rather than only within one.

The state example has the nicest hidden detail in the whole lab. The summoner gets an after_tool_callback that writes last_summon whenever a familiar is invoked — and the tool it hooks is one nobody wrote:

def save_last_summon_after_tool(tool, tool_context, ...):
    if tool.name == "transfer_to_agent":
        tool_context.state["last_summon"] = tool_context.args["agent_name"]

summoner = LlmAgent(
    name="summoner", ...,
    after_tool_callback=save_last_summon_after_tool,
)

A hierarchical coordinator gets an implicit transfer_to_agent tool per sub-agent under the hood. Delegating to a sub-agent — remote or not — is a tool call as far as the callback system is concerned. So the seam for "persist which agent was last delegated to" already exists, for a tool that never appears in your code. That's the kind of thing you only learn by watching someone build it.

## What I'd actually take

Stacked bottom to top it's one architecture: tools get wrapped into specialized workflow agents, which get tied together by an orchestrator over A2A once they're separate services, with callbacks and plugins as the cross-cutting seam and state/memory as the shared nervous system. The battle theme is scaffolding. The layering isn't.

Ranked by how likely I am to use it:

1. Callbacks and plugins, immediately. Framework-independent, and the cheapest way to get guardrails that don't live in a prompt.
2. Workflow agents over coordinator prompts. Wherever determinism is actually required. The producer-critique loop is the reusable one.
3. Check for a declarative toolbox before hand-rolling a database MCP server. Cheap check, and I'd have skipped it.
4. Agent cards as a forcing function, even without adopting A2A wholesale. Writing the capability manifest sharpens the description either way.
5. Not memory. Explicit key/value state answers "does B know what A just did" most of the time. Semantic memory is for genuine cross-session recall, and reaching for it early buys you an eventual-consistency problem you didn't need.

The one thing the lab can't teach, and doesn't pretend to, is where the seams should go in your system — which work is a tool, which is an agent, and which is a function call that never should have involved a model. That decision is still yours, and it's the one that matters most.