AgentCore Gateway's new fallback chain keeps your agent up when a model is throttled — and tells the caller nothing about which model answered. Here is what to instrument before you switch it on.
Last Updated: November 13, 2026
AgentCore Gateway's model fallback is an availability feature with a quality side effect: the gateway serves the request from a different model and, by its own documentation, adds no caller-facing indicator of which destination was used. Your agent stays up; your output distribution changes; nothing in the response tells you. Separately, the new Step Functions connector lets a probabilistic agent trigger deterministic workflows that previously owned their own trigger conditions — which moves the blast radius from the model layer to the money-moving layer. Both are worth adopting. Neither is safe to adopt blind.
What actually shipped, and what it replaces
AWS published its October recap for Bedrock AgentCore and Strands on November 11, 2026. Four items in it change how a production agent stack is wired, and the release notes carry more detail than the recap does.
AgentCore CLI reached version 1.0 — a rebuild with direct resource command groups for harness, runtime, memory, identity, gateway and payment, plus a thin CDK app that replaces generated CDK code and turns on CloudWatch Transaction Search by default. v1 recognises v0 projects and hands their deployments back to the v0 CLI, but any command that mutates a project requires the v1 format, so agentcore migrate is not optional once you want to add a resource.
AgentCore Gateway gained two things: an ordered model fallback chain on inference routing rules, and a built-in AWS Step Functions connector that exposes existing Standard and Express state machines as MCP tools. Bedrock billing gained a product-attribute dimension in Cost Explorer. And on the Strands side, Bidi Agents (speech-to-speech) went GA while Strands Box, a local sandbox combining OS isolation with per-tool semantic policy, entered developer preview.
targetFallback entries
Source: AWS, Model routing and fallback, 2026
Fallback changes behaviour, not just availability
The mechanics are clean. A gateway rule selects traffic with a matchModels condition, and the routeToTarget action carries a fallback block with an ordered targetFallback list, a triggerOn set, a firstByteTimeoutMilliseconds value and a circuit breaker that is enabled by default. The gateway walks the chain, attempts each destination once, and returns the last error if everything fails.
"fallback": {
"circuitBreaker": {"mode": "ENABLED"},
"triggerOn": {
"statusCodes": ["5XX", "429"],
"errors": ["CONNECTION_ERROR", "FIRST_BYTE_TIMEOUT", "UNROUTABLE"]
},
"firstByteTimeoutMilliseconds": 5000,
"targetFallback": [
{"targetName": "openai"},
{"targetName": "bedrock-mantle", "model": "gpt-5.4"}
]
}Read the second targetFallback entry again. The first hop is the same model on a different provider — a true availability substitution. The second hop is a different model. AWS's own documentation calls this degrading, and it is the line where an availability feature becomes a behaviour change. A smaller or older model has different tool-call formatting habits, different JSON schema adherence, different refusal thresholds and a different calibration on whatever confidence heuristic your downstream code keys off. The request succeeds. The HTTP status is 200. The evaluation harness that only watches for errors sees nothing.
Three pieces of fine print matter more than the headline:
The caller cannot tell. From the client's perspective fallback is transparent — one request, one response, no indicator of which destination served it. The routing decision is recorded in the routing logs the gateway vends, which attribute the serving destination and note when the circuit breaker skipped one. So the audit trail exists, but it lives in logs you have to join against, not in the response your application already handles.
Streaming breaks the guarantee. Once a streaming response has started, the gateway can no longer fall back; the caller gets the partial response plus an error and has to retry. If your voice or chat surface streams — and with Strands Bidi Agents now GA, more of them will — fallback protects you only up to first byte.
The chain is short. One to two fallback entries, every destination must be a distinct target-and-model combination, and triggerOn accepts 1 to 10 status codes and at most 3 connection-level error types. This is not a general-purpose router with arbitrary depth. It is a two-hop safety net, and designing a four-tier cost-optimisation ladder on top of it will not validate.
Treating the fallback chain as a retry policy
A retry re-runs the same computation and should produce the same class of result. A fallback to a different model re-runs a different computation. Teams port their existing retry config across — same trigger codes, same alarm thresholds — and inherit an error-rate dashboard that now hides a quality regression. Error rate stays flat during a throttling event precisely because fallback worked.
The instrumentation you need before you enable a chain
The good news is AWS shipped most of the gating primitives in the same window. AgentCore Evaluations added custom multi-dimension evaluators — one LLM-as-a-judge call that rates a single target across several named dimensions, each with its own instruction and scale, usable in on-demand, online and batch runs but not in A/B tests or recommendations. It also added Builtin.FailureAnalysis, which clusters root causes across sessions and returns them in insightResults. Those are the right tools for the job; they just are not wired to fallback for you.
Gating a fallback chain
Run your full suite against hop two and hop three before they are ever reachable. A destination you have not evaluated is not a fallback, it is an untested deploy with a trigger condition.
Emit the served destination onto your own span attributes so a fallback-served session is queryable in the same place as latency and tool errors. The gateway will not do this for you at the response layer.
Score fallback-served sessions separately. A 2% aggregate quality dip during an outage window is invisible; a 30% dip on the 6% of traffic that fell back is actionable.
Structured outputs are now available for all Claude models on Bedrock, but a cross-provider hop may not offer the same guarantee. Schema-validate on your side of the gateway, not on the model's promise.
For some workloads the honest answer is: fail the request rather than answer it with a weaker model. Encode that by giving the rule a chain of same-model-different-provider hops only, and no degrade hop at all.
Step five is the one most teams skip. Fallback is a default-on availability bias, and availability is not always the right trade. An agent summarising a support thread should degrade. An agent producing a figure that flows into a filing should not.
Step Functions as tools inverts who owns the trigger
The connector is genuinely well designed. You create a target with connectorId: "states", choose which of ten operations become tools, and pin a stateMachineArn to the pinnable ones. The gateway generates the schemas, authenticates with the gateway service role, strips pinned parameters out of the advertised schema, and rejects any call that tries to set them. Four operations are pinnable: StartExecution, StartSyncExecution, ListExecutions and DescribeStateMachine. Standard workflows record each step in execution history and can run for up to a year, so work survives the agent's session.
The inversion is the point and the risk. Previously your refund workflow was triggered by an event you controlled — a queue message, a schedule, an API call behind your own authorisation. Now a model decides when it runs and what input it gets. The state machine still enforces how the work happens, which is exactly why this is better than asking a prompt to follow a procedure. But the whether moved to the probabilistic layer.
Two specifics from the docs deserve a security review slot:
The six unpinned operations are bounded only by the gateway service role's IAM policy. You can scope DescribeExecution, StopExecution and GetExecutionHistory to specific execution ARNs. You cannot scope SendTaskSuccess, SendTaskFailure or SendTaskHeartbeat — AWS states these have no resource-level permissions, so you grant them on "*", and the task token becomes the only control. Any agent holding a valid token can complete that task.
And StartExecution exposes only input by default; name sits in the optional-to-expose column. Step Functions uses the execution name for deduplication, so if you do not expose it you have handed the agent a tool with no idempotency key — an agent that retries after a timeout starts a second execution. That is an inference from how the two services compose rather than something AWS calls out, and it is the first thing I would test on a non-critical workflow.
Pinning one state machine and calling it scoped
Pinning hides stateMachineArn from the agent for four operations. It does nothing for the six that identify their target by execution ARN or task token, and nothing for the service role's reach. Teams read "the agent can't change it" and stop there.
What belongs in AgentCore and what stays in n8n
The question we get most often is whether a managed agent gateway retires the external automation layer. It does not, and the split is fairly clean once you separate who decides from what executes. n8n's own MCP story runs the other direction too — its MCP Server Trigger node lets a single workflow act as its own MCP server, and the MCP Client Tool node lets a workflow call external servers — so these stacks compose rather than compete.
| Job to be done | AgentCore Gateway + Step Functions | n8n / Make | Where I'd put it |
|---|---|---|---|
| Model routing and provider failover | Native: rules, chains, circuit breaker, cross-dialect translation | Hand-rolled per node or per credential | AgentCore |
| Money-moving or compliance-bound sequences | Step Functions enforces the order; agent chooses the moment | Deterministic, but trigger logic lives in the same canvas as the AI node | Step Functions, invoked narrowly |
| Scheduled and event-driven back-office automation with no agent in the loop | Possible, but you are paying for an agent runtime to run a cron | Purpose-built; cheaper and faster to change | External orchestrator |
| Long-tail SaaS connectors (CRM, billing, ticketing, sheets) | Build OpenAPI or MCP targets yourself | Hundreds of maintained integrations | External orchestrator, surfaced as one MCP tool |
| Human-in-the-loop approval mid-process | .waitForTaskToken callbacks, Standard workflows only | Wait nodes and forms, easier to iterate | Depends on audit requirement |
| Per-session identity, token vaulting, policy enforcement | Identity and Policy are first-class, included with Runtime or Gateway | Credential store, not per-agent-session identity | AgentCore |
The pattern that holds up: AgentCore owns identity, routing and the agent loop; Step Functions owns anything irreversible; the external orchestrator owns breadth and the things that should never have needed a model. We write up this layering in more detail across our integration work and case studies.
Cost attribution when spend splits across tiers
Fallback splits a single logical workload across two or three price points, which makes per-model visibility a prerequisite rather than a nice-to-have. As of October 8, 2026, Cost Explorer, Budgets and Cost Management Dashboards support a Bedrock product-attribute dimension that breaks spend down by model, model provider, inference type (input versus output tokens) and feature. You can group and filter in Explorer and Dashboards, filter budgets by the same attributes, and combine them with cost allocation tags on application inference profiles or IAM principal tags to see which application or role drives spend on each model. It is available at no additional charge outside GovCloud and the China regions.
What it does not give you is a fallback flag. Cost Explorer will show that 8% of last month's tokens went to your secondary provider; it will not tell you whether that was deliberate routing or an outage. The join you need is product attributes on the billing side against gateway routing logs on the behaviour side, keyed by time window.
On the gateway's own charges, keep the magnitudes straight. Each outbound attempt is billed as a Gateway API invocation at $0.005 per 1,000, so a request that walks all three destinations costs you three invocations — about fifteen thousandths of a cent. That is noise. The real cost of an unplanned fallback is the token price of whatever answered, plus the failed primary attempt's latency. The circuit breaker is the lever that matters: when open it skips the failing primary entirely and AWS does not bill the skipped attempt. Turning it off is a deliberate choice to keep paying failed attempts on a cheap primary rather than fail fast onto a costlier secondary — a reasonable trade, but make it on purpose. Our model coverage notes track where the per-token deltas currently sit.
The runtime layer is now a procurement question
This landed in a crowded fortnight. Google released llm-d 1.0 at the PyTorch Conference on October 20, 2026 — a Kubernetes-native distributed inference stack co-founded with Red Hat and IBM, built by more than 500 contributors across 25 organisations — and on November 5 donated Agent Substrate, gVisor and OpenRL to the CNCF while taking Agent Substrate to 1.0 and GA on GKE. Agent Substrate snapshots idle agent sessions to storage and restores them in under 500 milliseconds, claiming 10x the density of standard container runtimes.
So the choice in front of platform teams this quarter is real: a managed, opinionated, deeply AWS-integrated agent runtime versus an open, CNCF-governed stack you operate yourself. The honest framing is not which is better. It is which lock-in you prefer. AgentCore's gateway rules, pinned connectors and routing logs are AWS-shaped artefacts; they do not port. The Kubernetes stack ports, and you pay for that in operations headcount.
The one piece that does travel is the protocol boundary. AgentCore Runtime now supports MCP version 2026-07-28, and Gateway supports the MCP Tasks extension so a long-running tool can return a task ID instead of holding a connection open. If you keep your tools behind MCP and your evals outside the vendor's harness, you keep the option to move. That is the architectural discipline worth enforcing now, before either stack is load-bearing. If you want a second pair of eyes on where that boundary should sit in your stack, start here.
Frequently Asked Questions
Does AgentCore Gateway tell the client which model actually answered?
No. AWS documents fallback as transparent to the caller: one request, one response, and no caller-facing indicator of which destination served it. The serving destination is recorded in the routing logs the gateway vends, along with circuit-breaker skips, so the information exists — you just have to join logs to traces yourself rather than read it off the response.
How many models can one fallback chain contain?
A primary plus one to two targetFallback entries, so three destinations maximum. Every destination must be a distinct target-and-model combination or the gateway rejects the rule. Trigger conditions allow 1 to 10 HTTP status codes or class patterns in the 4xx and 5xx ranges, plus up to 3 connection-level errors: CONNECTION_ERROR, FIRST_BYTE_TIMEOUT and UNROUTABLE.
Does model fallback protect streaming responses?
Only until the first byte. Once a streaming response has started, the gateway cannot fall back; the caller receives the partial response plus an error and must retry. The firstByteTimeoutMilliseconds setting exists precisely to catch a provider that stalls before streaming begins. For voice and chat surfaces, keep client-side retry logic even with a chain configured.
Is it safe to expose Step Functions workflows as agent tools?
With scoping, yes — but read the IAM section first. Four operations can be pinned to a state machine the agent never sees. The other six are bounded only by the gateway service role, and SendTaskSuccess, SendTaskFailure and SendTaskHeartbeat have no resource-level permissions, so they must be granted on "*". The task token is the only control there. Start with a reversible workflow.
Does AgentCore replace n8n or Make?
Not for most stacks. AgentCore is strong where identity, model routing and the agent loop live; external orchestrators are strong on connector breadth and on automation that needs no model at all. The composition that works is to keep deterministic, scheduled, no-agent work in the external tool and surface it to the agent as a single MCP tool, rather than rebuilding integrations inside the agent.
How do I budget against spend that splits across fallback tiers?
Use the Bedrock product-attribute dimension that reached Cost Explorer, Budgets and Dashboards in October 2026. It breaks cost down by model, provider, inference type and feature, and combines with cost allocation tags on application inference profiles and IAM principal tags. It will not label spend as fallback-induced, so pair it with gateway routing logs over the same time window.
Should we migrate to AgentCore CLI 1.0 now?
If you expect to add resources to an existing project, yes — v1 hands v0 deployments back to the v0 CLI, but mutating commands like add and remove require the v1 project format. Run agentcore migrate for a read-only plan first; it checks git and CDK, refactors the CloudFormation stack per deployment target, keeps v0 physical resource names, and can resume an interrupted migration.
Research digest
AI Research Briefing
Honest insights on AI agents, Small Language Models, and local RAG. No hype. Only when we have something worth sending.
- No hype, just measurable outcomes
- Read by 2,400+ engineers
- Unsubscribe anytime
