Observing AWS Lambda durable functions with the Durable Execution Telemetry Exporter

retained text

AWS Compute Blog
Observing AWS Lambda durable functions with the Durable Execution Telemetry Exporter
A durable function is an AWS Lambda function that might run for minutes, days, or even up to a year. You build these functions with the Durable Execution SDK, which provides operations such as steps, waits, and parallel. A single durable execution might be spread across many Lambda invocations, with waits and retries in between. So observing a durable execution, made of many invocations, needs different tools. The question that you want answered is rarely “what did this invocation do?” It’s “how far did my durable execution get, which step failed, how many times did it retry, and how long did the whole thing take?”
After you run thousands of executions, the questions widen from one execution to all of them. Some questions are operational. Which step fails most often, and with which error? Which step causes the most retries? Did a step get slower after last week’s deployment? Other questions are about the business the workflow runs. Which customers did last night’s payment failures affect? How long does each manual approval step take? Of the orders that started, how many reached the payment step, and where did the rest stop? Answering these questions needs per execution telemetry data that includes relevant data such as status, timing, steps, and the input and output fields that you choose to emit.
Traditional Lambda observability works well for a single invocation. Lambda writes a log stream per invocation, so a durable execution that suspends four times writes to four separate log streams, interleaved with every other concurrent execution. Amazon CloudWatch metrics tell you the function’s error rate, but not which of your workflow’s steps is responsible. Distributed tracing, such as AWS X-Ray or OpenTelemetry, shows the timing of one execution. It isn’t built to answer questions across many executions, such as which step failed most often this week. For that, you need one queryable record per execution.
Today we’re introducing the Durable Execution Telemetry Exporter, an observability plugin for AWS Lambda durable functions. We’re also introducing the Durable Execution Telemetry MCP server, a Model Context Protocol (MCP) server that lets an AI coding agent query the telemetry data. The exporter is open source for TypeScript, Python, Java, and Go, in each language’s Durable Execution SDK repository. AWS provides the MCP server as open source in the aws-durable-execution-sdk-js repository.
With a one-line configuration change, you get a record of each execution, with every step, wait, callback, and invoke, including timing, retry attempts, status, and errors. The exporter delivers it to a destination you choose. There’s no custom instrumentation to write and no separate agent to run.
How the durable execution telemetry exporter works
The Durable Execution Telemetry Exporter is a plugin for the durable execution SDK. It runs inside your Lambda function, in the same process as your handler, and subscribes to the SDK’s instrumentation lifecycle. It doesn’t run in your request path.
On every invocation of a durable execution, the plugin builds a cumulative snapshot of that execution, rather than a delta or per-invocation view. Because the durable execution SDK replays your handler from the beginning on each resume and the SDK passes the checkpointed state to the plugin, the plugin can reconstruct the whole execution so far from any invocation. That snapshot is a single JSON document called a DurableExecutionTelemetryRecord:
Each record is complete on its own. One row in a table is one durable execution, which means the questions you ask (slowest step, most-retried step, executions that failed after the payment step) become ordinary queries instead of requiring you to correlate logs across streams.
Getting started
Install the plugin alongside the durable execution SDK:
Then pass it to withDurableExecution:
That is the entire integration. With no configuration, the default LambdaLogExporter writes records as JSON to your function’s own CloudWatch log group. You don’t need to create any infrastructure or grant additional AWS Identity and Access Management (IAM) permissions, because your function already writes its own logs. You can query the result immediately with Amazon CloudWatch Logs Insights:
Choosing where records go
The plugin includes 13 destination exporters so you can send records where they are useful:
The plugin sends records to all configured exporters in parallel. This allows you to route data to different stores based on their characteristics, such as using a fast store for real-time lookups and a less expensive, high-capacity store for long-term storage:
With the S3 exporter, the plugin writes records under Hive-style partitions (year=YYYY/month=MM/day=DD/) that an AWS Glue crawler discovers automatically, so Athena queries work without additional modeling:
Controlling frequency and content
Three settings govern what a production deployment costs and exposes.
emitMode determines when the plugin sends a record. The default, on-complete, emits once when an execution reaches SUCCEEDED or FAILED. Use on-failure for the lowest overhead when you only care about errors, or on-change when you want to watch executions progress in near real time.
samplingRate reduces frequency. The plugin derives the sampling decision from a hash of the execution ARN. The execution ARN is stable across replays, so the decision is deterministic: a sampled-in execution emits all of its records, and a sampled-out execution emits none. You never receive a half-recorded workflow.
content controls what leaves your function. By default, the plugin includes execution input and output as-is, which is convenient in development and often unacceptable in production. Pass a transform to redact:
Each destination supports different payload sizes: CloudWatch Logs events at 256 KB, DynamoDB items at 400 KB. Each exporter has a maxRecordSizeBytes, and when a record exceeds it the plugin truncates for one destination without affecting other destinations: operation results first, then whole operations oldest-first, then execution input and output last. The plugin never drops identity and timeline fields, and it marks a trimmed record truncated: true, so a truncated record is always distinguishable from a complete one. We recommend using content transforms to bound payloads deliberately rather than relying on truncation.
Querying execution history from an AI agent
Answering “why did last night’s batch fail” requires knowing which data store you configured, which query dialect it uses, and what the record schema looks like. DynamoDB PartiQL, Athena’s Trino SQL, Aurora PostgreSQL, Amazon Redshift, OpenSearch, and CloudWatch Logs Insights all differ.
The Durable Execution Telemetry MCP server removes that requirement. It’s a local stdio server that exposes your configured destination to any MCP-capable agent, including Kiro, Claude Code, and Cursor. Add it with one command:
Configuration is environment-only and credentials come from the standard AWS SDK provider chain, so there’s no state file and no secret. The server exposes five tools:
- test_destination runs read-only connectivity checks. If a required variable is unset, it names the exact variable and returns without making any AWS call.
- describe_schema returns the record schema, query dialect, and row cap for whichever destination is configured. Agents should call this before writing a query.
- list_executions filters by status, function name, and time range with no query language involved. This handles most questions.
- get_execution fetches one record by execution ARN.
- query handles questions the filters cannot express.
You ask a question and the agent picks the tools:
        Which steps in order-processor failed most often in the last 24 hours, and what was the average retry count?
On the SQL destinations, write each query as a SELECT statement, or as a WITH statement that ends in a SELECT. These are the only forms the server runs. It checks a query before it makes any AWS call, so a statement that would change data never reaches your destination. On the CloudWatch Logs destinations, the Logs Insights query language has no statements that write data. So every query through the server is read-only.
Best practices and considerations
Choose operationDetail deliberately. The default, top-level, records only top-level operations and produces the same set of operations no matter when a record is emitted. full-tree includes children of contexts such as map items and parallel branches, but by default the durable execution backend prunes finished contexts’ children from the state passed to later invocations. To keep the full tree across a suspend and resume, also set pluginsConfig.childOperationsDepth on withDurableExecution. That preservation has a cost: it forces a replay pass over each preserved context on resume, so enable only the depth you need.
Choose operation names that your destination can query. The plugin uses names verbatim as keys and identifiers and never sanitizes them. A . in a name is a field-path separator in CloudWatch Logs Insights and OpenSearch. Letters, digits, -, and _ are portable everywhere.
Understand what the plugin cannot see. The Durable Execution Telemetry Exporter runs inside your function, so it can only emit records when your function is invoked. Terminal states that originate in the Lambda service without an invocation never reach the plugin: STOPPED from a manual stop, and TIMED_OUT when the execution timeout is exceeded. For complete lifecycle coverage, subscribe to Lambda durable execution lifecycle events on the default Amazon EventBridge bus and update your destination from a small handler:
Account for the cost of the destination. The plugin adds at most one export each time it emits a record, because it sends only the latest of any updates that pile up. Each record is a complete snapshot, and the latest supersedes all earlier ones. If updates arrive faster than an exporter can handle, the plugin drops the intermediate snapshots. The charges that matter are the ones your destination incurs: CloudWatch Logs ingestion, DynamoDB writes, Aurora capacity, and so on. Combine samplingRate and emitMode to control them.
Conclusion
The Durable Execution Telemetry Exporter gives AWS Lambda durable functions an observability model that matches how they run: one complete record per execution, sent to a destination that fits your team. The Durable Execution Telemetry MCP server lets an AI agent answer questions against that data directly.
To get started with durable functions, see the AWS Lambda Developer Guide. For the plugin’s full exporter reference, configuration options, and an AWS Cloud Development Kit (AWS CDK) stack that deploys every destination for testing, see the package README.

document fetched 2026-10-29 03:01 · canonical https://aws.amazon.com/blogs/compute/observing-aws-lambda-durable-functions-with-the-durable-execution-telemetry-exporter