loading
loading
Announcing Spanner queues: Transactional messaging for agentic workloads and beyond Nitin Sagar Product Manager Matthew Mucklo Engineering Manager & Lead Cloud Spanner Data Streaming AI agents don't just answer queries — they can autonomously issue refunds, manage inventory, execute multi-step handoffs, and orchestrate sub-agents, to name but a few complex agentic workflows. That frequently requires the agents to maintain internal state in an operational database while dispatching asynchronous actions through a separate messaging or event queue system. And unfortunately for the teams building these applications, managing two systems with disjointed commit points destroys transactional consistency in agentic systems. Today, we are excited to announce the general availability of Spanner queues: native transactional messaging embedded directly within Spanner. Designed specifically for reliable agentic execution, with Spanner queues, creating a message is simply another write in your transaction. An agent's state change and intended downstream actions commit together atomically or fail completely. Compare this to traditional approaches: When a database state update succeeds, but the action dispatch fails, your AI agent decides to act but doesn’t execute. If the message dispatch succeeds but the state transaction rolls back, your agent executes an action based on an invalid state. In asynchronous multi-agent coordination, retries, speculation, and race conditions amplify these failures, forcing developers to build complex outbox patterns, idempotency layers, and reconciliation workers — a heavy reliability tax on agentic architecture. Key capabilities for agentic architectures Spanner queues introduces several core capabilities to support these complex asynchronous workflows without introducing infrastructure overhead. Atomic decide-and-act enqueue. Within a single Spanner read-write transaction, agents can update internal memory or state tables and enqueue tasks to peer agents simultaneously. Backed by Spanner's strict serializability and global external consistency, state changes and execution intent commit as a single atomic unit. Scheduled execution and delays. Queue messages can be dispatched immediately upon commit or scheduled for future delivery. Agentic patterns such as delayed retries, scheduled agent check-ins, or SLA escalation timers can be enqueued transactionally alongside memory updates without requiring external cron schedulers or polling infrastructure. Streaming SQL pull for agent workers. Autonomous agents consume tasks dynamically using streaming SQL reads. Agent runtimes stream incoming tasks, process them as capacity becomes available, and acknowledge task completion within a transaction to guarantee end-to-end task execution state reliability. Episodic memory persistence and handoffs. Agent memory updates — including long-term episodic summaries, reflective state transitions, and context handoffs across sub-agents — can be persisted asynchronously and transactionally via queues. This guarantees that an agent's internal memory remains fully synchronized with its execution history without blocking real-time interactive turns. Why Spanner queues are essential for autonomous agents Transactional exactly-once agent execution: When an agent evaluates tool call results, state modification, reasoning persistence, and downstream tool invocation tasks are saved in one transaction. We guarantee at-least-once delivery and at-most-once ACK, enabling you to achieve exactly-once processing. Robust multi-agent orchestration and handoffs: In multi-agent systems (A2A), handing off state from a primary agent to a specialist agent is represented as a durable, transactionally committed message. Specialist agent workers receive deliverable messages while maintaining a fully auditable lineage of agent interactions. First-class timeouts and human-in-the-loop workflows: Agent workflows frequently require pausing for human approvals or scheduled follow-ups. Spanner queues handles timeout management natively: A single transaction records the pending approval state and schedules an automated escalation message, resolving whichever triggers first. SQL-native observability for agent queues: Inspecting in-flight agent workloads, monitoring task backlogs, or auditing agent execution history can be accomplished with standard SQL queries over queue tables, avoiding opaque message-store black boxes. Beyond agentic workflows Beyond AI agents, Spanner queues serves as a flexible, multi-purpose messaging platform across a variety of traditional event-driven architectures. Whether powering real-time activity feeds in social applications, delivering live updates in news publishing, orchestrating order processing and inventory workflows in retail, or handling high-throughput asynchronous task processing and transaction notifications in financial services, Spanner queues provides a robust foundation for asynchronous message delivery and transactional event-driven workflows within your primary database. Under the hood: Transactional mechanics of Spanner queues Because Spanner queues are represented as first-class relational structures in Spanner, you define, inspect, and manage queues using familiar GoogleSQL. 1. Defining a queue and enqueuing atomically When an agent decides to approve a customer refund, it updates the Orders table and dispatches an execution task to the OrderAgentTasks queue within a single ACID transaction: This ensures that the EXECUTE_REFUND task exists if and only if the order status successfully transitioned to REFUND_APPROVED. 2. Temporal scheduling and atomic cancellation For workflows that depend on time — such as waiting up to 72 hours for a manager's approval before escalating — agents populate the system DeliverTime column to defer message visibility: If the manager approves the request after four hours, your application doesn't have to deal with phantom escalation alerts firing days later. In a single transaction, you update the order status and cancel the pending escalation task using a standard SQL DELETE: 3. Streaming consumption, lease renewal, and atomic acknowledgment Downstream agent workers consume tasks using the RECEIVE_<QueueName> table-valued function (TVF) over a streaming SQL connection (ExecuteStreamingSql). Spanner automatically manages message leases, returning a unique SpannerLeaseToken and expiration timestamp with each leased task: Because AI agent tasks often involve multi-turn LLM reasoning or external API calls that take longer than default lease windows, workers can actively extend their lease using the RENEWLEASE_<QueueName> function: When the agent finishes executing its external tool (passing TaskId as the external API's idempotency key), it opens a read-write transaction to record the final state and acknowledge the message by deleting it with ASSERT_ROWS_MODIFIED 1: Using ASSERT_ROWS_MODIFIED 1 protects your system against lease-expiration races. If a worker stalled due to a network pause and its lease expired, another worker may have already processed and deleted the task. When the stalled worker resumes and attempts to execute the statement, ASSERT_ROWS_MODIFIED 1 detects that the queue row is already gone and throws a statement-level error. Catching that error and aborting that transaction prevents stale workers from overwriting newer database state. Spanner change streams vs. Spanner queues Spanner change streams capture database data changes (inserts, updates, and deletes) in near real-time for downstream integration and auditing. While both change streams and queues allow applications to react to data changes, Spanner change streams are designed for continuous change data capture (CDC) and data streaming to downstream analytics or storage. In contrast, Spanner queues are explicitly designed for transactional task orchestration, supporting native message leases, scheduled deliveries, SQL-based pulling, and atomic acknowledgments within read-write transactions. Get started Spanner provides a unified foundation for agentic data, combining relational, hybrid search, graph, and key-value capabilities under strict global consistency. Spanner queues completes the agentic loop by enabling agents to transition seamlessly from reasoning over data to executing transactional actions within a single unified platform. Spanner queues are now generally available. Sign up for the Spanner 90-day free trial and read up our public documentation to start building resilient, exactly-once agentic workloads by creating a queue table in your Spanner database today.
document fetched 2026-10-03 02:54 · canonical https://cloud.google.com/blog/products/databases/spanner-queues-provide-native-transactional-messaging