Outstanding Transactions in SoC Design: Principles, AMBA AXI, and Architecture

Outstanding Transactions in SoC Design: A Deep Dive

Latency hiding, pipelined bus throughput, and AMBA AXI architecture

In modern SoC design, outstanding transactions are a decisive factor in system performance. By converting physical memory latency into logical parallelism, this technique underpins the performance architecture of every high-throughput chip — from mobile application processors to data center SoCs.

What Is an Outstanding Transaction?

In SoC interconnect terminology, "outstanding" refers to a master's ability to issue the next transaction before receiving the response to the previous one. The master does not stall; it keeps the address channel busy while data is in flight.

Traditional bus protocols such as early AHB used a blocking request-response model: the master sent one request, waited for the completion signal, and only then proceeded. This is analogous to a checkout counter where the next customer cannot place items on the belt until the cashier finishes ringing up the previous order — serialized and inefficient.

Protocols that support outstanding transactions pipeline the communication channel, hiding latency behind subsequent requests and driving bus utilization toward its theoretical maximum.

Key Terms

Term	Definition
Latency	Time from request issuance to completion. Outstanding transactions mask this latency by overlapping it with subsequent requests.
Throughput	Transactions completed per unit time. Increasing the outstanding count directly raises bus throughput by eliminating idle stall cycles.
ID Tagging	A unique identifier attached to each transaction so the master can match out-of-order responses to their originating requests.
AMBA AXI	Arm's Advanced eXtensible Interface — the industry-standard interconnect protocol that defines the five-channel architecture enabling outstanding transactions in SoC designs.

Why Outstanding Transactions Matter — A Performance Comparison

In a modern SoC, the round-trip from a CPU request to a DDR memory controller and back can take hundreds of clock cycles. Without outstanding support, the master holds the bus idle for that entire duration — burning cycles that could be spent issuing the next request.

Non-Outstanding: Serial Execution

Address A → ⏳ Wait → Data A → Address B → ⏳ Wait → Data B

Outstanding: Pipelined Execution

Address A → Address B → Address C → ⚙ → Data A → Data B → Data C

This is not a theoretical exercise. In practice, raising the outstanding depth from 1 to 4 typically improves effective memory bandwidth utilization by 2–3×. For masters with high-bandwidth demands — GPU command processors, DMA engines, or streaming accelerators — the gain is even more pronounced because their access patterns are inherently latency-tolerant.

The Mechanism: Decoupled Address and Data Channels

Outstanding capability is made possible by separating the address channel from the data channel. While the slave is still fetching data for the first request, the master forwards the second and third addresses. The slave can then apply internal optimizations — pre-fetching, bank interleaving, row-hit reordering — across the queued requests. Think of it as a restaurant kitchen receiving the next table's order while the current dish is still cooking: the kitchen stays busy and overall wait time drops.

AMBA AXI Protocol — Channel Architecture and Data Flow

Five Independent Channels

AXI's defining design choice is five fully independent channels, each with its own VALID/READY handshake. Because no channel depends on another being idle, parallel operation is structural — not a special mode that must be explicitly enabled.

Channel	Name	Direction	Role
AW	Write Address	Master → Slave	Delivers the destination address for a write transaction
W	Write Data	Master → Slave	Carries the actual write payload (with byte-enable strobes)
B	Write Response	Slave → Master	Signals write completion and reports error status (BRESP)
AR	Read Address	Master → Slave	Delivers the source address for a read transaction
R	Read Data	Slave → Master	Returns read data together with response status (RID, RRESP)

Read Transaction Sequence

① Address Issuance (master side)

The master drives ARVALID and waits for ARREADY to accept the read address (ARADDR). Without stalling for the first response, it continues issuing subsequent addresses — as many as the slave's queue depth permits. Each address carries a unique ARID that identifies the transaction for later matching.

② Request Queuing (slave side)

The slave stores incoming addresses in an internal request queue. The depth of this queue directly defines the maximum number of outstanding transactions the slave can handle simultaneously. Sizing this queue is a key microarchitectural decision that balances area against achievable throughput.

③ Data Preparation (slave side)

A memory controller does not necessarily service requests in FIFO order. DDR access time varies by bank and row state, so an internal scheduler reorders requests to minimize row-activation overhead — a technique called bank interleaving. Having multiple outstanding requests in the queue gives the scheduler enough in-flight work to keep DRAM banks continuously busy.

④ Data Transfer (slave → master)

When data is ready, the slave asserts RVALID and waits for RREADY. Each beat carries RID — matching the ARID issued at step ①. The master uses RID to route the data to the correct requester, even when responses arrive out of issue order.

Write Transaction Flow

For writes, the master first issues multiple addresses on the AW channel, then streams the corresponding data bursts on the W channel. The slave returns a completion status on the B channel per transaction. Outstanding writes mean the master can send the next AW and W before the B response arrives for the previous write. One important AXI4 constraint worth noting: WID was removed from the W channel, so write data must arrive in the same order as its corresponding AW addresses. This simplifies slave implementations but requires the master's write engine to maintain strict ordering on that channel.

Slave-Side Processing Logic

A slave that accepts outstanding transactions is not a passive receiver. It must implement several layers of management logic — this is one of the most demanding aspects of SoC interconnect design.

Buffer Management and Back-Pressure

Each slave defines a maximum outstanding count based on its internal queue depth. When the queue fills, the slave de-asserts its READY signal (ARREADY / AWREADY), temporarily blocking further requests from the master. This is the back-pressure mechanism — the fundamental flow-control safety valve that prevents data loss without requiring the master to track the slave's internal state. It operates analogously to a highway on-ramp metering signal: when the main lanes are saturated, new vehicles are held at the ramp until space opens.

Out-of-Order Completion

One of the most powerful aspects of AXI outstanding support is out-of-order completion: the slave may return data for a later request before data for an earlier one, provided both requests carry different IDs.

⚠ The invariant: transactions sharing the same ID must complete in issue order. AXI imposes no ordering constraint across different IDs — that is precisely what lets the scheduler exploit bank-level parallelism.

Concrete example: The master issues a read to a slow peripheral (ID=0) followed by a read to a fast SRAM (ID=1). Because the IDs differ, the slave may return the SRAM data first. The master routes it by RID, and overall system latency drops significantly without any protocol violation.

Reorder Buffer (ROB) on the Master Side

Because the slave can return responses out of order, the master (or interconnect) must include a reorder buffer (ROB) that reassembles responses into the sequence software originally expected. The ROB tracks in-flight IDs and holds early-arriving responses until any missing predecessors arrive. This is especially critical for cache-line fill operations, where the CPU pipeline stalls until every beat of the line is present. Arm's CCI-550 and CMN-700 coherent interconnects implement this logic with considerable sophistication to support hundreds of in-flight transactions across multiple CPU clusters.

Design Trade-offs and Architectural Considerations

Area vs. Performance Trade-off

Increasing the outstanding count improves throughput but directly expands the queue and buffer structures in both the slave and the interconnect — raising silicon area and dynamic power. Mobile APs typically cap outstanding depth at 4–16 to meet tight power budgets; server-class SoCs may support 32 or more. The right number is not a universal constant — it is determined by profiling the target workload and identifying the knee in the throughput-versus-depth curve.

Dependency and Hazard Management

Multiple in-flight transactions can create data hazards. The classic case is a read-after-write (RAW) or write-after-read (WAR) to the same address. If the ordering is not enforced, stale data reaches the consumer. Interconnect and slave logic must include hazard detection that compares in-flight addresses and stalls or flushes conflicting requests. Getting this wrong produces data corruption that is notoriously difficult to reproduce deterministically in simulation.

Evolution to Network-on-Chip (NoC)

In contemporary NoC (network-on-chip) architectures — such as Arm CMN-700 and Synopsys FlexNoC — the outstanding transaction concept extends to packet-based, multi-hop networks. Outstanding management moves from a single bus segment to a full network topology, with credits and virtual channels replacing simple READY signals. The same principles apply to die-to-die interfaces: UCIe (Universal Chiplet Interconnect Express) uses analogous in-flight credit schemes to saturate high-bandwidth links between chiplets. As chiplet-based heterogeneous integration becomes the norm, mastery of outstanding transaction mechanics is increasingly essential.

Core Insight

Outstanding transactions are not simply about “sending faster.” They represent a fundamental architectural technique: converting unavoidable physical latency into exploitable logical parallelism. The three pillars — decoupled address/data channels, ID-based tagging, and managed queuing — work together to push modern SoC performance well beyond what any single-channel, blocking protocol could achieve.

References: Arm AMBA AXI and ACE Protocol Specification (IHI0022E) · Digital Design and Computer Architecture, ARM Edition

S

SoC Design

Semiconductor & SoC Design Notes

Curated and verified notes on semiconductor and SoC design, with an emphasis on correctness before publishing.

Blog

Content is based on publicly available data and referenced sources. Last updated: June 8, 2026