For over three decades, modern CPUs have relied heavily on speculative execution as a means to enhance performance. This architectural technique, which emerged in the 1990s, was celebrated as a significant advancement in microarchitecture, akin to earlier innovations such as pipelining and superscalar execution. By predicting the outcomes of branches and memory loads, processors could maintain full pipelines, thereby avoiding stalls and keeping execution units busy. However, this reliance on speculation has not come without its drawbacks.
The costs associated with speculative execution are becoming increasingly apparent, particularly as workloads in artificial intelligence (AI) and machine learning (ML) grow in complexity and demand. The inefficiencies inherent in speculative execution manifest as wasted energy when predictions fail, increased architectural complexity, and vulnerabilities that have led to high-profile security exploits like Spectre and Meltdown. These challenges have paved the way for a new architectural paradigm: deterministic, time-based execution models.
This innovative approach is encapsulated in a series of six recently issued U.S. patents, which introduce a radically different instruction execution model. Departing from conventional speculative techniques, this deterministic framework replaces guesswork with a time-based, latency-tolerant mechanism. Each instruction is assigned a precise execution slot within the pipeline, resulting in a rigorously ordered and predictable flow of execution. This reimagined model fundamentally alters how modern processors can handle latency and concurrency, enhancing both efficiency and reliability.
At the heart of this deterministic execution model is a simple time counter that deterministically sets the exact time for future instruction execution. Each instruction is dispatched to an execution queue with a preset execution time based on resolving its data dependencies and the availability of resources—such as read buses, execution units, and the write bus to the register file. Instructions remain queued until their scheduled execution slot arrives, marking a significant departure from the speculative execution paradigm that has dominated CPU design.
The architecture naturally extends into matrix computation, with a RISC-V instruction set proposal currently under community review. Configurable general matrix multiply (GEMM) units, ranging from 8×8 to 64×64, can operate using either register-based or direct-memory access (DMA)-fed operands. This flexibility supports a wide range of AI and high-performance computing (HPC) workloads. Early analyses suggest that this new architecture offers scalability that rivals Google’s Tensor Processing Units (TPUs), all while maintaining significantly lower cost and power requirements.
When comparing this deterministic design to traditional general-purpose CPUs, it becomes clear that the more accurate reference point is vector and matrix engines. Conventional CPUs still depend on speculation and branch prediction, whereas this new design applies deterministic scheduling directly to GEMM and vector units. The efficiency of this approach stems not only from the configurable GEMM blocks but also from the time-based execution model, where instructions are decoded and assigned precise execution slots based on operand readiness and resource availability.
Execution in this deterministic model is never a random or heuristic choice among many candidates; instead, it represents a predictable, pre-planned flow that keeps compute resources continuously busy. Planned matrix benchmarks will provide direct comparisons with TPU GEMM implementations, showcasing the ability to deliver datacenter-class performance without incurring the overhead typically associated with datacenter operations.
Critics of deterministic scheduling may argue that it introduces latency into instruction execution. However, it is essential to recognize that latency already exists due to waiting on data dependencies or memory fetches. Conventional CPUs attempt to mask this latency with speculation, but when predictions fail, the resulting pipeline flush introduces delays and wastes power. The time-counter approach acknowledges this latency and fills it deterministically with useful work, effectively avoiding rollbacks.
As noted in one of the foundational patents, instructions retain out-of-order efficiency. A microprocessor equipped with a time counter for statically dispatching instructions enables execution based on predicted timing rather than speculative issue and recovery. This model allows for preset execution times without the overhead of register renaming or speculative comparators.
The limitations of speculative execution have become increasingly evident, particularly in the context of modern AI and ML workloads, where vector and matrix operations dominate. Speculative execution boosts performance by predicting outcomes before they are known—executing instructions ahead of time and discarding them if the guess proves incorrect. While this approach can accelerate workloads, it also introduces unpredictability and power inefficiency. Mispredictions lead to the injection of “No Ops” into the pipeline, stalling progress and wasting energy on work that ultimately does not complete.
These issues are exacerbated in AI and ML environments, where irregular memory access patterns, long fetches, non-cacheable loads, and misaligned vectors frequently trigger pipeline flushes in speculative architectures. The result is a phenomenon known as performance cliffs, where performance varies wildly across datasets and problem sizes, making consistent tuning nearly impossible. Furthermore, the side effects of speculation have exposed vulnerabilities that have led to significant security exploits. As data intensity grows and memory systems become strained, speculation struggles to keep pace, undermining its original promise of seamless acceleration.
The core innovation of the deterministic execution model lies in its vector coprocessor, which utilizes a time counter for statically dispatching instructions. Rather than relying on speculation, instructions are issued only when data dependencies and latency windows are fully understood. This eliminates guesswork and costly pipeline flushes while preserving the throughput advantages of out-of-order execution. Architectures built on this patented framework feature deep pipelines—typically spanning 12 stages—combined with wide front ends supporting up to 8-way decode and large reorder buffers exceeding 250 entries.
In practical terms, a typical program running on the deterministic processor begins similarly to any conventional RISC-V system. Instructions are fetched from memory and decoded to determine whether they are scalar, vector, matrix, or custom extensions. The difference arises at the point of dispatch. Instead of issuing instructions speculatively, the processor employs a cycle-accurate time counter, working in conjunction with a register scoreboard, to decide precisely when each instruction can be executed. This mechanism provides a deterministic execution contract, ensuring that instructions complete at predictable cycles and reducing wasted issue slots.
The integration of a time counter and register scoreboard strategically positioned between the fetch/decode stages and the vector execution units marks a significant innovation. Instead of relying on speculative comparators or register renaming, this architecture utilizes a Register Scoreboard and Time Resource Matrix (TRM) to deterministically schedule instructions based on operand readiness and resource availability. By monitoring dependencies such as read-after-write (RAW) and write-after-read, it ensures hazards are resolved without incurring costly pipeline flushes.
Once operands are ready, the instruction is dispatched to the appropriate execution unit. Scalar operations utilize standard arithmetic logic units (ALUs), while vector and matrix instructions execute in wide execution units connected to a large vector register file. Because instructions are launched only when conditions are safe, these units maintain high utilization without the wasted work or recovery cycles caused by mispredicted speculation.
The key enabler of this deterministic approach is the simple time counter that orchestrates execution according to data readiness and resource availability. This ensures that instructions advance only when operands are ready and resources are available. The same principle applies to memory operations, where the interface predicts latency windows for loads and stores, allowing the processor to fill those slots with independent instructions and keep execution flowing smoothly.
From a programming perspective, the flow remains familiar; RISC-V code compiles and executes in the usual manner. The crucial distinction lies in the execution contract: rather than relying on dynamic speculation to hide latency, the processor guarantees predictable dispatch and completion times. This eliminates the performance cliffs and wasted energy associated with speculation while still providing the throughput benefits of out-of-order execution.
This deterministic execution model preserves the familiar RISC-V programming model while eliminating the unpredictability and wasted effort of speculation. As John Hennessy aptly stated, “It’s stupid to do work in runtime that you can do in compile time.” This sentiment reflects the foundational principles of RISC and its forward-looking design philosophy.
The RISC-V Instruction Set Architecture (ISA) provides opcodes for custom and extension instructions, including floating-point, digital signal processing (DSP), and vector operations. The outcome is a processor that executes instructions deterministically while retaining the benefits of out-of-order performance. By eliminating speculation, the design simplifies hardware, reduces power consumption, and avoids pipeline flushes.
These efficiency gains become even more pronounced in vector and matrix operations, where wide execution units require consistent utilization to achieve peak performance. Vector extensions necessitate wide register files and large execution units, which in speculative processors require expensive register renaming to recover from branch mispredictions. In contrast, the deterministic design executes vector instructions only after commit, thereby eliminating the need for renaming.
Each instruction is scheduled against a cycle-accurate time counter, which provides a deterministic execution contract, ensuring that instructions complete at predictable cycles and reducing wasted issue slots. The vector register scoreboard resolves data dependencies before issuing instructions to the execution pipeline, allowing for a known order of dispatch at the correct cycle, making execution both predictable and efficient.
Vector execution units, whether integer or floating point, connect directly to a large vector register file. Because instructions are never flushed, there is no renaming overhead. The scoreboard ensures safe access, while the time counter aligns execution with memory readiness. A dedicated memory block predicts the return cycle of loads, allowing the processor to schedule independent instructions into latency slots, keeping execution units busy without stalling or speculative execution.
In contemporary CPUs, compilers and programmers typically write code with the expectation that the hardware will dynamically reorder instructions and speculatively execute branches. The hardware manages hazards through register renaming, branch prediction, and recovery mechanisms. While programmers benefit from enhanced performance, this comes at the expense of unpredictability and increased power consumption.
In the deterministic time-based architecture, instructions are dispatched only when the time counter indicates that their operands will be ready. This means that the compiler or runtime system does not need to insert guard code for misprediction recovery. Consequently, compiler scheduling becomes simpler, as instructions are guaranteed to issue at the correct cycle without rollbacks. For programmers, the ISA remains RISC-V compatible, but deterministic extensions reduce reliance on speculative safety nets.
The application of this deterministic model in AI