Coral NPU's **Zvt** matrix execution engine implements the RISC-V ISA extensions
for high-throughput matrix multiplication computation. These instructions are
described in the RISC-V specification document [Zvt Family of Vector Matrix
Extensions
(v0.4)](https://lists.riscv.org/g/tech-vme/attachment/411/0/zvt-20260831.pdf).
The official v1.0 release of this specification is scheduled for late 2026. This
functionality is also referred to as the Vector Matrix Extensions (VME).

The ISA extensions provide specialized configuration registers and instructions
for high-throughput tensor manipulation, matrix multiplication, and efficient
data orchestration.

The VME-Zvt matrix extensions build on the Zve32x [RVV vector
extensions](https://developers.google.com/coral/guides/hardware/rvv-vector-engine). The 128-bit vector
registers v0 -- v31 serve as a data source for matrix operands, and the existing
vector instructions are used for pre- and post-processing of matrices.

> [!NOTE]
> **Note:** The definitive reference for the matrix instructions is the official RISC-V specification -- always refer to that document for complete information. This page gives an overview and summary of the matrix instructions.

In Coral NPU, the matrix execution engine accelerates compute-dense ML
workloads:

- **Outer-product multiply-accumulator (MAC)**: A Multiply-Accumulate (MAC) operation multiplies two numbers together and adds the result to an existing value stored in an accumulator.
- **Transformer prefill**: The prefill phase in transformer models is the initial step of LLM inference in which the model processes the entire input prompt in parallel to build its foundational internal state and compute the Key-Value (KV) cache.
- **Speculative decoding**: Speculative decoding is an inference-time optimization for large language models (LLMs) that generates multiple tokens per decoding step instead of one. This technique speeds up AI text generation by using a small, fast model to guess multiple words ahead and a large model to check all the guesses at once.
- **Enables [Gemma 3 270M](https://developers.googleblog.com/en/introducing-gemma-3-270m/)
  end-to-end**: Gemma is a family of lightweight, state-of-the-art open models built from the same research and technology used to create the Gemini models. Gemma models are provided with open weights, allowing you to tune and deploy them in your applications.

The Coral MLIR compiler explicitly prepares the matrix math for the Coral NPU
hardware, slicing massive data blocks to fit perfectly into the 16 internal
matrix tile accumulators.

## RISC-V VME-Zvt extensions

The Vector Matrix Extensions (VME) are a matrix compute extension for AI/ML
applications based on an outer-product formulation of matrix-multiply. A basic
VME *matmul* operation takes two vector registers, computes the outer product of
the operands, and accumulates the results into a large accumulator. This
targets efficient scaling from small- to high-performance implementations with
high power efficiency and architectural simplicity.

VME is one of several approaches to matrix extensions for RISC-V, alongside IME
(Integrated Matrix Extension). IME operates solely out of the existing RVV
vector register file, so its performance for embedded and application-class
processors is limited by vector register file capacity. VME instead remains
closely coupled to a hart and its RVV implementation, reading vector registers
for source operands and leveraging RVV for surrounding AI/ML computation, while
adding an accumulator register file into which *matmul* results accumulate.

The VME extension has the following advantages:

- A relatively simple matrix extension based on outer-product matrix multiplication, with no new 2D load/store instructions, operating on matrix memory data in standard row- or column-major order
- Leverages the existing RVV engine of Coral NPU and the AI/ML vector extensions.
- Instructions for *matmul* compute and for moving data into and out of accumulator registers, plus `vsetvl`-like instructions to set matrix parameters for subsequent *matmul* instructions.
- Matrix-dimension-agnostic operation, similar to RVV being vector-length-agnostic and using similar `vsetvl` and `vl` mechanisms.

## VME-Zvt micro-architecture in Coral NPU

Coral NPU's matrix engine is the core computational module for high-throughput
matrix multiplication and tensor operations. It includes a dedicated array of
processing elements (PEs) and accumulators.

The Coral NPU matrix engine operates as a decoupled co-processor to the vector
engine, receiving micro-ops (µops) and data through dedicated command and
register file interfaces. It uses a multi-stage decoupled pipeline to maximize
throughput:

- **Command dequeue**: Commands are received from the vector dispatcher and staged in a handshake-based command queue.
- **Data orchestration**: Synchronizes with the Vector Register File (VRF) and accumulators, fetching source operands (va, vb) and managing subtile access.
- **PE array computation** : The core PE array performs fused multiply-add (FMA) operations. The pipeline includes internal logic to handle bubbles (`REMVPIPEBUBB`) and optimize multiplier/adder utilization.
- **Accumulator management**: Results are written back to dedicated high-speed accumulators or local memory, supporting saturating arithmetic and floating-point exception tracking.

### Edge cases

Coral NPU handles the following edge cases:

- **Precise exceptions**: A fault during matrix accumulation is trapped and reported at the end of the command sequence.
- **Partial matrix operations**: Operations with dimensions not perfectly mapping to PE array width are handled through operand padding and masked writes.
- **Accumulator saturation**: Saturating arithmetic triggers when values exceed the defined precision, setting the appropriate status flags.

## Matrix instructions

The definitive reference for these instructions is the official specification
document [Zvt Family of Vector Matrix Extensions
(v0.4)](https://lists.riscv.org/g/tech-vme/attachment/411/0/zvt-20260831.pdf). A
summary of the instructions is given here for quick reference.

Software must execute `mset*` configuration instructions to establish the
M x K x N
tiling dimensions, before issuing systolic matrix compute micro-ops.

Other instructions are provided to move rows and columns of the matrix arrays
--- moving them either to or from the vector registers or memory.

| Instruction | Description |
|---|---|
| *Matrix configuration* |   |
| `msetmtype rs1,rs2` | Write the `mtype` and `vtype` CSRs to the values in the `rs1` and `rs2` registers. |
| `msetmtypei` | Similar to `msetmtype` except: 1) the new `mtype` CSR value is supplied by a zero-extended immediate, 2) the `vtype.vsew` field value is supplied by a zero-extended immediate, and 3) all other `vtype` fields are zeroed. |
| `msettn rd,rs1` | Set tn/vl to the value in `rs1`, constrained by the current matrix configuration; sets matrix dimension N. Operates similarly to `vsetvl`, updating `vl` from `rs1` and writing the new `vl` to `rd`. (See note below.\*) |
| `msettm rd,rs1` | Set tm to the value in `rs1`, constrained by the current matrix configuration -- sets tm to be the lesser of `rs1`, LMULxEVE, and ETE. Sets matrix dimension M. (See note below.\*) |
| `msettk rd,rs1` | Set tk to the value in `rs1`, constrained by the current matrix configuration -- sets tk to the lesser of `rs1` and KMAX. Sets matrix dimension K. (See note below.\*) |
| *Load/store tile subset to memory* |   |
| `vtle* rs2,rs1` | Load tile subset from memory, with `rs1` = memory address, `rs2` = TSS. For example: `vtle32 rs2,rs1` |
| `vtse* rs2,rs1` | Store tile subset to memory, with `rs1` = memory address, `rs2` = TSS. For example: `vtse32 rs2,rs1` |
| *Move tile subset between tile state and vector registers* |   |
| `vtmv.v.t vd,rs1` | Move tile subset to vector register group, with `vd` = destination vector register and `rs1` = TSS of source. |
| `vtmv.t.v rs1,vs2` | Move vector register group to tile subset, with `vs2` = source vector register and `rs1` = TSS of destination. |
| *Matrix arithmetic* |   |
| `vtfmm.tvv mtd,vs2,vs1` | Floating-point matrix multiply; use the values of tm, tn, and tk to determine what computations to perform. `vs2` specifies A vector register group, `vs1` specifies B vector register group, `mtd` is C destination tile specifier. \*\* |
| `vtfmm.alt.tvv mtd,vs2,vs1` | Floating-point matrix multiply; use the values of tm, tn, and tk to determine what computations to perform. `vs2` specifies A vector register group, `vs1` specifies B vector register group, `mtd` is C destination tile specifier. \*\* |
| `vtmmu.tvv mtd,vs2,vs1` | Integer matrix multiply; use the values of tm, tn, and tk to determine what computations to perform. `vs2` specifies A vector register group, `vs1` specifies B vector register group, `mtd` is C destination tile specifier. \*\* |
| `vtmms.tvv mtd,vs2,vs1` | Integer matrix multiply; use the values of tm, tn, and tk to determine what computations to perform. `vs2` specifies A vector register group, `vs1` specifies B vector register group, `mtd` is C destination tile specifier. \*\* |
| *Miscellaneous* |   |
| `vtzero mtd` | Write 0 to each element of the tm by tn submatrix of the destination matrix tile. |
| `vtdiscard` | Inform the runtime that the matrix tile state is no longer useful and need not be saved. |

\* For `msettm` and `msettk`, the new field value is written back to the
destination register `rd`. The `msettn` instruction uses the standard
`vsetvl` datapath to write the updated `vl` to `rd`.

\*\* `vtype` holds SEW and `altfmt`; `mtype` holds TWIDEN

- data type of `vs2` is supplied by opcode,SEW
- data type of `vs1` is supplied by `altfmt`,SEW
- data type of `mtd` is supplied by opcode,SEW,TWIDEN

<br />


*** ** * ** ***

<br />