## Overview

Coral NPU is an open-source IP designed by Google Research and is freely
available for integration into ultra-low-power System-on-Chips (SoCs) targeting
wearable devices such as hearables, augmented reality (AR) glasses and smart
watches. Coral is a neural processing unit (NPU), also known as an AI
accelerator or deep-learning processor.

Coral NPU is a highly efficient hardware accelerator for ML inferencing. It
lowers SoC energy consumption by reducing both the control overhead of processor
(ALU) operations and the distance data must travel within the memory hierarchy.

Coral NPU's instruction set is domain-optimized for ML, featuring
application-specific arithmetic operations and data paths built to feed the
compute engines efficiently.

This specialized design contrasts with traditional CPUs. CPUs are highly
flexible, offering general-purpose instruction sets, an accessible memory model,
and uniform software APIs. This allows them to use standard compilers and tools,
leveraging broad ecosystem investments. However, this general-purpose
programmability makes CPUs inefficient for AI, with a much lower
compute-per-Joule, as they lack domain specialization.

Older ML accelerators, on the other hand, are not robust. While often efficient,
they are difficult to program, have a limited ability to reuse compute resources
for general tasks, and have poor support for execution-as-function-call. The
Coral NPU, in contrast, is designed to be both efficient and robust. It is an
energy-efficient, C-programmable accelerator that resolves this tradeoff.

Coral NPU is based on the 32-bit RISC-V Instruction Set Architecture (ISA). The
extensible industry-standard RISC-V ISA empowers developers to create optimized,
customizable computing solutions, from custom processors to hardware/software
co-designs. Extending the ISA with custom compute features makes Coral NPU the
ideal hardware accelerator for running ML inferencing operations such as:

- Image classification
- Person detection
- Pose estimation
- Transformer (neural network)

Coral NPU enables a software-focused approach to AI hardware, as well as a
unified programming model across AI workloads running on CPUs, GPUs, and NPUs.
RISC-V allows for domain-specific customization tailored to particular
applications and workloads by enabling the selection of appropriate ratified
extensions to the standard --- for example vector and matrix operations. For the
complete list of RISC-V specifications, see the official [RISC-V
specifications](https://riscv.org/specifications/ratified/).

This Coral NPU documentation provides:

- Hardware reference and IP integration guidelines for commercial silicon designers and academia
- Guidance for using the software toolchain to compile and run ML models on the NPU processor

The Coral NPU hardware design and software tools codebase is available on
[GitHub](https://github.com/google-coral/coralnpu).

To help you understand the design, Google publishes a [Gemini code wiki](https://codewiki.google/github.com/google-coral/coralnpu)
for Coral NPU. This
[structured wiki](https://developers.googleblog.com/introducing-code-wiki-accelerating-your-code-understanding/)
provides
Gemini-generated documentation for the open-source repository. The information
is automatically refreshed and always up-to-date --- no stale docs ever!

The MPACT-CoralNPU behavioral simulator is housed in a separate GitHub
repository located [here](https://github.com/google-coral/coralnpu-mpact).

## Coral NPU components

Coral NPU includes three distinct processor components that work together:
matrix, vector (SIMD), and scalar. The terms SIMD (single-instruction, multiple
data) and vector are used synonymously here.

![Coral NPU components](https://developers.google.com/static/coral/images/components_detail_notitle.svg)

| Subcomponent | Description |
|---|---|
| **Fetch / decode** | Retrieves and interprets program instructions |
| **Scoreboard** | Manages instruction dependencies to prevent conflicts |
| **ALU** | Arithmetic logic unit |
| **BRU** | Branch unit: controls the program's flow (loops, jumps) |
| **LSU** | Load/store unit: moves data between registers and memory |
| **CSR** | Control/status registers |
| **ITCM** | Instruction tightly-coupled SRAM memory |
| **DTCM** | Data tightly-coupled SRAM memory |
| **SIMD** | Single-instruction, multiple data |
| **MAC** | Multiply-accumulate execution engine |

## Coral NPU features

Coral NPU offers the following top-level feature set:

- **RV32IMF_Zve32f_Zvt** RISC-V instruction set
- 32-bit address space for applications and operating system kernels
- Four-stage processor, in-order dispatch, out-of-order retire
- Four-way scalar, two-way vector dispatch
- 128-bit SIMD, 256-bit pipeline
- 8 KB ITCM memory (tightly-coupled memory for instructions)
- 32 KB DTCM memory (tightly-coupled memory for data)
- Both memories are single-cycle-latency SRAM, more efficient than cache memory
- AXI4 bus interfaces, functioning as both master and slave, to interact with external memory and allow external CPUs to configure the Coral NPU
- BFloat16 data type for native transformer model support
- GNU Debugger (GDB) support

Coral NPU's **RV32IMF_Zve32f_Zvt** specification includes the following extensions
to the RISC-V ISA base:

- RV32IMF -- this label indicates the RISC-V 32-bit base instruction set with I (integer) and M (multiplication, division) extensions (operations).
- I -- Base integer instruction set
- M -- Standard Extension for Integer Multiplication and Division
- F -- Standard Extension for Single-Precision Floating-Point
- V -- Standard Extension for RVV vector operations; see [RISC-V vector
  specification PDF](https://github.com/riscvarchive/riscv-v-spec/releases/download/v1.0/riscv-v-spec-1.0.pdf)
- Zve32f -- subset of V (vector operations), refer to *Zve\* Vector
  Extensions for Embedded Processors* in the RISC-V vector specification
- Zvt -- Vector Matrix Extensions (VME) for matrix operations

Note that this is a partial list is given here for quick reference; see [RISC-V
specifications](https://riscv.org/specifications/)
for the complete list of ratified and in-progress extensions.

The complete RISC-V label for Coral NPU is
`rv32imf_zve32f_zvl128b_zicsr_zifencei_zbb_zfbfmin_zvfbfa_zvt`.

### Scalar core features

- RV32IMF frontend that drives command queues of the ML and SIMD backend
- Thirty-one 32-bit scalar registers (x1 -- x31), with x0 hardwired to zero
- Thirty-two 32-bit floating-point registers (f0 -- f31)
- Various control and status registers (CSRs) per RV32_Zicsr

### Vector execution engine features

- Supports data widths of 8, 16, and 32 bits
- Vector floating-point data: FP32, FP16, BF16
- Thirty-two 128-bit vector registers (v0 -- v31), with VLEN = 128
  - For example, int16 x 8-bit
- 8 x 8 x 32-bit accumulator (acc\<8\>\<8\>)
- Parameterizable VLEN (vector register width)
- Wider vector paths (up to VLEN = 1024) unlock LLM-class workloads, including TinyGemma 270M

### Matrix execution engine features

- Quantized outer-product multiply-accumulate (MAC) engine
- 4 x 8-bit multiplies reduced into 32-bit accumulators
- 256 MACs per-cycle
- Functions as a tensor processing engine (TPE)

### Supported data types

Coral NPU supports the following data types: `int8`, `int16`, IEEE 754
single-precision 32-bit `float` (FP32), FP16, and BF16.

BFloat16 (Brain floating-point) is a shortened (16-bit) version of the 32-bit
IEEE 754 single-precision floating-point format that accelerates machine
learning and near-sensor computing. This data type supports native ML
transformer models.

## NPU block diagram

A more detailed view of the Coral NPU architecture is shown below.

![Coral NPU architecture](https://developers.google.com/static/coral/images/arch_data_flow_notitle.png)

## GitHub repository layout

The [GitHub repo](https://github.com/google-coral/coralnpu)
for Coral NPU
includes the following subdirectories, organized by design component:

| Directory | Contents |
|---|---|
| `/hdl/chisel/src/common/` | Primitive building blocks used widely (Fifos, arbiters, ring buffers, etc.) |
| `/hdl/chisel/src/coralnpu/` | Top-level Coral NPU folder, contains instantiation of the core, memories, bus connections |
| `/hdl/chisel/src/coralnpu/scalar/` | RISC-V scalar core: fetch, decode, dispatch, scalar ALUs/Mul/Div, register file |
| `/hdl/chisel/src/coralnpu/float/` | Integration with cvfpu floating-point unit |
| `/hdl/chisel/src/coralnpu/rvv/` | Integration touchpoints with the RVV core |
| `/hdl/verilog/rvv/` | RVV core SystemVerilog and testbenches |
| `/tests/cocotb/` | Test suites for scalar and vector cores, and tests for bus integrations |

<br />


*** ** * ** ***

<br />