Vortex GPGPU ASIC Implementation

Vortex is a full-stack open-source RISC-V GPGPU. Vortex supports multiple backend drivers, including our C++ simulator (simx), an RTL simulator, and physical Xilinx and Altera FPGAs-- all controlled by a single driver script. The chosen driver determines the corresponding code invoked to run Vortex. Generally, developers will prototype their intended design in simx, before completing going forward with an RTL implementation. Alternatively, you can get up and running by selecting a driver of your choice and running a demo program.

You can find the original Vortex repo: https://github.com/vortexgpgpu/vortex

This repository is an implementation of Vortex into an ASIC using open-source EDA tools.

The target configuration is a single Vortex core with L1 cache and external main memory, with the FPU disabled and the larger FPGA-oriented hierarchy reduced or bypassed. The long-term goal is to establish an ASIC platform of Vortex in order to do research GPGPU ASIC metrics such as power, area, timing etc..

Toolchain

The project currently uses:

sv2v        SystemVerilog-to-Verilog conversion
Verilator   RTL linting and simulation
Icarus      Optional RTL/gate-level simulation
OpenLane    ASIC flow orchestration
OpenROAD    Floorplanning, placement, CTS, routing, timing
Yosys       Logic synthesis
KLayout     DRC/signoff checking
Netgen      LVS
sky130A     Target PDK

The OpenLane flow expects the sky130 SRAM macro files to be available under the local PDK installation.

Setup

The reported results were generated using OpenLane 2.3.10 in its Nix development environment.

Install the tested OpenLane revision:

./scripts/install_openlane_env.sh

---

Directory Structure

• build/
  • sv2v - Generated sv2v Verilog output
• constraints/ -
  • macro_placement.cfg - Manual SRAM macro placement
  • pin_order.cfg - Optional pin ordering
  • VX_top.sdc - Timing constraints
• rtl/
  • blackbox/ - SRAM blackbox stubs for OpenLane
  • cache/ - Vortex cache
  • core/ - Vortex core
  • interfaces/ - Vortex interface headers
  • libs/ - Common RTL utils
  • mem/ - Vortex memory
  • VX_top.sv - Vortex ASIC top level
  • VX_socket.sv - Vortex socket
• run/ - Openlane run directories
• scripts/ - Utility scripts
• tests/ - RTL simulation (Verilator + Icarus Verilog)
• config.json - Default openlane config without sv2v
• config_sv2v.json - Openlane config with sv2v
• Makefile

Current Status

The current flow uses sv2v to convert the adapted SystemVerilog RTL into Verilog before running OpenLane/OpenROAD.

Current ASIC-oriented configuration:

1 cluster
1 core
8 warps
1 thread per warp
RV32
FPU disabled
L1 cache disabled
local memory disabled
external memory interface exposed
GPR/register file implemented with a sky130 SRAM macro

Current SRAM macro integration:

GPR/register file:
  sky130_sram_1kbyte_1rw1r_32x256_8

The 8-warp, 1-thread configuration is intentionally chosen so that the scalar RV32 GPR file maps cleanly to the 1 KiB SRAM macro:

32 registers × 8 warps × 32 bits = 8192 bits = 1 KiB

The current physical-design focus is resolving routing congestion for the SRAM-integrated 8-warp configuration.

---

Interface Flattening Baseline

Purpose

The original Vortex RTL uses SystemVerilog interfaces and modports such as:

VX_dcr_bus_if.slave dcr_bus_if;
VX_mem_bus_if.master mem_bus_if [`L1_MEM_PORTS];

Many open-source ASIC flows have limited support for complex SystemVerilog interface syntax. To make the design more tool-compatible, these interfaces are flattened into explicit Verilog/SystemVerilog ports using header macros. This will be a long-term project, but will rely on sv2v for now.

Flattened DCR bus

The DCR bus is flattened into:

dcr_bus_if_write_valid
dcr_bus_if_write_addr
dcr_bus_if_write_data

The corresponding header should provide macros such as:

`VX_DCR_BUS_SIGNALS(...)
`VX_DCR_BUS_CONSUMER_PORTS(...)
`VX_DCR_BUS_PRODUCER_PORTS(...)
`VX_DCR_BUS_PASS_PORTS(...)

Flattened memory bus

The memory bus is flattened into request and response signals:

req_valid
req_ready
req_data_rw
req_data_addr
req_data_data
req_data_byteen
req_data_flags
req_data_tag_uuid
req_data_tag_value

rsp_valid
rsp_ready
rsp_data_data
rsp_data_tag_uuid
rsp_data_tag_value

The memory bus address is a line address, not a byte address:

line_addr = byte_addr >> log2(DATA_SIZE)

For example, if L1_LINE_SIZE = 64 and STARTUP_ADDR = 0x80000000, then:

STARTUP_LINE_ADDR = 0x80000000 >> 6 = 0x02000000

TODO

Pipeline/module focus

• Document setting up the EDA tools
• Document metrics