Linux on a Simulated Rocket Core, over Ethernet

#FireSim#Linux#RISC-V#FPGA#Ethernet#TxHDL#Bazel#auto

Linux 6.18 now boots on FireSim’s Rocket design on an Alinx AX7A200B board. The host reaches the board over an Ethernet cable, as in the last post. The whole run takes 4 min 36 s: 4 min 22 s to load the software, and 13 s from the firmware’s banner to a shell that shows the machine and powers off.

Count the layers. The FPGA runs a simulator. The simulator runs a RISC-V core, cycle by cycle. The core runs Linux, and Linux runs a shell. We must go deeper. The shell is the bottom level of this dream, and it stays there: it powers off, and FireSim’s host program exits with *** PASSED ***.

The run

This is the recorded terminal of one run, on October 30, 2026. The load prints nothing for 4 min 22 s, so the replay cuts every wait to 2 s. The clock shows the time of the real run.

loading the recording 
Press Play.

The end of the run, shortened:

OpenSBI v1.6
Linux version 6.18.55 (txhdl@txhdl) (clang version 20.1.8, LLD 20.1.8)
Machine model: ucb-bar,chipyard
Run /init as init process
txhdl: userspace is up on Rocket
Linux (none) 6.18.55 #1 riscv64 GNU/Linux
uarch		: sifive,rocket0
MemTotal:        1027732 kB
txhdl: powering off
reboot: Power down
*** PASSED *** after 288686162 cycles

The software

The driver loads one ELF file of 4.6 MB, 3.45 MB of which goes into memory:

  • OpenSBI 1.6, the RISC-V firmware, which runs in machine mode.
  • Linux 6.18.55, configured from tinyconfig with only what this machine has.
  • BusyBox 1.37 on musl, static, in an initramfs inside the kernel.

Bazel builds all three from sources pinned by checksum, with the LLVM 20 toolchain, and nothing installed on the host. TxHDL already built Linux this way for its own RV32 core, Vreteno. The same scripts now take a 64-bit target, and Vreteno’s outputs are the same bytes as before.

The kernel needs to know the machine. FireSim’s Rocket design keeps its device tree in its boot ROM, and the boot ROM is in the generated Verilog. The device tree says: one RV64 hart with Sv39 paging, the CLINT timer, the PLIC interrupt controller, a SiFive serial port at 0x10020000, and 1 GiB of memory at 0x80000000, which is the board’s DDR3.

Two things that had to match FireSim

The serial port’s speed. The first boot printed the kernel’s log up to the interrupt controller, and then 1,680 NUL bytes. At that point the kernel moves its console from the early driver to the real one, and the real one sets the speed: 115,200 baud by default. FireSim’s UART bridge decodes the speed that the port’s divisor resets to, 271 on the 1 GHz bus, which is 3,686,400 baud. The kernel’s command line now says console=ttySIF0,3686400.

Where the ELF goes. OpenSBI links itself as a position-independent executable. FireSim’s front end loads such a file 0x80000000 higher than its link address. The memory ignores the extra address bit, so the load still landed in the right place, and Linux booted. But the front end then looked for the tohost word, which the firmware writes to power off, at 0x1_00042a90. That run took 18 minutes and 36 billion target cycles, and ended two minutes after the kernel powered off. The build now marks the ELF as an ordinary executable, since its segments are at their run addresses already.

Where the time goes

Stage Time
Load 3.45 MB into the DDR3 4 min 22 s
OpenSBI to power-off 13 s
Whole run 4 min 36 s

FireSim writes the target’s memory 4 bytes per register write. Over this link, each register write is one Ethernet frame and its answer, about 0.3 ms. That is 860,000 round trips for this image.

Two changes would make the load much faster:

  • Let the board take a stream of writes without an answer to each. The board’s host bridge does one access per frame today.
  • Lower the interrupt moderation of the host’s USB Ethernet adapter, which holds a received frame back before it reports it.

The bitstream as a release

The bitstream, the host driver and the example programs from the last post are now a release, built from one commit of TxHDL and checked on the board. With them, you can run programs on an AX7A200B without building anything. You still need a tool to program the FPGA, such as AMD’s free Vivado Lab Edition.

Also fixed

Golden Gate, FireSim’s compiler from a design to a simulator, is a jar that bazel_firesim_goldengate builds and publishes. The published jar held about 190 file paths of the machine that built it, in Chisel’s source locators. A build step now rewrites them to paths inside each source repository, and two builds in different directories give the same jar, byte for byte.

What is not done yet

  • A root file system on FireSim’s block-device bridge. Every block access would be a register exchange over the cable.
  • A faster load, as above.

The code is in the TxHDL repository, under firesim/ax7a200b/linux/ and linux/. The language itself is described in the TxHDL book.

If you are interested in the method I use to build all of this code, see my book Hardware Development with Bazel.