8.2. Communicating with the DUT
There are two types of DUTs that can be made: tethered or standalone DUTs. A tethered DUT is where a host computer (or just host) must send transactions to the DUT to bringup a program. This differs from a standalone DUT that can bringup itself (has its own bootrom, loads programs itself, etc). An example of a tethered DUT is a Chipyard simulation where the host loads the test program into the DUTs memory and signals to the DUT that the program is ready to run. An example of a standalone DUT is a Chipyard simulation where a program can be loaded from an SDCard out of reset. In this section, we mainly describe how to communicate to tethered DUTs.
There are two ways the host (otherwise known as the outside world) can communicate with a tethered Chipyard DUT:
Using the Tethered Serial Interface (TSI) or the Debug Module Interface (DMI) with the Front-End Server (FESVR) to communicate with the DUT
Using the JTAG interface with OpenOCD and GDB to communicate with the DUT
The following picture shows a block diagram view of all the supported communication mechanisms split between the host and the simulation.
8.2.1. Using the Tethered Serial Interface (TSI) or the Debug Module Interface (DMI)
If you are using TSI or DMI to communicate with the DUT, you are using the Front-End Server (FESVR) to facilitate communication between the host and the DUT.
22.214.171.124. Primer on the Front-End Server (FESVR)
FESVR is a C++ library that manages communication between a host machine and a RISC-V DUT. For debugging, it provides a simple API to reset, send messages, and load/run programs on a DUT. It also emulates peripheral devices. It can be incorporated with simulators (VCS, Verilator, FireSim), or used in a bringup sequence for a taped out chip.
Specifically, FESVR uses the Host Target Interface (HTIF), a communication protocol, to speak with the DUT. HTIF is a non-standard Berkeley protocol that uses a FIFO non-blocking interface to communicate with the DUT. It defines a protocol where you can read/write memory, load/start/stop the program, and more. Both TSI and DMI implement this HTIF protocol differently in order to communicate with the DUT.
126.96.36.199. Using the Tethered Serial Interface (TSI)
By default, Chipyard uses the Tethered Serial Interface (TSI) to communicate with the DUT.
TSI protocol is an implementation of HTIF that is used to send commands to the RISC-V DUT.
These TSI commands are simple R/W commands that are able to access the DUT’s memory space.
During simulation, the host sends TSI commands to a simulation stub in the test harness called
(C++ class) that resides in a
SimSerial Verilog module (both are located in the
SimSerial Verilog module then sends the TSI command recieved by the simulation stub
to an adapter that converts the TSI command into a TileLink request.
This conversion is done by the
SerialAdapter module (located in the
After the transaction is converted to TileLink, the
TLSerdesser (located in
generators/testchipip) serializes the
transaction and sends it to the chip (this
TLSerdesser is sometimes also referred to as a digital serial-link or SerDes).
Once the serialized transaction is received on the chip, it is deserialized and masters a TileLink bus on the chip
which handles the request.
In simulation, FESVR resets the DUT, writes into memory the test program, and indicates to the DUT to start the program
through an interrupt (see Chipyard Boot Process).
Using TSI is currently the fastest mechanism to communicate with the DUT in simulation (compared to DMI/JTAG) and is also used by FireSim.
188.8.131.52. Using the Debug Module Interface (DMI)
Another option to interface with the DUT is to use the Debug Module Interface (DMI).
Similar to TSI, the DMI protocol is an implementation of HTIF.
In order to communicate with the DUT with the DMI protocol, the DUT needs to contain a Debug Transfer Module (DTM).
The DTM is given in the RISC-V Debug Specification
and is responsible for managing communication between the DUT and whatever lives on the other side of the DMI (in this case FESVR).
This is implemented in the Rocket Chip
Subsystem by having the
During simulation, the host sends DMI commands to a
simulation stub called
SimDTM (C++ class) that resides in a
SimDTM Verilog module
(both are located in the
generators/rocket-chip project). This
SimDTM Verilog module then
sends the DMI command recieved by the simulation stub into the DUT which then converts the DMI
command into a TileLink request. This conversion is done by the DTM named
DebugModule in the
When the DTM receives the program to load, it starts to write the binary byte-wise into memory.
This is considerably slower than the TSI protocol communication pipeline (i.e.
which directly writes the program binary to memory.
184.108.40.206. Starting the TSI or DMI Simulation
All default Chipyard configurations use TSI to communicate between the simulation and the simulated SoC/DUT. Hence, when running a software RTL simulation, as is indicated in the Software RTL Simulation section, you are in-fact using TSI to communicate with the DUT. As a reminder, to run a software RTL simulation, run:
cd sims/verilator # or cd sims/vcs make CONFIG=RocketConfig run-asm-tests
If you would like to build and simulate a Chipyard configuration with a DTM configured for DMI communication, then you must tie-off the serial-link interface, and instantiate the SimDTM.
class dmiRocketConfig extends Config( new chipyard.harness.WithSerialAdapterTiedOff ++ // don't attach an external SimSerial new chipyard.config.WithDMIDTM ++ // have debug module expose a clocked DMI port new freechips.rocketchip.subsystem.WithNBigCores(1) ++ new chipyard.config.AbstractConfig)
Then you can run simulations with the new DMI-enabled top-level and test-harness.
cd sims/verilator # or cd sims/vcs make CONFIG=dmiRocketConfig run-asm-tests
8.2.2. Using the JTAG Interface
Another way to interface with the DUT is to use JTAG.
Similar to the Using the Debug Module Interface (DMI) section, in order to use the JTAG protocol,
the DUT needs to contain a Debug Transfer Module (DTM) configured to use JTAG instead of DMI.
Once the JTAG port is exposed, the host can communicate over JTAG to the DUT through a simulation stub
SimJTAG (C++ class) that resides in a
SimJTAG Verilog module (both reside in the
This simulation stub creates a socket that OpenOCD and GDB can connect to when the simulation is running.
The default Chipyard designs instantiate the DTM configured to use JTAG (i.e.
As mentioned, default Chipyard designs are enabled with JTAG. However, they also use TSI/Serialized-TL with FESVR in case the JTAG interface isn’t used. This allows users to choose how to communicate with the DUT (use TSI or JTAG).
220.127.116.11. Debugging with JTAG
Roughly the steps to debug with JTAG in simulation are as follows:
Build a Chipyard JTAG-enabled RTL design. Remember default Chipyard designs are JTAG ready.
cd sims/verilator # or cd sims/vcs make CONFIG=RocketConfig
Run the simulation with remote bit-bang enabled. Since we hope to load/run the binary using JTAG, we can pass
noneas a binary (prevents FESVR from loading the program). (Adapted from: https://github.com/chipsalliance/rocket-chip#3-launch-the-emulator)
# note: this uses Chipyard make invocation to run the simulation to properly wrap the simulation args make CONFIG=RocketConfig BINARY=none SIM_FLAGS="+jtag_rbb_enable=1 --rbb-port=9823" run-binary
This section was adapted from the instruction in Rocket Chip and riscv-isa-sim. For more information refer to that documentation: Rocket Chip GDB Docs, riscv-isa-sim GDB Docs
8.2.3. Example Test Chip Bringup Communication
18.104.22.168. Intro to Typical Chipyard Test Chip
Most, if not all, Chipyard configurations are tethered using TSI (over a serial-link) and have access to external memory through an AXI port (backing AXI memory). The following image shows the DUT with these set of default signals:
In this setup, the serial-link is connected to the TSI/FESVR peripherals while the AXI port is connected
to a simulated AXI memory.
However, AXI ports tend to have many signals, and thus wires, associated with them so instead of creating an AXI port off the DUT,
one can send the memory transactions over the bi-directional serial-link (
TLSerdesser) so that the main
interface to the DUT is the serial-link (which has comparatively less signals than an AXI port).
This new setup (shown below) is a typical Chipyard test chip setup:
22.214.171.124. Simulation Setup of the Example Test Chip
To test this type of configuration (TSI/memory transactions over the serial-link), most of the same TSI collateral would be used. The main difference is that the TileLink-to-AXI converters and simulated AXI memory resides on the other side of the serial-link.
Here the simulated AXI memory and the converters can be in a different clock domain in the test harness than the reference clock of the DUT. For example, the DUT can be clocked at 3.2GHz while the simulated AXI memory can be clocked at 1GHz. This functionality is done in the harness binder that instantiates the TSI collateral, TL-to-AXI converters, and simulated AXI memory. See Creating Clocks in the Test Harness on how to generate a clock in a harness binder.
This type of simulation setup is done in the following multi-clock configuration:
class MulticlockAXIOverSerialConfig extends Config( new chipyard.config.WithSystemBusFrequencyAsDefault ++ new chipyard.config.WithSystemBusFrequency(250) ++ new chipyard.config.WithPeripheryBusFrequency(250) ++ new chipyard.config.WithMemoryBusFrequency(250) ++ new chipyard.config.WithFrontBusFrequency(50) ++ new chipyard.config.WithTileFrequency(500, Some(1)) ++ new chipyard.config.WithTileFrequency(250, Some(0)) ++ new chipyard.config.WithFbusToSbusCrossingType(AsynchronousCrossing()) ++ new testchipip.WithAsynchronousSerialSlaveCrossing ++ new freechips.rocketchip.subsystem.WithAsynchronousRocketTiles( AsynchronousCrossing().depth, AsynchronousCrossing().sourceSync) ++ new chipyard.harness.WithSimAXIMemOverSerialTL ++ // add SimDRAM DRAM model for axi4 backing memory over the SerDes link, if axi4 mem is enabled new chipyard.config.WithSerialTLBackingMemory ++ // remove axi4 mem port in favor of SerialTL memory new freechips.rocketchip.subsystem.WithNBigCores(2) ++ new chipyard.config.AbstractConfig)
126.96.36.199. Bringup Setup of the Example Test Chip after Tapeout
Assuming this example test chip is taped out and now ready to be tested, we can communicate with the chip using this serial-link.
For example, a common test setup used at Berkeley to evaluate Chipyard-based test-chips includes an FPGA running a RISC-V soft-core that is able to speak to the DUT (over an FMC).
This RISC-V soft-core would serve as the host of the test that will run on the DUT.
This is done by the RISC-V soft-core running FESVR, sending TSI commands to a
TLSerdesser programmed on the FPGA.
Once the commands are converted to serialized TileLink, then they can be sent over some medium to the DUT
(like an FMC cable or a set of wires connecting FPGA outputs to the DUT board).
Similar to simulation, if the chip requests offchip memory, it can then send the transaction back over the serial-link.
Then the request can be serviced by the FPGA DRAM.
The following image shows this flow:
In fact, this exact type of bringup setup is what the following section discusses: Introduction to the Bringup Design.