Clarification on Bitstream Generation and FPGA Programming for Emulation

[Guha37]

Hello,
I came across the emulation setup webpage, which mentions that the bitstream for Verilog emulation is generated. I’d like to confirm whether the emulation is performed on a commercially available FPGA and could you provide any recommendations on which FPGA would be suitable for this purpose?
Moreover, my understanding is that for programming an FPGA, the bitstream must also include IO constraints specific to the target FPGA. Could you clarify how to incorporate these constraints into the bitstream?
Any guidance on the tool and procedure to be followed to incorporate these constraints into the bitstream would be greatly appreciated.
Thanks!

[KelvinChung2000]

We have run the design on Xilinx FPGA before fabrication. I believe you treat the eFPGA as a normal RTL design and synthesise it with Vivado with all the usual Xilinx flow. I have not done it myself, @IAmMarcelJung will have a better idea.

[IAmMarcelJung]

Hi @Guha37,

what is described there as "Emulation" in this case means that the Bitstream is hard-annotated to the Fabric, essentially resulting in an FPGA that is "configured" during synthesis/implementation and cannot be configured at runtime (so technically not an FPGA anymore). You can read a bit more about it in #219.

As you mentioned, you want to do an actual emulation on a commercial FPGA and as Kelvin said, this can be done by just using the generated RTL (directories Fabric and Tile in your FABulous project). The vendor or actual FGPA should not matter, but so far I personally only tried it on a Basys3 board which uses the Xilinx/AMD XC7A35T-1CPG236C FPGA and needs the Vivado software.

Sidenote: I also tried it using open source tools (Yosys, nextpnr, openxc7), but I was not successful since nextpnr-xilinx could not place LDCE cells. If you want to give it a try, I would like to here how it went!

Since the design by definition results in many combinatorial loops, I had to add the following constraints (not guaranteed to be optimal/perfect, but works for me):

Synthesis & Implementation (not sure if its actually needed for both):

set_property ALLOW_COMBINATORIAL_LOOPS true [get_nets -hierarchical -filter {NAME =~ *BEG*}]

Implementation only:

set_disable_timing [get_pins -hierarchical *Q_reg*]
set_disable_timing [get_pins -hierarchical *inferred_i_1__*]
set_disable_timing [get_pins -filter {REF_PIN_NAME =~ "*BEG*"} -of_objects [get_cells -hierarchical -quiet -filter {NAME =~ "*switch_matrix*"}]]

I would also advise to set all tiles as Out-of-Context modules to reduce the runtime. Be aware that this needs a good amount of RAM (10-16GiB) if you run them in parallel.

Then you have to instantiate eFPGA_top in your actual top module, for which you create the usual constraints as you would for any other RTL design. To get a rough estimate on how long the flow takes on my i5 8250u: ~2mins per Tile, ~5 mins for a small 4x4 fabric when the tiles where run as Out-of-Context modules.

For uploading the FABulous bitstream to the Fabric running on the FPGA the most practical way is to use a USB serial adapter and e.g. create a small python script to just transmit the bitstream. Make sure to use the correct baudrate! The current setting in config_UART.v results in a baud rate of 115200 at a clock frequency of 25MHz. So adjust either the frequency, the baud rate or the ComRate parameter in config_UART.v (or in the instantiation).

There also is this, which contains some software to upload a bitstream, but its still WIP and a bit broken.

To summarize, you have three flows: One for generating the fabric using FABulous, one for creating a bitstream for a commercial FPGA using e.g. Vivado and one for generating the bitstream for the emulated FABulous fabric FABulous, although the latter is also run when using the FABulous.tcl script.

I hope this helps a bit! If you have more questions, always feel free to ask :)

[IAmMarcelJung]

I also forgot to mention that you might have to swap out the instantiation of the blackbox sram cell in the BlockRAM_1KB and also have to add the clk_buf and break_comb_module to your vivado project. Probably you already found that out, but I still want to mention it :)

Probably we will create a separate emulation project with some examples for FPGAs from different vendors and a better documentation, so the bar to use FABulous on an actual FPGA gets lowered a bit. I myself won't have the time until may or june (and then there are also many things in my backlog...), but if you want to contribute feel free to do so!

For now I hope this thread can help people a bit to get an emulated design at least when using Vivado.

[Guha37]

We noticed the clk_buf and break_comb_module and added them. But we couldn't understand what the purpose of those blocks as they seem to connect the input to the output directly. Also, as you mentioned a better documentation would be really helpful. Thanks a lot for your responses, we are eager to setup meetings with you, and we will let you know when it would be possible for us. :)

[KelvinChung2000]

All the buffers are mainly for ASIC implementation.

The break_comb_module is for simulation purposes. Since a fabric will have many combinatorial loops, and the simulator might run into an infinite loop, those prevent situations like this.

[KelvinChung2000]

Feel free to reach out to me via email, and we can schedule something from there. My email is [email protected]

[IAmMarcelJung]

My email is [email protected] if you want to keep me in the loop :)