Ways to use BFM continued from Issues

[thrakkor]

The idea behind the PCIe hard IP core interface modules in this repo is that you connect them to your design in place of the hard IP core instance itself. So you would have your user logic that has CQ/CC/RQ/RC streaming interfaces (HDL) -> Xilinx US+ PCIe HIP model in this repo (Python) -> root complex model in this repo (Python) -> your testbench/driver model code (Python). Take a look at how it's used in https://github.com/corundum/corundum as that's the most comprehensive example at the moment.

Understood. While not impossible to rearrange like this, in my design, the HIP is buried down inside my EP hierarchy and segregated to eliminate AXI from the rest of the design (I can't stand AXI and use WB or simple FIFO interfaces or a more spec pure streaming TLP record throughout).

In the connections you described above, I am wondering if there is a way to bypass the HIP model in the repo and translate the root complex TLPs to the Xilinx descriptors, which would allow the root complex model in the repo to drive the Xilinx BFM's HIP (eliminating the crappy Rx/Tx APP modules). (I am still trying to digest how the above pieces talk to each other, so this may have no hope of working).

MMIO, DMA, P2P DMA, and MSI interrupts are all currently supported, and you can absolutely set the max read request and max payload size to whatever you want. And MSI interrupts are properly caught and you can handle them either as events or callbacks.

excellent.

Eventually I'm planning on supporting PIPE so you can use this library to talk to the actual hard IP core model, but it will be some time before that's possible. I have to implement quite a bit more of the PCIe protocol stack before that will be possible.

when you do this, will you be adding support for both Xilinx and Altera PIPE interfaces?

I also just realized that I forgot to turn on discussions on this repo; that's now active.

[alexforencich]

I have no idea how the Xilinx BFM works, unfortunately. I have been using open source simulators that are not capable of running either the PCIe IP core model or the Xilinx root complex BFM. The IP core model in the repo here basically just converts the internal TLP objects to and from the Xilinx descriptors and then drives those into the design through the simulator interface. There is a bit more going on inside the model, but that's the most important part of it. It may be possible to connect the model to ports buried in the hierarchy so you don't have to pull the AXI interfaces all the way up to the top level, but I have not tried this myself as what I do is put the hard IP core instance in the top-level (fpga.v) and then only simulate the core logic (fpga_core.v).

As far as PIPE is concerned, isn't it an intel standard so it should be vendor-independent? At any rate, if Xilinx and Altera IP cores implement it slightly differently, then it wouldn't be difficult to create a couple of different variations to support both. The hard part is all of the link layer and physical layer functionality that needs to be implemented, once that's done then implementing a couple of different flavors of PIPE wouldn't be that big of a deal.

[thrakkor]

I have no idea how the Xilinx BFM works, unfortunately. I have been using open source simulators that are not capable of running either the PCIe IP core model or the Xilinx root complex BFM. The IP core model in the repo here basically just converts the internal TLP objects to and from the Xilinx descriptors and then drives those into the design through the simulator interface. There is a bit more going on inside the model, but that's the most important part of it. It may be possible to connect the model to ports buried in the hierarchy so you don't have to pull the AXI interfaces all the way up to the top level, but I have not tried this myself as what I do is put the hard IP core instance in the top-level (fpga.v) and then only simulate the core logic (fpga_core.v).

The current XUS RP BFM is basically the same HIP bolted up to some modules containing dozens of VLOG tasks to interface to the RQ/CQ/RC/CC interfaces. They go thru link bring up, init, BAR cfg, etc. CFG and Mem accesses to EP are provided. It has some auto completion generation when MemRd requests are received from EP. Some expected data tasks. Things like that. But, it could be much better.

user EP/HIP <-PIPE-> Xil RP HIP <-RQ/CQ/RC/CC-> Xilinx RP BFM VLOG Tasks

As far as PIPE is concerned, isn't it an intel standard so it should be vendor-independent? At any rate, if Xilinx and Altera IP cores implement it slightly differently, then it wouldn't be difficult to create a couple of different variations to support both. The hard part is all of the link layer and physical layer functionality that needs to be implemented, once that's done then implementing a couple of different flavors of PIPE wouldn't be that big of a deal.

not sure about the details of PIPE, other than I have used on Altera (Arria10, Arria5) and Xilinx (V6, KUS) and they look a bit different to me, at least at the top level.

my ultimate goal is to not only simulate the AXI CQ/RQ/CC/RC interfaces, but the actual HIP in my EP connected to a RP. The closer I can make my simulation to the HW, the better. I realize without the right simulator, this is not possible.

I think an awesome way to use what you have would be to bolt the RP model to the same interface the Xilinx RP APP uses, which is just the user CQ/RQ/CC/RC interface. I also realize this is far, far from trivial. Maybe a future enhancement or stop gap until the PIPE version of your stuff is in place.

[alexforencich]

Oh I see, that makes a lot more sense now; Xilinx is using a second hard IP core model in root port mode, cross-connecting that with a model in endpoint mode, and then driving the CQ/CC/RQ/RC interfaces on the root port model with their root complex model. Well, I think a lot of the pieces are already in the repo, it's just a matter of connecting them together properly. Currently, I don't have a root port version of the hard IP core model, but in order to test such a model I would also need to make an interface module that would connect to the root complex model to the Xilinx hard IP core interface. If I make such a model, then it sounds like it could work for your setup as a drop-in replacement for the Xilinx root complex model.