Bug 1004 - FPGA bring up for platform definitions
Summary: FPGA bring up for platform definitions
Status: CONFIRMED
Alias: None
Product: Libre-SOC's first SoC
Classification: Unclassified
Component: Source Code (show other bugs)
Version: unspecified
Hardware: PC Linux
: --- enhancement
Assignee: Cesar Strauss
URL:
Depends on:
Blocks: 961
  Show dependency treegraph
 
Reported: 2023-02-21 20:51 GMT by Luke Kenneth Casson Leighton
Modified: 2024-04-15 23:33 BST (History)
5 users (show)

See Also:
NLnet milestone: NLnet.2022-08-107.ongoing
total budget (EUR) for completion of task and all subtasks: 1500
budget (EUR) for this task, excluding subtasks' budget: 1500
parent task for budget allocation: 1037
child tasks for budget allocation:
The table of payments (in EUR) for this task; TOML format:
cesar=1100 lkcl=400


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Luke Kenneth Casson Leighton 2023-02-21 20:51:31 GMT
the digilent nexys-video board is one of the few that has a 200k LUTs Xilinx A7.
however it needs a nmigen-boards platform definition for the pinouts, converted
from migen.  also, nextpnr-xilinx (and associated devscripts) need to be
updated to add the 200k xilinx A7 FPGA, at present only the xc7100t and xc35t
are compiled up: the xc7200t database is also needed
Comment 2 Cesar Strauss 2023-09-08 02:31:38 BST
Summary of experimentation done so far:

1) Added support, to the developer script, for the FPGA chip on the Nexys Video  

commit b2dc3deced1a94d7305adc3301c6d9f076787652
Author: Cesar Strauss <cestrauss@gmail.com>
Date:   Mon Jul 10 18:00:24 2023 +0000

    nextpnr-xilinx-install: generate database for XC7A200T
    
    Enables bit file support for the FPGA in the Nexys Video board.

2) Patched nmigen-boards to add a small board definition, sufficient to run a "Blinky lights" design. Untested patch was not submitted upstream, pending a successful confirmation in hardware.

https://libre-soc.org/irclog/%23libre-soc.2023-08-11.log.html#t2023-08-11T10:30:46

3) Konstantinos reported upload failure, xc3prog found buggy

https://libre-soc.org/irclog/%23libre-soc.2023-08-11.log.html#t2023-08-11T11:41:05

4) Konstantinos tried it over USB. Done led lights up, but no blinking lights

https://libre-soc.org/irclog/%23libre-soc.2023-08-13.log.html#t2023-08-13T12:27:13

5) Compiled a Digilent demo with the Vivado toolchain. Konstantinos successfully
uploaded it with openocd. It means we can reliably upload a bit file, the problem does not lie there.

https://libre-soc.org/irclog/%23libre-soc.2023-08-26.log.html#t2023-08-26T13:36:10

6) Compiled a simpler verilog file, still with Vivado. Konstantinos reported it to work. Also, openFPGALoader is also able to upload a bit file, which is more convenient than openOCD.

https://libre-soc.org/irclog/%23libre-soc.2023-08-27.log.html#t2023-08-27T12:44:25

7) Compiled the previous verilog file (without nMigen) with yosys+next-pnr-xilinx. Didn't work. Toolchain confirmed non functional for the XC7A200T.

https://libre-soc.org/irclog/%23libre-soc.2023-08-27.log.html#t2023-08-27T17:57:58

8) Installed Symbiflow, an alternative toolchain. Compiled a Symbiflow demo. Konstantinos reported it to work.

https://libre-soc.org/irclog/%23libre-soc.2023-09-07.log.html#t2023-09-07T18:20:59

Conclusion: Use Symbiflow+openFPGALoader for boards with XC7A200T FPGA.

Next steps:

1) Update Symbiflow dev script to include support for XC7A200T
2) Konstantinos to reproduce the build
3) Update nmigen-boards to use the Symbiflow toolchain
4) Try nMigen Blinky
5) Try a UART demo
6) Submit pull request to include board definition on upstream nmigen-boards
7) Extend board definition to remaining peripherals
8) Try microwatt and ls2
Comment 3 Luke Kenneth Casson Leighton 2023-09-08 06:40:20 BST
(In reply to Cesar Strauss from comment #2)
> Summary of experimentation done so far:

good report.

> 44:25
> 
> 7) Compiled the previous verilog file (without nMigen) with
> yosys+next-pnr-xilinx. Didn't work. Toolchain confirmed non functional for
> the XC7A200T.

it really should not fail. if it does that needs a report upstream with a
suitable repro case. please keep interaction with upstream developers to
an absolute minimum.

> https://libre-soc.org/irclog/%23libre-soc.2023-08-27.log.html#t2023-08-27T17:
> 57:58
> 
> 8) Installed Symbiflow, an alternative toolchain. Compiled a Symbiflow demo.
> Konstantinos reported it to work.

last i checked symbiflow was awful.  many Cells within the FPGA are
unsupported (including some IO pads!), vtr was made multi-threaded and
spends 95% of its time contending for a global lock, resulting in a 7x
performance slowdown compared to nextpnr, and there are the most awful
hacks exporting multi-gigabyte files into JSON, parsing them in *python*
scripts then *re-importing* them as workarounds on carry-save propagation
which don't do the job.

then it was also incapable of
dealing with add/sub/cmp (anything using the Xilinx CARRY4 block)
greater than 96 bits in length because in each "Tile" of a XIlinx
FPGA there are ony 25 such CARRY4 blocks and symbiflow fails to
take that into account.

unless there has been a massive amount of work done, counting on symbiflow
for large designs such as ours would be a serious mistake.
Comment 4 Cesar Strauss 2023-10-03 10:15:41 BST
Konstantinos, please do:

cd ~/src/nmigen-boards
git pull
cd nmigen_boards/test/nexys_video/ledon
./prog-symbiflow.sh

Check that two leds turn on, the DONE led, and the LD0(T14) led. Then:

./prog-nextpnr-xilinx-master.sh

Please tell if these same leds still turn on, or not.

This is a minimal test case (just a led pin tied to VCC), but I need to check against nextpnr-xilinx git master before I can submit a bug report.
Comment 5 Cesar Strauss 2023-10-11 10:57:47 BST
Andrey, glad to hear you got a Nexys Video as well. Please do:

cd ~/src/nmigen-boards
git remote add cesar https://gitlab.com/cestrauss/nmigen-boards.git
git fetch cesar
git checkout digilent-gpio
cd ~/src/nmigen-boards/nmigen_boards/test/nexys_video/symbiflow

Now, make sure that the jumper JP1 is in the JTAG position and the Nexys Video is plugged into your computer via micro-USB cable into the PROG port. Turn on the board, and run:

./prog.sh

You should see a binary counter on four leds.

If it works, follow the steps on comment 4.
Comment 6 Andrey Miroshnikov 2023-10-24 17:52:04 BST
(In reply to Cesar Strauss from comment #5)
> Andrey, glad to hear you got a Nexys Video as well. Please do:
> ...

Just for clarity, I setup the chroot environment with the usual mk-deb-chroot, then inside the chroot run the following:
install-hdl-apt-reqs
hdl-dev-repos
fpga-boot-load-prog-install

After which I added your nmigen-boards digilen-gpio branch.

> Now, make sure that the jumper JP1 is in the JTAG position and the Nexys
> Video is plugged into your computer via micro-USB cable into the PROG port.

Just to clarify, I set JP4 (that's the silkscreen on my board) to connect the QSPI and JTAG. I may be wrong, so let me know (but the example worked, so my jumpers must be in right place).


> Turn on the board, and run:
> 
> ./prog.sh
> 
> You should see a binary counter on four leds.

Yep, worked well.

> 
> If it works, follow the steps on comment 4.

Will continue on to the other example.
Comment 7 Andrey Miroshnikov 2023-10-24 21:12:03 BST
(In reply to Andrey Miroshnikov from comment #6)
> Will continue on to the other example.

(In reply to Cesar Strauss from comment #4)
> Check that two leds turn on, the DONE led, and the LD0(T14) led. Then:

Both LEDs turn on.

> 
> ./prog-nextpnr-xilinx-master.sh
> 
> Please tell if these same leds still turn on, or not.

Both LEDs turn on.

> 
> This is a minimal test case (just a led pin tied to VCC), but I need to
> check against nextpnr-xilinx git master before I can submit a bug report.

Looks like these examples are working as expected.
Comment 8 Cesar Strauss 2023-10-26 11:33:58 BST
(In reply to Andrey Miroshnikov from comment #7)
> Just to clarify, I set JP4 (that's the silkscreen on my board) to connect the
> QSPI and JTAG. I may be wrong, so let me know (but the example worked, so my 
> jumpers must be in right place).

Indeed, seems correct. See the entry "JTAG", on the "Programming mode" table, on the picture:

https://digilent.com/reference/_detail/nexys-video/config_options.png?id=programmable-logic%3Anexys-video%3Areference-manual

> Looks like these examples are working as expected.

Excellent!

This might mean that we don't need Symbiflow after all, and that updating nextpnr-xilinx is all that's needed.

Let's just check that this test design doesn't really work with the outdated nextpnr-xilinx version provided by the developer scripts.

Please try:

cd ~/src/nmigen-boards
git pull
cd nmigen_boards/test/nexys_video/ledon
./build.sh
./prog.sh

I expect it not to work (just the DONE led turns on), but we never know...
Comment 9 Andrey Miroshnikov 2023-11-01 09:54:08 GMT
(In reply to Cesar Strauss from comment #8)
> Let's just check that this test design doesn't really work with the outdated
> nextpnr-xilinx version provided by the developer scripts.
> 
> Please try:
> 
> cd ~/src/nmigen-boards
> git pull
> cd nmigen_boards/test/nexys_video/ledon
> ./build.sh
> ./prog.sh
> 
> I expect it not to work (just the DONE led turns on), but we never know...

Had to run the yosys and nextpnr-xilinx devscripts as well:
hdl-tool-yosys
nextpnr-xilinx-install
As user, export variables for prjxray toolchain:
export PATH=/usr/local/nextpnr-xilinx/bin:$PATH
export XRAY_DIR=/usr/local/nextpnr-xilinx


And it worked! Both LEDs go on (DONE and LD0).
Comment 10 Cesar Strauss 2023-11-04 14:48:13 GMT
(In reply to Andrey Miroshnikov from comment #9)
> And it worked! Both LEDs go on (DONE and LD0).

Interesting.

As it turns out, I made a mistake. The schematics show the LED voltage being 2,5V not 3,3V. I fixed it in the branch. Let's see if it still works:

cd ~/src/nmigen-boards
git pull
cd nmigen_boards/test/nexys_video/ledon
./build.sh
./prog.sh
Comment 11 Andrey Miroshnikov 2023-11-05 17:06:03 GMT
(In reply to Cesar Strauss from comment #10)
> As it turns out, I made a mistake. The schematics show the LED voltage being
> 2,5V not 3,3V. I fixed it in the branch. Let's see if it still works:
Now just DONE LED lights up, looks like the fix worked.
Comment 13 Cesar Strauss 2024-03-24 18:00:55 GMT
I got Blinky to work on the Nexys Video!

It seems the leds didn't light up before because the 2,5V I/O buffers were only recently implemented in nextpnr-xilinx, and our version was too old. Updating it to current git master worked.

However this brings a massive regression when building ls2 for Arty... Routing no longer converges on that design, it goes on forever.

Some options are:

1) Don't do anything, and forget about the leds on the Nexys Video (also, Ethernet and HDMI).
2) Update nextpnr-xilinx to the commit that fixed the leds
3) Update nextpnr-xilinx to the last commit that still worked for ls2
4) Backport just the led fix, to the commit we currently use.

As for #3, I did find a fork of nextpnr-xilinx called openXC7 (https://github.com/openXC7/nextpnr-xilinx), which seems actively developed, and whose latest release does work with both leds and ls2. In fact, for ls2 on the Arty, it increases the maximum clock speed from 19 MHz to 25 MHz...

Also, someone would need to run the bitsream on a real Arty, to see if the update really works.
Comment 14 Luke Kenneth Casson Leighton 2024-03-24 22:23:15 GMT
(In reply to Cesar Strauss from comment #13)
> I got Blinky to work on the Nexys Video!

hooray!

> It seems the leds didn't light up before because the 2,5V I/O buffers were
> only recently implemented in nextpnr-xilinx, and our version was too old.
> Updating it to current git master worked.
> 
> However this brings a massive regression when building ls2 for Arty...
> Routing no longer converges on that design, it goes on forever.

worth reporting, with repo instructions.


> Some options are:
> 
> 1) Don't do anything, and forget about the leds on the Nexys Video (also,
> Ethernet and HDMI).
> 2) Update nextpnr-xilinx to the commit that fixed the leds
> 3) Update nextpnr-xilinx to the last commit that still worked for ls2
> 4) Backport just the led fix, to the commit we currently use.
> 
> As for #3, I did find a fork of nextpnr-xilinx called openXC7
> (https://github.com/openXC7/nextpnr-xilinx), which seems actively developed,
> and whose latest release does work with both leds and ls2.

if that is easily a drop-in replacement, same commands, it may be
worth adding and commenting a link to this bugreport. they *may*
take merge/fixes upstream at some point

> In fact, for ls2
> on the Arty, it increases the maximum clock speed from 19 MHz to 25 MHz...

niice

> Also, someone would need to run the bitsream on a real Arty, to see if the
> update really works.

i have mine buried somewhere in a box... :)
Comment 15 Cesar Strauss 2024-03-26 10:13:54 GMT
(In reply to Luke Kenneth Casson Leighton from comment #14)
> (In reply to Cesar Strauss from comment #13)
> > However this brings a massive regression when building ls2 for Arty...
> > Routing no longer converges on that design, it goes on forever.
> 
> worth reporting, with repo instructions.

Sure. I think the openXC7 project is already aware of regressions upstream regarding routing, they mention:

"This release backports all the latest changes on the old, stable codebase,
giving you the best stability and performance. Further development will be made to both branches, until the upstream branches' router2 issues (like endless loops), will be solved."

> if that is easily a drop-in replacement, same commands, it may be
> worth adding and commenting a link to this bugreport.

It is a direct drop-in replacement yes, even build instructions are the same, just adjust the git repo and hash.

Even Litex now accepts "openxc7" as an alternate toolchain for Xilinx devices.

> they *may* take merge/fixes upstream at some point

The author seems to be a major contributor upstream, many of their pull requests are already there.
Comment 16 Cesar Strauss 2024-04-03 02:02:13 BST
> Also, someone would need to run the bitsream on a real Arty, to see if the
> update really works.

Jacob tried:

1) ls2 with Microwatt's "hello world", built with the old nextpnr-xilinx, as installed by the current devscripts:

https://ftp.libre-soc.org/openxc7-ls2/ls2_nextpnr_xilinx.bit

2) same, built with openXC7 0.7.0:

https://ftp.libre-riscv.org/openxc7-ls2/ls2_openxc7.bit

Both gave identical results, so I'll go ahead and update the devscripts.

See: https://libre-soc.org/irclog/latest.log.html#t2024-04-03T00:01:10
Comment 17 Cesar Strauss 2024-04-07 13:23:28 BST
Jacob's UART demo works as well.

I created a pull request against nmigen-boards to merge my work so far.

https://gitlab.com/nmigen/nmigen-boards/-/merge_requests/6

Luke, please review and merge when convenient.

For the moment it is enough for running the Blinky and UART demos on the Nexys Video. It should be able to run ls2 as well, I'll try that next.

After that, I'll add the remaining definitions for SDRAM, Ethernet, PMODs, etc.
Comment 18 Luke Kenneth Casson Leighton 2024-04-07 23:42:08 BST
(In reply to Cesar Strauss from comment #17)
> Jacob's UART demo works as well.

fantastic.

> 
> I created a pull request against nmigen-boards to merge my work so far.
> 
> https://gitlab.com/nmigen/nmigen-boards/-/merge_requests/6
> 
> Luke, please review and merge when convenient.

willdo. tank you c3sar
Comment 19 Cesar Strauss 2024-04-08 10:49:30 BST
Got ls2 working on the Nexys Video!

UART only, no SPI, SDRAM, etc.

My steps, adapted from https://libre-soc.org/HDL_workflow/ls2/

cd ~/src/soc
make microwatt_external_core_bram_arty
cp external_core_top.v ../ls2
cd ../ls2
export PATH=/usr/local/nextpnr-xilinx/bin:$PATH
export XRAY_DIR=/usr/local/nextpnr-xilinx
python3 src/ls2.py nexys_video coldboot/coldboot.bin

Output on serial console:

 Soc signature: 00010001F00DAA55  Soc features: UART 
done

I'll push my local changes to ls2 once Nexys Video support is merged into nmigen-boards.
Comment 20 Cesar Strauss 2024-04-14 21:22:39 BST
Added Nexys Video support (UART only) to ls2:

commit 16e1f4c799d8bc059f6ef6868b31a2ba8a40c677
Author: Cesar Strauss <cestrauss@gmail.com>
Date:   Sun Apr 14 15:51:43 2024 -0300

    ls2: add support for the Nexys Video board

https://git.libre-soc.org/?p=ls2.git;a=commitdiff;h=16e1f4c799d8bc059f6ef6868b31a2ba8a40c677

For now, the import is guarded with try...catch, while my pull request to nmigen-boards waits to be merged.

I also added support on Jacob's UART demo, but it lives in a branch in utils.git, and I don't have write permission for it.

Will continue working on the platform definition for nmigen-boards, and updating the pull request.
Comment 21 Luke Kenneth Casson Leighton 2024-04-14 22:41:58 BST
(In reply to Cesar Strauss from comment #20)

> 
> For now, the import is guarded with try...catch, while my pull request to
> nmigen-boards waits to be merged.

i need to add you.

> I also added support on Jacob's UART demo, but it lives in a branch in
> utils.git, and I don't have write permission for it.

sorted

> Will continue working on the platform definition for nmigen-boards, and
> updating the pull request.

star
Comment 22 Cesar Strauss 2024-04-15 23:33:27 BST
* Added support for the Nexys Video on Jacob's UART demo.

https://git.libre-soc.org/?p=utils.git;a=commitdiff;h=1ed2c3bef98679fd5368fe6785b86b13143bd07b

* Added OLED and HDMI IN/OUT to the Nexys Video platform definition, and updated pull request:

https://gitlab.com/nmigen/nmigen-boards/-/merge_requests/6/diffs

The Digilent Genesys 2 board already has a nmigen definition on nmigen-boards, and has similar peripherals, which is helping.