(In reply to Andrey Miroshnikov from comment #0)
> The nmigen Python module also depends on pyvcd and toml.
> I used toml from the debian sources, while pyvcd came from github (0.4.0
> tag, latest).
> https://git.libre-soc.org/?p=dev-env-setup.git;a=commitdiff;
> h=b63c7c26cc938ebd10eaeeaebc431b0a2e41d4c5

in case it helps any i'm on pyvcd 0.2.4 and don't recall running into any problems caused by pyvcd.

(In reply to Jacob Lifshay from comment #1)
> (In reply to Andrey Miroshnikov from comment #0)
> > The nmigen Python module also depends on pyvcd and toml.
> > I used toml from the debian sources, while pyvcd came from github (0.4.0
> > tag, latest).
> > https://git.libre-soc.org/?p=dev-env-setup.git;a=commitdiff;
> > h=b63c7c26cc938ebd10eaeeaebc431b0a2e41d4c5
> 
> in case it helps any i'm on pyvcd 0.2.4 and don't recall running into any
> problems caused by pyvcd.

note that afaict nmigen requires a 0.2.x version of pyvcd, not 0.4.0
> "pyvcd~=0.2.2",

(In reply to Andrey Miroshnikov from comment #0)

> The nmigen Python module also depends on pyvcd and toml.
> I used toml from the debian sources, while pyvcd came from github (0.4.0
> tag, latest).

which will be why it failed.

> 
> Hitting an issue during making of the external Libre-SOC core, but will
> re-create chroot to make sure I didn't stray from the devscripts.

always always always use the devscripts.

please DON'T use latest versions

+# pyvcd - needed for nmigen
+cd pyvcd
+git checkout 0.4.0 # latest version tested
+python3 setup.py develop --no-deps
+

eek please don't make arbitrary modifications to the devscripts
without first verifying FULLY that there is no damage done,
this is so critically important.

even if it takes days to re-run you MUST be patient and not
make arbitrary unverified changes.

(In reply to Jacob Lifshay from comment #1)
> (In reply to Andrey Miroshnikov from comment #0)
> in case it helps any i'm on pyvcd 0.2.4 and don't recall running into any
> problems caused by pyvcd.

pip3 list | grep vcd
pyvcd                           0.2.3

andrey please back that down to 0.2.4 real fast.

anyone who uses the scripts will find that they likely FAIL
causing us "support" issues and affecting our reputation.

(In reply to Luke Kenneth Casson Leighton from comment #3)
> 
> please DON'T use latest versions

Sorry about that.

> 
> andrey please back that down to 0.2.4 real fast.
> 
> anyone who uses the scripts will find that they likely FAIL
> causing us "support" issues and affecting our reputation.

Changed to 0.2.4:
https://git.libre-soc.org/?p=dev-env-setup.git;a=commitdiff;h=6019036dc70bde41ff284f729f8e896ce88c723d

(In reply to Andrey Miroshnikov from comment #4)

> Changed to 0.2.4:
> https://git.libre-soc.org/?p=dev-env-setup.git;a=commitdiff;
> h=6019036dc70bde41ff284f729f8e896ce88c723d

wheww, thank you andrey.  at some point that'll have to be
updated but it will require scheduling some additional time
to check it.

After running through the setup scripts in the following order (after creating chroot and copying over the dev scripts):
./install-hdl-apt-reqs
./hdl-dev-repos
./hdl-tools-yosys
./nextpnr-xilinx-install

I was able to make the external Libre-SOC core.

Looking at the log of nextpnr-xilinx-install, I saw that antlr parser was not present. There's an automatic fallback to textX parser, but I haven't tested this software (not working on FPGA yet).

After running hdl-dev-ls2, I noticed the following in the log:

- The nmigen got overwritten (was at nmigen 0.1.dev1205+g29dec30, which isn't good, given that we needed 0.2). An older nmigen version being installed by hdl-dev-repos was also a problem I encountered back in January:
https://libre-soc.org/irclog/%23libre-soc.2023-01-16.log.html#t2023-01-16T19:16:58

- Pyvcd was overwritten from 0.2.4 to 0.1.7.

From this point I was also not able to compile the external Libre-SOC core anymore because of older Pyvcd:
ImportError: cannot import name 'GTKWColor' from 'vcd.gtkw' (/usr/local/lib/python3.7/dist-packages/pyvcd-0.1.7-py3.7.egg/vcd/gtkw.py)



I uploaded the logs in a tar.gz file (20220521_ls2_scriptlogs.tar.gz) in my home directory of libre-soc.org (should be readable). I only included the successful run of make microwatt_external_core.


NOTE: All the scripts were run *UNMODIFIED* (latest git commit 7e125043b8ac0030a47188f85c1e2128f8ae6cec), so you should be able to replicate this.

(In reply to Andrey Miroshnikov from comment #6)
> After running through the setup scripts in the following order (after
> creating chroot and copying over the dev scripts):
> ./install-hdl-apt-reqs
> ./hdl-dev-repos
> ./hdl-tools-yosys
> ./nextpnr-xilinx-install
> 
> I was able to make the external Libre-SOC core.
> 
> Looking at the log of nextpnr-xilinx-install, I saw that antlr parser was
> not present. There's an automatic fallback to textX parser, but I haven't
> tested this software (not working on FPGA yet).

as long as it "works" it's good.

> After running hdl-dev-ls2, I noticed the following in the log:
> 
> - The nmigen got overwritten (was at nmigen 0.1.dev1205+g29dec30, which
> isn't good, given that we needed 0.2). 

remember i mentioned about pip3 performing tasks without consent and
without informing you?

so that is almost certainly this line:

  28         "nmigen>=0.1,<0.5",

https://git.libre-soc.org/?p=lambdasoc.git;a=blob;f=setup.py;h=2fca63720db579d4f7e14584d3c84c0a1e7bbd05;hb=HEAD

for now just comment all of these out:

  28         "nmigen>=0.1,<0.5",
  29         "nmigen-soc",
  30         "nmigen-stdio",
  31         "nmigen-boards",

and also these:

  34     install_requires=['nmigen', 'nmigen_boards'],

https://git.libre-soc.org/?p=gram.git;a=blob;f=setup.py;h=2243981d2c1c298e0ba8750a425bf5ed9bf32aff;hb=0b606145a2febf3ea93a5f345a316a147f74ac73

you'll then need to re-run the entire lot again. patience needed, yes
it's an O(N^2) process.

(In reply to Luke Kenneth Casson Leighton from comment #7)
>
> remember i mentioned about pip3 performing tasks without consent and
> without informing you?
> 
> so that is almost certainly this line:
> 
>   28         "nmigen>=0.1,<0.5",
> 
> https://git.libre-soc.org/?p=lambdasoc.git;a=blob;f=setup.py;
> h=2fca63720db579d4f7e14584d3c84c0a1e7bbd05;hb=HEAD
> 
> for now just comment all of these out:
> 
>   28         "nmigen>=0.1,<0.5",
>   29         "nmigen-soc",
>   30         "nmigen-stdio",
>   31         "nmigen-boards",
> 
> and also these:
> 
>   34     install_requires=['nmigen', 'nmigen_boards'],
> 
> https://git.libre-soc.org/?p=gram.git;a=blob;f=setup.py;
> h=2243981d2c1c298e0ba8750a425bf5ed9bf32aff;
> hb=0b606145a2febf3ea93a5f345a316a147f74ac73
> 
> you'll then need to re-run the entire lot again. patience needed, yes
> it's an O(N^2) process.

Thanks. Doesn't matter if its slow, at least I know how to proceed now :)

I'll continue with it tomorrow.

> Thanks. Doesn't matter if its slow, at least I know how to proceed now :)
> 
> I'll continue with it tomorrow.

Removed dependencies on nmigen and its companion packages in lambdasoc and gram repos:
https://git.libre-soc.org/?p=lambdasoc.git;a=commitdiff;h=d15878b945b012dc4047f7b5a4a761a603789baa
https://git.libre-soc.org/?p=gram.git;a=commitdiff;h=35dab5291968e60ffa5290c1032cac290676b311

Will update on the results tomorrow.

(In reply to Andrey Miroshnikov from comment #9)
> Will update on the results tomorrow.

Re-ran the devscripts.
Looking back at IRC log from January 17th, saw Luke's thought on running hdl-tools-yosys before hdl-dev-repos, and so I did:
https://libre-soc.org/irclog/%23libre-soc.2023-01-17.log.html#t2023-01-17T16:08:54

After running hdl-tools-yosys, no nmigen was installed (good, expected behaviour).

After running hdl-dev-repos, I was able to build the soc successfully. All the nmigen libs were still at 0.1 (which both Jacob and Luke mentioned in the above IRC log is an older version than the 0.3 or 0.2). Is this a problem?

c4m-jtag               0.1.dev152+gf5322d8 
nmigen                 0.1.dev1205+g29dec30         
nmigen-boards          0.1.dev217+ga949308  
nmigen-soc             0.1.dev53+gfd2aaa3     
nmigen-stdio           0.1.dev7+g475cb6f
pyvcd                  0.2.4

The hdl-dev-ls2 added the other modules without changing nmigen or pyvcd (good, expected behaviour):
gram                   0.1.dev546+g35dab52
lambdasoc              0.1.dev86+gd15878b


I tried running 'make clean' of the soc repo (before re-generating external core), I get an error related to a depricated Jinja function.
ImportError: cannot import name 'contextfunction' from 'jinja2' (/usr/local/lib/python3.7/dist-packages/Jinja2-3.1.2-py3.7.egg/jinja2/__init__.py)
make: *** [Makefile:127: clean] Error 1

The version pip installs is this:
Jinja2                 3.1.2

After a quick search, this github issue shows that 'contextfunction' has been depricated since Jinja 3.0, but could be bodged by using the newer function 'pass_context':
(see https://github.com/tgen/jetstream/issues/139)

Turns out the Makefile doesn't have a 'clean' rule, and it's actually running 
" Catch-all target: route all unknown targets to Sphinx using the new
"make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS)."
I guess this is for document generation?


Generating the FPGA bitstream with hello_world worked and I was able to run it on fpga. The flow up to this point has been validated (will add fpga-boot-load-prog-install script to the wiki page).
I'll continue with the verilator sim.

(In reply to Andrey Miroshnikov from comment #10)

> c4m-jtag               0.1.dev152+gf5322d8 
> nmigen                 0.1.dev1205+g29dec30         
> nmigen-boards          0.1.dev217+ga949308  
> nmigen-soc             0.1.dev53+gfd2aaa3     
> nmigen-stdio           0.1.dev7+g475cb6f

all from git tags. good.

jinja2 may need install from source and/or specific version.
should be fine with deb 10 packages.

(In reply to Luke Kenneth Casson Leighton from comment #11)
> 
> jinja2 may need install from source and/or specific version.
> should be fine with deb 10 packages.

The python3-jinja2 (version 2.10-2) already comes from deb10 oldstable, so probably need to compile from source. I won't pursue it for now however.


As for verilator sim, additional dev script needs to be called
./verilator-install

Then compiling hello world binary to start at 0xff000000.

Then calling
python3 src/ls2.py sim /tmp/ff000000_hw.bin
to generate the ls2.v

Calling 'make microwatt-verilator' fails
%Error: Exiting due to too many errors encountered; --error-limit=50

The errors that are occuring at to do with the timescale:
%Error-TIMESCALEMOD: ls2.v:5:8: Timescale missing on this module as other modules have it (IEEE 1800-2017 3.14.2.2)
%Error-TIMESCALEMOD: ls2.v:1285:8: Timescale missing on this module as other modules have it (IEEE 1800-2017 3.14.2.2)
%Error-TIMESCALEMOD: ls2.v:1370:8: Timescale missing on this module as other modules have it (IEEE 1800-2017 3.14.2.2)
...

All 50 occurred on uart_top (but I'm guessing this error would occur elsewhere if the error limit was higher).

Examples for where errors occurred on lines 5, 1285, 1370 of generated ls2.v:
module \U$$0$10 (clk, sink__valid, sink__ready, sink__payload__we, sink__payload__addr, source__valid, source__ready, source__first, source__last, source__payload__we, source__payload__addr, rst);
module \U$$0 (clk, activate, activate_row, precharge, write_data, write_mask, write, write_col, read, read_col, read_data, rst);
module \U$$0$17 (clk, sink__valid, sink__ready, sink__payload__we, sink__payload__addr, source__valid, source__ready, source__first, source__last, source__payload__we, source__payload__addr, rst);

Is there something missing during the ls2.py script?

(In reply to Andrey Miroshnikov from comment #12)
>
> Calling 'make microwatt-verilator' fails
> %Error: Exiting due to too many errors encountered; --error-limit=50
> 
> The errors that are occuring at to do with the timescale:
> %Error-TIMESCALEMOD: ls2.v:5:8: Timescale missing on this module as other
> modules have it (IEEE 1800-2017 3.14.2.2)

After hunting through the mailing list I found that Tobias went through a similar experience trying to get ls2 verilator sim working:
https://lists.libre-soc.org/pipermail/libre-soc-dev/2022-April/004778.html
https://lists.libre-soc.org/pipermail/libre-soc-dev/2022-April/004779.html
https://libre-soc.org/irclog/%23libre-soc.2022-04-24.log.html#t2022-04-24T14:23:30

To check, I commented out lines #78-79:
https://git.libre-soc.org/?p=ls2.git;a=blob;f=Makefile;h=a41fe72aae852011623794b808f4db85d6332e5c;hb=426e2d9585cd4b1fb96a38987f97878285ee5ba7

So the verilator command must be failing. Any ideas and/or past experience as to why it might be failing?

Before the timescalemod errors, there was one warning about a missing pin:
%Warning-PINMISSING: ls2.v:47044:21: Cell has missing pin: 'insn'
47044 |   external_core_top core_top (
      |                     ^~~~~~~~
                     ... Use "/* verilator lint_off PINMISSING */" and lint_on around source to disable this message.

But this shouldn't be causing the program to stop.

Tobias had not mentioned how he fixed it.

(In reply to Andrey Miroshnikov from comment #13)

> 
> But this shouldn't be causing the program to stop.
> 
> Tobias had not mentioned how he fixed it.

put it on the list otherwise he will not see this.

(In reply to Andrey Miroshnikov from comment #13)
> (In reply to Andrey Miroshnikov from comment #12)
> >
> > Calling 'make microwatt-verilator' fails
> > %Error: Exiting due to too many errors encountered; --error-limit=50
> > 
> > The errors that are occuring at to do with the timescale:
> > %Error-TIMESCALEMOD: ls2.v:5:8: Timescale missing on this module as other
> > modules have it (IEEE 1800-2017 3.14.2.2)
>
> ...snip...
> 
> Tobias had not mentioned how he fixed it.

Finally figured it out... The Makefile was missing the "-Wno-TIMESCALEMOD" flag.

I did see Tobias' commit which added this flag (dated 26th April 2022):
https://git.libre-soc.org/?p=ls2.git;a=commitdiff;h=0ed190756075447abdf96cb7e508e7ed92118236

But I didn't spot that the commit being used by the ls2 tutorial is from the 4th April 2022:
https://git.libre-soc.org/?p=ls2.git;a=commitdiff;h=426e2d9585cd4b1fb96a38987f97878285ee5ba7

Now that I added the flag, verilator is producing the .cpp files correctly.

Familiarising myself with the gtkwave traces for now, as the hello world didn't run as expected.

(In reply to Andrey Miroshnikov from comment #15)
> (In reply to Andrey Miroshnikov from comment #13)

> But I didn't spot that the commit being used by the ls2 tutorial is from the
> 4th April 2022:
> https://git.libre-soc.org/?p=ls2.git;a=commitdiff;
> h=426e2d9585cd4b1fb96a38987f97878285ee5ba7

now you know why the hard rule is "follow the instructions".
it applies to *everything*.

if you had started - right at the beginning - from that known-good
position it would have saved you what... two weeks at least?

and you could have gone sequentially through diffs (or use binary-search)
*because* you had a known-good position.

Copying in previous chats:
https://libre-soc.org/irclog/%23libre-soc.2023-06-03.log.html#t2023-06-03T13:38:21

On the 3rd June, mentioned to Luke that the ls2 verilator sim shows the uart peripheral's tx/rx are not being driven. I noticed this because the hello world wasn't being printed to stdout.

I haven't had a chance to continue debugging this, but found bugs that may be relevant to the issue.

First one is bug #795 which encountered 'no console output from ls2 builds' and is related to a define in the uart16550 src code.

Another is bug #812, where Luke mentioned that the core needs to be generate either with spi or bram, to get the reset vector 0xff000000.

I'll look over the commits (which I've already started doing, in order to eventually bring ls2 up to latest commit) and try the suggestions in those bugs.