Bug 412 - set up litex for peripherals and linking to core
Summary: set up litex for peripherals and linking to core
Alias: None
Product: Libre-SOC's first SoC
Classification: Unclassified
Component: Source Code (show other bugs)
Version: unspecified
Hardware: Other Linux
: High enhancement
Assignee: Luke Kenneth Casson Leighton
Depends on: 414
Blocks: 22 383
  Show dependency treegraph
Reported: 2020-07-02 13:53 BST by Luke Kenneth Casson Leighton
Modified: 2021-04-20 15:12 BST (History)
1 user (show)

See Also:
NLnet milestone: NLnet.2019.02.012
total budget (EUR) for completion of task and all subtasks: 2000
budget (EUR) for this task, excluding subtasks' budget: 2000
parent task for budget allocation: 22
child tasks for budget allocation:
The table of payments (in EUR) for this task; TOML format:
"florent" = {amount=800, paid=2020-09-04} "lkcl" = {amount=1200, paid=2020-09-04}

first version core.py for litex libre-soc (4.90 KB, text/x-python)
2020-07-20 17:33 BST, Luke Kenneth Casson Leighton

Note You need to log in before you can comment on or make changes to this bug.
Description Luke Kenneth Casson Leighton 2020-07-02 13:53:01 BST
litex seems to be a common standard way to set up socs. both microwatt and minerva use it.  the combination of both should get us started.
Comment 1 Luke Kenneth Casson Leighton 2020-07-02 14:15:46 BST
not quite the same thing, may prove useful.
Comment 3 Luke Kenneth Casson Leighton 2020-07-12 16:31:20 BST
(In reply to Luke Kenneth Casson Leighton from comment #1)
> https://git.m-labs.hk/M-Labs/HeavyX/src/branch/master/examples/
> simplesoc_ecp5.py
> not quite the same thing, may prove useful.

contains a UART example which should do well after adaptation, for the helloworld test.
Comment 4 Luke Kenneth Casson Leighton 2020-07-18 15:30:59 BST
hi florent, thank you for offering to help with this.


* we have a POWER9 compliant core written in nmigen
* therefore we need a hybrid combination of how both microwatt
  and Minerva are set up
* the do_finalize from "litex/soc/cores/cpu/minerva/core.py"
* the reset_address gcc_flags and setup from
* the wishbone buses are *64* bit data
* we use the *Minerva* Load/Store and Fetch code therefore the
  ibus and dbus can be set up according to how *Minerva* does it,
  *not* the "stall" method from microwatt.
* we actually support big *and* little-endian however please assume
  little for now.
* we do not unfortunately have a Debug bus (examining microwatt core.py)
* nor interrupts (yet)

now, i did notice that there is a general practice of creating a
repository that takes a *copy* of the core, in a repository named

the libre-soc codebase is so large (45,000 for the IEEE754 FP unit,
20,000 for the main core, 5,000 for the support library, nmutil)
that it would not only be impractical, it would be detrimental to
development to follow this pattern.

if it is absolutely necessary to have this subdirectory (named
pythondata-cpu-libre-soc) then what i would advocate is that we
add such a subdirectory *DIRECTLY* to the libre-soc main git
repository, populate it with the necessary files, and for development
purposes create a symbolic link in the litex directory to the
libre-soc directory, pythondata-cpu-libre-soc.

i am happy to give you commit access to the relevant libre-soc git
repositories, if you send me an id_rsa.pub.
Comment 5 Luke Kenneth Casson Leighton 2020-07-18 15:32:07 BST
copy of test_issuer.v, generated with
$ python3 issuer.py generate -t v > test_issuer.v
(followed by hand-editing to get rid of some debug print statements)

Comment 7 Luke Kenneth Casson Leighton 2020-07-19 21:00:34 BST
> To be able to do the LiteX integration and use the different features i need from you:
> - the nmigen command to run to generate the top level of the CPU (is it what test_issuer is doing?)

yes, although it spews additional debug output at the moment.  let us instead go with "soc/simple/issuer_verilog.py filename.v"


> - a description of the different signals of the CPU and the ones mandatory/ the ones optional.


these should be exactly as in Minerva: i really did copy the code as-is (except to make it 64 bit)

hmm however it seems i have some additional signals that get it "going".  a sort-of debug interface, these are, mandatory:

* go_insn_i

these are optional:

* pc_i
* pc_i_ok
* core_start_i
* core_stop_i
* core_bigendian_i
* halted_o
* busy_o

> - the example you already simulated with the RAM and code from Microwatt.


line 40 is where the unit test is added to the list of binaries to run.

line 89 shows how to kick it out of "halted"

lines at 119 show how to set the pc externally.  it defaults to zero including after reset so this is not strictly necessary.

lines at 124 show how to kick it into running instructions.  go_insn_i setting to 1 is mandatory

when we have a "proper" debug interface (JTAG, other) this will be a little different

if you can help get us started i can send you patches.
Comment 8 Luke Kenneth Casson Leighton 2020-07-19 22:23:38 BST
florent, i think it is possible to literally copy minerva cpu_params, the full set of ibus and dbus names exactly the same, but removing interrupts (for now).

and i believe this should do the trick of allowing the core to start running immediately from the default pc=0

    self.cpu_params = dict(
          i_go_insn_i = 1
Comment 9 Luke Kenneth Casson Leighton 2020-07-20 17:33:31 BST
Created attachment 74 [details]
first version core.py for litex libre-soc

this is a first cut at combining minerva and microwatt to create
a libre-soc litex core.py.  taking minerva i/d cache cpu signal
definitions, widening them to 64 bit, taking microwatt's ppc64le
flags and so on.
Comment 10 Luke Kenneth Casson Leighton 2020-07-20 18:32:09 BST
It's possible to plug your CPU to LiteX without modify LiteX itself, to so do, you could have a look at https://github.com/enjoy-digital/litex_vexriscv_smp and see how the CPU wrapper/software is created in the vexriscv_smp repository.

You can also look at https://github.com/enjoy-digital/litex_vexriscv_smp/blob/master/sim.py for a Verilator simulation. (capable of loading binaries and booting linux in the case of VexRiscv SMP).

That will probably be easier for you to develop like this, this will avoid forking LiteX. Once you have something that compiles correctly with Verilator (just execute sim.py on the case of VexRiscv SMP), I could help you getting this running if this is not already the case. (You can add --trace to the simulation to generate the waveform).
Comment 11 Luke Kenneth Casson Leighton 2020-07-22 16:31:41 BST
added to soc for now, following Florian's advice

commit 7ef2953a3787258d218941d1ab3b91d705670248 (HEAD -> master)
Author: Luke Kenneth Casson Leighton <lkcl@lkcl.net>
Date:   Wed Jul 22 16:30:35 2020 +0100

    first version of litex core (to be submitted upstream once tested)
Comment 12 Luke Kenneth Casson Leighton 2020-07-22 20:52:45 BST
encountering this error (which was first encountered when using
platform "None", and is coming up again when trying to use platform

Traceback (most recent call last):
  File "litex/sim.py", line 155, in <module>
  File "litex/sim.py", line 147, in main
    trace_fst   = 1)
  File "/home/lkcl/src/libresoc/litex/litex/soc/integration/builder.py", line 219, in build
    vns = self.soc.build(build_dir=self.gateware_dir, **kwargs)
  File "/home/lkcl/src/libresoc/litex/litex/soc/integration/soc.py", line 922, in build
    return self.platform.build(self, *args, **kwargs)
  File "/home/lkcl/src/libresoc/litex/litex/build/sim/platform.py", line 44, in build
    return self.toolchain.build(self, *args, **kwargs)
  File "/home/lkcl/src/libresoc/litex/litex/build/sim/verilator.py", line 187, in build
  File "/home/lkcl/src/libresoc/litex/litex/build/generic_platform.py", line 314, in finalize
    "No default clock and no clock domain defined")
NotImplementedError: No default clock and no clock domain defined
Comment 13 Luke Kenneth Casson Leighton 2020-07-22 21:21:00 BST
trying this - taken from a random/arbitrary file:

+from migen import ClockDomain
 from litex.build.generic_platform import Pins, Subsignal
 from litex.build.sim import SimPlatform
 from litex.build.sim.config import SimConfig
@@ -73,6 +76,12 @@ class SoCSMP(SoCCore):
         self.platform.name = "sim"
+        self.clock_domains.cd_sys = ClockDomain()
+        self.comb += [
+            self.cd_sys.clk.eq(platform.request("sys_clk")),
+            self.cd_sys.rst.eq(platform.request("sys_rst"))
+        ]
Comment 14 Luke Kenneth Casson Leighton 2020-07-22 21:43:20 BST
It's possible to avoid the interrupts in a first time, if your CPU wrapper does not have an interrupt Signal, UART POLLING mode will be used: https://github.com/enjoy-digital/litex/blob/master/litex/soc/integration/soc.py#L1063
I'm not sure which platform/target you are using, but you can find a minimal platform here:
Or can adapt one of the existing targets.
Comment 15 Luke Kenneth Casson Leighton 2020-07-29 12:50:41 BST
On Wed, Jul 29, 2020 at 12:04 PM Florent Kermarrec
<florent@enjoy-digital.fr> wrote:
> Hi,
> would you mind sending me the last libre_soc.v file you used for the simulation. I'm having a look at it but just want to use the verilog in a first time and have compilation issues with the first test_issuer.v you provided.

attached.  yes, the first version i found doesn't "start" (remains in
"halted" state).  this one is "up-to-date" with core.py and sim.py.

one thing: do note that we do full 8-bit (byte level) granularity on
the wishbone buses.  i do not know if microwatt does that.

the current bug that i encountered - apart from MEMTEST_BUS_xxx not
being correctly set up for 64-bit - is that if the litex bus is set to
32 bit this _does_ actually work except that the PC is read - even for
an even word - into the *UPPER* 32 bits of the 64-bit Wishbone dat_r,
*not* the lower 32 bits.
Comment 16 Luke Kenneth Casson Leighton 2020-09-03 12:18:11 BST
getting unit tests against microwatt (1.bin, 2.bin) was sufficient to get up and running, here.  running on an ECP5 FPGA has been confirmed https://www.youtube.com/watch?v=72QmWro9BSE