https://github.com/antonblanchard/microwatt/blob/master/xics.vhdl an XICS controller is needed, to be reasonably compatible with other power implementations implementation: https://git.libre-soc.org/?p=soc.git;a=blob;f=src/soc/interrupts/xics.py;hb=HEAD
external interrupt causes trap to 0x500 irq_valid := '0'; if ctrl.msr(MSR_EE) = '1' then if ctrl.dec(63) = '1' then v.f.redirect_nia := std_logic_vector(to_unsigned(16#900#, 64)); report "IRQ valid: DEC"; irq_valid := '1'; elsif ext_irq_in = '1' then v.f.redirect_nia := std_logic_vector(to_unsigned(16#500#, 64)); report "IRQ valid: External"; irq_valid := '1'; end if; end if;
https://git.libre-soc.org/?p=nmigen-soc.git;a=blob;f=nmigen_soc/test/test_csr_wishbone.py;h=2cde268d2b2075d62a54fe7c382a3c65287f6118;hb=87c34e8c810dc4f148be1af8b238b610dc808141 use nmigen-soc registers, should work well
commit 0ee5e92264b539e4938427c296e8d1477ae704a7 (HEAD -> master, origin/master) Author: Luke Kenneth Casson Leighton <lkcl@lkcl.net> Date: Sun Jul 26 21:27:53 2020 +0100 start on conversion of xics.vhdl to nmigen
commit 46a368d998f225d2b07c0c9c2a2a4e14a48a39fc (HEAD -> master, origin/master) Author: Luke Kenneth Casson Leighton <lkcl@lkcl.net> Date: Mon Jul 27 22:36:55 2020 +0100 add 2nd part of XICS interrupt interface
Author: Luke Kenneth Casson Leighton <lkcl@lkcl.net> Date: Tue Jul 28 13:28:22 2020 +0100 add preliminary investigative test of XICS ICS documentation (a specification) on the XICS appears not to be locatable. there exists *implementations* (qemu, linux kernel), just no documentation. this preliminary unit test therefore explores how the (converted microwatt) xics ICS works.
http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2020-July/008678.html
It looks like the documentation they have here, except for the part specific to qemu was taken from the XICS spec. So odd that we can't find the actual spec. https://lists.sr.ht/~philmd/qemu/patches/5642#%3C20190517094435.32765-1-clg@kaod.org%3E+docs/specs/ppc-xive.rst Well mostly at the bottom third of the page: ``` +++ b/docs/specs/ppc-xive.rst @@ -0,0 +1,199 @@ +================================ +POWER9 XIVE interrupt controller +================================ + +The POWER9 processor comes with a new interrupt controller +architecture, called XIVE as "eXternal Interrupt Virtualization +Engine". + +Compared to the previous architecture, the main characteristics of +XIVE are to support a larger number of interrupt sources and to +deliver interrupts directly to virtual processors without hypervisor +assistance. This removes the context switches required for the +delivery process. + + +XIVE architecture +================= + +The XIVE IC is composed of three sub-engines, each taking care of a +processing layer of external interrupts: [more useful documentation of Power9 XIVE] ```
(In reply to Cole Poirier from comment #7) > It looks like the documentation they have here, except for the part specific > to qemu was taken from the XICS spec. So odd that we can't find the actual > spec. > > https://lists.sr.ht/~philmd/qemu/patches/5642#%3C20190517094435.32765-1- > clg@kaod.org%3E+docs/specs/ppc-xive.rst this again is the virtualisation of interrupts, not the actual interrupts. cinsequently whilst the same registers appear (CPRR, PIPR) there's nothing about those. my feeling is that the spec has never been released: the qemu XICS code for example is written *by* someone at IBM who had internal access to it.
(In reply to Luke Kenneth Casson Leighton from comment #8) > this again is the virtualisation of interrupts, not the actual interrupts. > cinsequently whilst the same registers appear (CPRR, PIPR) there's nothing > about those. > > my feeling is that the spec has never been released: the qemu XICS code for > example is written *by* someone at IBM who had internal access to it. Oy gevalt! Such a headache, although I think Jacob may have found what we are looking for:... will edit with reference to mailing list archives once they are up-to-date enough to include jacob's mail. ``` from: Jacob Lifshay <programmerjake@gmail.com> to: Libre-RISCV General Development <libre-riscv-dev@lists.libre-riscv.org> date: Jul 28, 2020, 9:51 AM PDT subject: Re: [libre-riscv-dev] how do we test external interrupts? I found what looks like the reference source: in https://github.com/torvalds/linux/blob/ab02b61f24c76b1659086fcc8b00cbeeb6e95ac7/Documentation/virt/kvm/devices/xics.rst > This device emulates the XICS (eXternal Interrupt Controller Specification) defined in PAPR. https://en.wikipedia.org/wiki/Power_Architecture_Platform_Reference https://github.com/OpenPOWERFoundation/Linux-Architecture-Reference Jacob ``` I think this is the key part: https://github.com/OpenPOWERFoundation/Linux-Architecture-Reference/blob/master/LoPAR/ch_interrupt_controller.xml PowerPC External Interrupt Architecture: ``` <chapter xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" xml:id="dbdoclet.50569331_37856" version="5.0" xml:lang="en"> <title>Interrupt Controller</title> <para>This chapter specifies the requirements for the LoPAR interrupt controller. Platforms may chose to virtualize the interrupt controller or to provide the PowerPC External Interrupt option. </para> <section> <title>Interrupt Controller Virtualization</title> <para>Virtualization of the interrupt controller is done through the Interrupt Support hcalls. See <xref linkend="dbdoclet.50569344_26787"/>.</para> </section> <section xml:id="dbdoclet.50569331_29157"> <title>PowerPC External Interrupt Option</title> <para>The PowerPC External Interrupt option is based upon a subset of the PowerPC External Interrupt Architecture. The PowerPC External Interrupt Architecture contains a register-level architectural definition of an interrupt control structure. This architecture defines means for assigning properties such as priority, destination, etc., to I/O and interprocessor interrupts, as well as an interface for presenting them to processors. It supports both specific and distributed methods for interrupt delivery. See also <!-- FIXME: xref linkend="error_section"/--><citetitle>A PowerPC External Interrupt</citetitle>.htm#38341.--></para> <para>In NUMA platform configurations, the interrupt controllers may be configured in disjoint domains. The firmware makes the server numbers visible to any single OS image appear to come from a single space without duplication. This may be done by appropriately initializing the interrupt presentation controllers or the firmware may translate the server numbers presented to it in RTAS calls before entering them into the interrupt controller registers. The OS is made aware that certain interrupts are only served by certain servers by the inclusion of the <emphasis role="bold"><literal>“ibm,interrupt-domain”</literal></emphasis> property in the interrupt controller nodes.</para> <section xml:id="sec_ext_int_opt_req"> <title>PowerPC External Interrupt Option Requirements</title> <para>The following are the requirements for the PowerPC External Interrupt option. Additional requirements and information relative to the MSI option, when implemented with this option, are listed in <xref linkend="dbdoclet.50569331_33067"/>.</para> ``` I'll upload just this 'chapter' (only 750 lines of text) to the wiki?
https://elixir.bootlin.com/linux/latest/source/arch/powerpc/kvm/book3s_xics.c
(In reply to Cole Poirier from comment #9) > (In reply to Luke Kenneth Casson Leighton from comment #8) > > > this again is the virtualisation of interrupts, not the actual interrupts. > > cinsequently whilst the same registers appear (CPRR, PIPR) there's nothing > > about those. > > > > my feeling is that the spec has never been released: the qemu XICS code for > > example is written *by* someone at IBM who had internal access to it. > > Oy gevalt! Such a headache, although I think Jacob may have found what we > are looking for:... will edit with reference to mailing list archives once > they are up-to-date enough to include jacob's mail. > > ``` > from: Jacob Lifshay <programmerjake@gmail.com> > to: Libre-RISCV General Development <libre-riscv-dev@lists.libre-riscv.org> > date: Jul 28, 2020, 9:51 AM PDT > subject: Re: [libre-riscv-dev] how do we test external interrupts? > > I found what looks like the reference source: > in > https://github.com/torvalds/linux/blob/ > ab02b61f24c76b1659086fcc8b00cbeeb6e95ac7/Documentation/virt/kvm/devices/xics. > rst > > This device emulates the XICS (eXternal Interrupt Controller > Specification) defined in PAPR. > > https://en.wikipedia.org/wiki/Power_Architecture_Platform_Reference > > https://github.com/OpenPOWERFoundation/Linux-Architecture-Reference > > Jacob > ``` > > I think this is the key part: > > https://github.com/OpenPOWERFoundation/Linux-Architecture-Reference/blob/ > master/LoPAR/ch_interrupt_controller.xml > PowerPC External Interrupt Architecture: > ``` > <chapter xmlns="http://docbook.org/ns/docbook" > xmlns:xl="http://www.w3.org/1999/xlink" > xml:id="dbdoclet.50569331_37856" > version="5.0" > xml:lang="en"> > <title>Interrupt Controller</title> > > <para>This chapter specifies the requirements for the LoPAR interrupt > controller. Platforms may chose to virtualize the interrupt controller or > to > provide the PowerPC External Interrupt option. </para> no it's not that one - XICS is like the PLIC for RISC-V. it's the central (generic) "hub" for interrupts and defines the layout of in-memory registers. for example for the source layer (microwatt xics.vhdl): -- Register map -- 0 : Config -- 4 : Debug/diagnostics -- 800 : XIVE0 -- 804 : XIVE1 ... -- -- Config register format: -- -- 23.. 0 : Interrupt base (hard wired to 16) -- 27.. 24 : #prio bits (1..8) -- -- XIVE register format: -- -- 31 : input bit (reflects interrupt input) -- 30 : reserved -- 29 : P (mirrors input for now) -- 28 : Q (not implemented in this version) -- 30 .. : reserved -- 19 .. 8 : target (not implemented in this version) -- 7 .. 0 : prio/mask and for the presentation layer there's something similar. so the interrupt wires come in from a "source" (a single bit, each given a number), and it is possible to set the individual priority (by writing to the XIVEnnn register). then the "presentation" layer receives a notification about one of those, and generates - to the core - a *single* interrupt to say "please come and look at the presentation layer registers XIRR and MFRR to find out which one of the source interrupts has occurred". from the microwatt source code, _reading_ the XIRR register actually clears the interrupt. so that's how it fits together.
(In reply to Luke Kenneth Casson Leighton from comment #11) > no it's not that one - XICS is like the PLIC for RISC-V. it's the central > (generic) "hub" for interrupts and defines the layout of in-memory registers. > > for example for the source layer (microwatt xics.vhdl): > > -- Register map > -- 0 : Config > -- 4 : Debug/diagnostics > -- 800 : XIVE0 > -- 804 : XIVE1 ... > -- > -- Config register format: > -- > -- 23.. 0 : Interrupt base (hard wired to 16) > -- 27.. 24 : #prio bits (1..8) > -- > -- XIVE register format: > -- > -- 31 : input bit (reflects interrupt input) > -- 30 : reserved > -- 29 : P (mirrors input for now) > -- 28 : Q (not implemented in this version) > -- 30 .. : reserved > -- 19 .. 8 : target (not implemented in this version) > -- 7 .. 0 : prio/mask > > > and for the presentation layer there's something similar. > > so the interrupt wires come in from a "source" (a single bit, > each given a number), and it is possible to set the individual > priority (by writing to the XIVEnnn register). > > then the "presentation" layer receives a notification about one > of those, and generates - to the core - a *single* interrupt to > say "please come and look at the presentation layer registers > XIRR and MFRR to find out which one of the source interrupts > has occurred". > > from the microwatt source code, _reading_ the XIRR register actually > clears the interrupt. > > so that's how it fits together. Ok, so if I understand correctly xics is the internal interrupt infrastructure and the "LoPAR interrupt controller [is something] platforms may chose to virtualize or to provide the PowerPC External Interrupt option." (https://github.com/OpenPOWERFoundation/Linux-Architecture-Reference/blob/master/LoPAR/ch_interrupt_controller.xml) That deals with PCIE interrupts? Sorry I'm still a little confused...
(In reply to Cole Poirier from comment #12) > master/LoPAR/ch_interrupt_controller.xml) That deals with PCIE interrupts? > Sorry I'm still a little confused... so the interrupt wires from PCIe would go *into* the source vector XICS_ICS. e.g. bit 5 would be PCIe device 1's hardware interrupt wire. or however it is done. bit 6 of the XICS source would be connected e.g. to SPI0 and so on.
from mikey: "Yeah I was just look at LoPAPR and it's not there" https://openpowerfoundation.org/wp-content/uploads/2020/07/LoPAR-20200611.pdf http://lists.mailinglist.openpowerfoundation.org/pipermail/openpower-hdl-cores/2020-July/000107.html
(In reply to Luke Kenneth Casson Leighton from comment #14) > from mikey: > "Yeah I was just look at LoPAPR and it's not there" > https://openpowerfoundation.org/wp-content/uploads/2020/07/LoPAR-20200611.pdf p437 the HCALLs are there. just no hardware details
https://lists.sr.ht/~philmd/qemu/patches/5642#%3C20190517094435.32765-1-clg@kaod.org%3E+docs/specs/ppc-xive.rst that doc is particularly informative. i have the preliminary investigative unit tests working, enough to be able to progress to adding XICS as a peripheral.
commit ffe70742b1fb287ba1766df8a8e55ec2190db797 (HEAD -> master, origin/master) Author: Luke Kenneth Casson Leighton <lkcl@lkcl.net> Date: Fri Sep 4 16:11:10 2020 +0100 adding option to include XICS external interrupts. XICS ICP and ICS are included, the wishbone slave ports added to TestIssuer then if ext_irq is raised in core, execution jumps to 0x500 through a TRAP commit 39f8d6aa73fd74610eeb354f76532fa2413ba2a4 (HEAD -> master) Author: Luke Kenneth Casson Leighton <lkcl@lkcl.net> Date: Fri Sep 4 16:16:40 2020 +0100 bring out XICS ICS interrupt levels so that they can be wired to peripherals
commit 2a187089be6e063f6bcaf8c27bfe82a5fe76570c Author: Luke Kenneth Casson Leighton <lkcl@lkcl.net> Date: Sat Sep 5 14:38:08 2020 +0100 add simple GPIO peripheral to verilog TestIssuer commit 63ccd4c6351103b904a956b7963cf42f53e748ca (HEAD -> master, origin/master) Author: Luke Kenneth Casson Leighton <lkcl@lkcl.net> Date: Sat Sep 5 15:03:21 2020 +0100 add simple GPIO wishbone bus to litex sim.py
microwatt xics.bin unit test works after some modifications to use a simple GPIO rather than a 16550 UART.