Bug 633 - custom run of ulx3s with 85k LUTs 5G part, and 128 mbyte (1 gigabit) SDRAM
Summary: custom run of ulx3s with 85k LUTs 5G part, and 128 mbyte (1 gigabit) SDRAM
Status: CONFIRMED
Alias: None
Product: Libre-SOC's first SoC
Classification: Unclassified
Component: Source Code (show other bugs)
Version: unspecified
Hardware: PC Linux
: --- enhancement
Assignee: Luke Kenneth Casson Leighton
URL:
Depends on:
Blocks:
 
Reported: 2021-04-27 00:03 BST by Luke Kenneth Casson Leighton
Modified: 2021-04-29 01:17 BST (History)
3 users (show)

See Also:
NLnet milestone: ---
total budget (EUR) for completion of task and all subtasks: 0
budget (EUR) for this task, excluding subtasks' budget: 0
parent task for budget allocation:
child tasks for budget allocation:
The table of payments (in EUR) for this task; TOML format:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Luke Kenneth Casson Leighton 2021-04-27 00:03:26 BST
https://www.digikey.co.uk/product-detail/en/lattice-semiconductor-corporation/LFE5UM5G-85F-8BG381I/220-2112-ND/6173749
https://www.digikey.co.uk/product-detail/en/alliance-memory-inc/AS4C64M16D1-6TCN/1450-1332-ND/5797372

http://lists.libre-soc.org/pipermail/libre-soc-dev/2021-April/002458.html

we've been donated around USD 5,000 in crypto-currency, we need reasonably
high-end FPGA boards.

according to gatecat, ECP5-5G parts have faster BRAMs (correction: this was part of a discussion - link needed - which involved running the ECP5-5G with 1.2v).

apparently, ECP5-5G has the SERDES (needed for PCI-e).

for running linux, we are pushing our luck with only 64 mbytes of memory.
therefore, the 128 mbyte part is needed.

https://github.com/emard/ulx3s
Comment 1 Jacob Lifshay 2021-04-27 00:39:01 BST
If we can find a good supply, it may be worth getting faster ram:
I found the MT46V64M16 series after a little searching -- idk if Micron still makes it though...
Comment 2 Luke Kenneth Casson Leighton 2021-04-27 00:55:19 BST
https://www.digikey.com/en/products/base-product/micron-technology-inc/557/MT46V64M16/37720

digikey lists it as obsolete unfortunately.

alliance may have faster

(edit: no)
https://www.digikey.co.uk/products/en?t=774&pv570=400796&pv142=120140&pv1291=263924

all the same part.  no other digikey. trying mouser.  nope.
only Alliance available for the TSOP-66 in mouser, and they
only have the 32x16 part (64 mbyte).
Comment 3 Jacob Lifshay 2021-04-27 01:04:43 BST
one other potential option (that may be waay too much work for a few k$):
replace chip with a ddr2/3 ram chip and reroute that small section of the pcb to accommodate a bga. iirc the ecp5 has a hw ddr3 interface and the gpio is fast enough to run ddr2.
Comment 4 Jacob Lifshay 2021-04-27 01:08:48 BST
Also, the total amount has gone down to around $4100 due to bitcoin's value going down
Comment 5 Luke Kenneth Casson Leighton 2021-04-27 01:09:16 BST
(In reply to Jacob Lifshay from comment #3)
> one other potential option (that may be waay too much work for a few k$):

exactly.

also, as you're aware: we're on a serious time-crunch.  we cannot add yet
more dependencies to the critical path.  i know how much PCB development
costs in both time and money.  the fact that the ulx3s PCB is done, designed,
tested and proven is extremely important.
Comment 6 Luke Kenneth Casson Leighton 2021-04-27 01:09:31 BST
(In reply to Jacob Lifshay from comment #4)
> Also, the total amount has gone down to around $4100 due to bitcoin's value
> going down

*sigh* ok
Comment 7 Staf Verhaegen 2021-04-27 08:06:15 BST
With current chip crunch and part lead times I'm not sure thinking about own custom PCB is a good idea; especially if it is time critical.
There is a 85F version of ULX3S available on crowdsupply:
https://www.crowdsupply.com/radiona/ulx3s
If speed is really important maybe better to think about Intel/Xilinx FPGA anyway. For example Trenz Electronics has a whole bunch of them:
https://shop.trenz-electronic.de/en/Products/Trenz-Electronic/
They also do custom modification to boards or your own design (like the MEGA65 prototype).
Comment 8 Luke Kenneth Casson Leighton 2021-04-27 12:15:49 BST
(In reply to Staf Verhaegen from comment #7)
> With current chip crunch and part lead times I'm not sure thinking about own
> custom PCB is a good idea; especially if it is time critical.

fortunately there is stock - not very much, but at least some.
(for now)

> There is a 85F version of ULX3S available on crowdsupply:
> https://www.crowdsupply.com/radiona/ulx3s

it only has 32 mbytes of RAM.  this is nowhere near enough.

> If speed is really important maybe better to think about Intel/Xilinx FPGA
> anyway. For example Trenz Electronics has a whole bunch of them:
> https://shop.trenz-electronic.de/en/Products/Trenz-Electronic/

https://shop.trenz-electronic.de/en/Products/Trenz-Electronic/TE0741-Kintex-7/

interesting.  modular design.

> They also do custom modification to boards or your own design (like the
> MEGA65 prototype).

these are non-libre tools, unfortunately.  our customer has a specific
need for Libre tools.  that means ECP5 at the moment, until nextprn
can support other FPGAs.
Comment 9 Luke Kenneth Casson Leighton 2021-04-28 14:43:46 BST
we've an additional requirement for BMC support: PCI-e and Ethernet.
Ethernet can be covered with this:
https://www.waveshare.com/product/DP83848-Ethernet-Board.htm

also here is a kicad board with PCIe 1x edge connections.
should be easy to either adapt or make a breakout
https://github.com/ghent360/PrntrBoardV2
Comment 10 Luke Kenneth Casson Leighton 2021-04-28 16:35:37 BST
https://github.com/KiCad/kicad-symbols/blob/master/Connector.lib

BUS_PCIexpress_x1
Comment 11 Jacob Lifshay 2021-04-28 17:58:38 BST
(In reply to Luke Kenneth Casson Leighton from comment #10)
> https://github.com/KiCad/kicad-symbols/blob/master/Connector.lib

That's their old repo.
> 
> BUS_PCIexpress_x1

Here's all the pcie edge connectors:
https://gitlab.com/kicad/libraries/kicad-footprints/-/blob/8fa36dfa3423d8777472e3475c1c2b0b2069624f/Connector_PCBEdge.pretty/BUS_PCI_Express_Mini.kicad_mod

https://gitlab.com/kicad/libraries/kicad-footprints/-/blob/8fa36dfa3423d8777472e3475c1c2b0b2069624f/Connector_PCBEdge.pretty/BUS_PCI_Express_Mini_Dual.kicad_mod

https://gitlab.com/kicad/libraries/kicad-footprints/-/blob/8fa36dfa3423d8777472e3475c1c2b0b2069624f/Connector_PCBEdge.pretty/BUS_PCI_Express_Mini_Full.kicad_mod

https://gitlab.com/kicad/libraries/kicad-footprints/-/blob/8fa36dfa3423d8777472e3475c1c2b0b2069624f/Connector_PCBEdge.pretty/BUS_PCI_Express_Mini_Half.kicad_mod

https://gitlab.com/kicad/libraries/kicad-footprints/-/blob/8fa36dfa3423d8777472e3475c1c2b0b2069624f/Connector_PCBEdge.pretty/BUS_PCIexpress_x1.kicad_mod

https://gitlab.com/kicad/libraries/kicad-footprints/-/blob/8fa36dfa3423d8777472e3475c1c2b0b2069624f/Connector_PCBEdge.pretty/BUS_PCIexpress_x16.kicad_mod

https://gitlab.com/kicad/libraries/kicad-footprints/-/blob/8fa36dfa3423d8777472e3475c1c2b0b2069624f/Connector_PCBEdge.pretty/BUS_PCIexpress_x4.kicad_mod

https://gitlab.com/kicad/libraries/kicad-footprints/-/blob/8fa36dfa3423d8777472e3475c1c2b0b2069624f/Connector_PCBEdge.pretty/BUS_PCIexpress_x8.kicad_mod
Comment 13 Luke Kenneth Casson Leighton 2021-04-28 20:22:48 BST
https://github.com/emard/ulx3s/blob/master/doc/schematics_v316.pdf
Comment 14 Luke Kenneth Casson Leighton 2021-04-28 20:34:54 BST
https://www.digikey.com/en/products/detail/lattice-semiconductor-corporation/ISPPAC-CLK5406D-01SN48I/2751347

nuts.  Lattice VERSA ECP5 has a block buffer/generator IC which creates
the PCIe master clock.

it'll be nowhere near as simple as "just wire out PCIe from the FPGA".
Comment 15 Jacob Lifshay 2021-04-28 20:47:35 BST
after a little searching:
whitequark has a pcie phy:
https://github.com/whitequark/Yumewatari/blob/master/yumewatari/gateware/phy.py

Also, found a board with an ecp5 and pcie:
https://github.com/gatecat/TrellisBoard