Bug 1130 - Build Debian SFFS
Summary: Build Debian SFFS
Status: RESOLVED FIXED
Alias: None
Product: Libre-SOC's first SoC
Classification: Unclassified
Component: Milestones (show other bugs)
Version: unspecified
Hardware: PC Linux
: --- enhancement
Assignee: Sadoon Albader
URL: https://libre-soc.org/SFFS/debian_boo...
Depends on: 860
Blocks:
  Show dependency treegraph
 
Reported: 2023-08-02 20:44 BST by Toshaan Bharvani
Modified: 2024-03-10 06:51 GMT (History)
3 users (show)

See Also:
NLnet milestone: NLnet.2022-08-107.ongoing
total budget (EUR) for completion of task and all subtasks: 2500
budget (EUR) for this task, excluding subtasks' budget: 2500
parent task for budget allocation: 999
child tasks for budget allocation:
The table of payments (in EUR) for this task; TOML format:
sadoon={amount=1500, submitted=2024-02-26, paid=2024-03-08} lkcl = {amount=1000, submitted=2024-02-26, paid=2024-03-08}


Attachments
sbuild build script (725 bytes, application/x-shellscript)
2023-08-08 14:44 BST, Sadoon Albader
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Toshaan Bharvani 2023-08-02 20:44:03 BST
We are rebuilding Debian 12 (Bookworm) specifically as it is the most recent version and was just released two months ago.

This task consists not only of the work of rebuilding Debian, but also documentation (https://libre-soc.org/SFFS/debian_bootstrap), and automated scripts (https://git.libre-soc.org/?p=dev-env-setup.git) that guarantee full reproduciblity and only require one to have a POWER8 or newer system to rebuild everything from source using the scripts.
Comment 1 Sadoon Albader 2023-08-04 17:11:37 BST
Debian's build flags are stored in /etc/dpkg/buildflags.conf
This means when working with a schroot (like with sbuild), these flags specifically need to be enabled *inside* the chroot itself, not for the host machine executing sbuild.

cat /srv/chroot/bookworm-ppc64el-sbuild/etc/dpkg/buildflags.conf

APPEND CFLAGS -mcpu=power9 -mno-altivec -mno-vsx -mno-crypto -mno-htm -mlong-double-64
APPEND CXXFLAGS -mcpu=power9 -mno-altivec -mno-vsx -mno-crypto -mno-htm -mlong-double-64

This works according to my test in https://bugs.libre-soc.org/show_bug.cgi?id=1128#c8
Comment 2 Sadoon Albader 2023-08-08 14:44:25 BST
Created attachment 199 [details]
sbuild build script

Created attachment 199 [details]
sbuild build script

Build script for use with sbuild
Requires sbuild, a list of packages "list", an sbuild chroot (schroot) to be set up, and for our use case requires the chroot to have the dpkg-buildflags set as in the comment above.
Will report successful builds in "success" and failures in "failed"

Multiple instances can be run in parallel provided they don't build the same packages. Could be split up by first letter for example, and given different list files. All directories and files are set up as bash variables and the code is self-explanatory (the pkg:0:1 takes the first letter of the package)
Comment 3 Sadoon Albader 2023-08-08 14:46:58 BST
Placeholder for the second script which creates a repository of the built packages.
Comment 4 Sadoon Albader 2023-08-08 14:47:22 BST
Placeholder for the final script to create a debootstrap using these packages
Comment 5 Sadoon Albader 2023-08-08 20:38:31 BST
gpg might need soft floats, fails with -mlong-double-64
Comment 6 Luke Kenneth Casson Leighton 2023-08-23 17:10:40 BST
i've moved the page to a new location, covering "everything SFFS".
at some point an SFFS page itself is needed

https://git.libre-soc.org/?p=libreriscv.git;a=commit;h=6dd2bd753e0f1ebe971fdcd29e4fd371f4ccb672

also updated the wiki page itself to point to this bugreport,
*and* updated the URL of this bugreport to point back to the wiki.
this gives us cross-referencing that makes navigation a hell of
a lot easier.

https://libre-soc.org/irclog/%23libre-soc.2023-08-23.log.html#t2023-08-23T17:12:56

sadoon remember to update comment #0 with a 2-line description
then answer michiel that "yes the actual results may be seen on the
bugreport and verified that they have in fact been done, so you can
answer the EU Auditor and also satisfy NLnet's 'Works for the Public
Good' mandate" https://bugs.libre-soc.org/show_bug.cgi?id=961#c6
Comment 7 Luke Kenneth Casson Leighton 2023-08-25 13:53:14 BST
sadoon i've made some corrections/formatting

* 80 char limit,
* indentation missing,
* $OBJ_DUMP replaced with `$OBJ_DUMP`
* added limited liability on downloads

https://git.libre-soc.org/?p=libreriscv.git;a=commitdiff;h=892206dcaae487109730c2f7dd8dbd7d03f70db2
Comment 8 Sadoon Albader 2023-08-25 15:49:53 BST
glibc will fail to build if attempted with the current buildflags, there is no way around this without patching glibc itself, needs a separate task. We will use vanilla built glibc for now.
Comment 9 Luke Kenneth Casson Leighton 2023-08-25 15:55:29 BST
(In reply to Sadoon Albader from comment #8)
> glibc will fail to build if attempted with the current buildflags, there is
> no way around this without patching glibc itself, needs a separate task.

this is already done as part of NGI POINTER but Toshaan has not yet
made available the patches (which were supposed to have been
released in order to comply with the NGI POINTER conditions).
Comment 10 Luke Kenneth Casson Leighton 2023-08-29 10:57:00 BST
see ideas here for next update, to bootstrap up by using qemu/vsx or POWER9
and recompiling packages individually, natively, one-by-one.
(edit: *within the chroot*. this involves installing build-essential
*from standard debian-ppc64le* and then recompiling even those packages,
ultimately *ending up* with debian-ppc64lesffs but incrementally)

https://libre-soc.org/irclog/%23libre-soc.2023-08-27.log.html#t2023-08-27T10:20:07
Comment 11 Luke Kenneth Casson Leighton 2023-09-01 13:02:11 BST
uploaded the debian-sffs.tar.xz and linked it to the wiki with
the appropriate warning(s)
https://git.libre-soc.org/?p=libreriscv.git;a=commitdiff;h=a90476aec