Bug 847 - dev-env-setup script for binutils-gdb for target powerpc64le-linux-gnu
Summary: dev-env-setup script for binutils-gdb for target powerpc64le-linux-gnu
Status: RESOLVED FIXED
Alias: None
Product: Libre-SOC's first SoC
Classification: Unclassified
Component: Source Code (show other bugs)
Version: unspecified
Hardware: PC Linux
: --- normal
Assignee: Veera
URL:
Depends on:
Blocks:
 
Reported: 2022-06-09 00:53 BST by Veera
Modified: 2022-07-24 10:37 BST (History)
4 users (show)

See Also:
NLnet milestone: NLNet.2019.10.032.Formal
total budget (EUR) for completion of task and all subtasks: 650
budget (EUR) for this task, excluding subtasks' budget: 650
parent task for budget allocation: 577
child tasks for budget allocation:
The table of payments (in EUR) for this task; TOML format:
vklr={amount=450, submitted=2022-07-12, paid=2022-07-22} # was originally submitted 2022-06-13 ghostmansd={amount=200, submitted=2022-07-19, paid=2022-07-22}


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Veera 2022-06-09 00:53:43 BST
Automated build and needed dependency installation script for binutils-gdb for target powerpc64le-linux-gnu- from source code to be made. And should only use the svp64 branch. Host are x86_64 and powerpc64le.

to be added to dev-env-setup:
https://git.libre-soc.org/?p=dev-env-setup.git;a=summary

Get svp64 branch of binutils-gdb from:
https://git.libre-soc.org/?p=binutils-gdb.git;a=shortlog;h=refs/heads/svp64

Take note from gcc and gdb install script:
https://git.libre-soc.org/?p=dev-env-setup.git;a=blob_plain;f=ppc64-gdb-gcc;hb=HEAD
Comment 1 Veera 2022-06-10 15:50:47 BST
Uploaded initial binutils-gdb-install script:

uri: https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=binutils-gdb-install;h=aedbddfa5ca24fa0b6bb1abf0b9042d769396bd3;hb=HEAD

I have run this in Raptor Talos Power server and it builds and installs successfully.
Checked the installed binaries with plain invocation.

By hand ran in my x86_64 machine it builds and installs fine. And binaries work.
Have to run the script and see if it passes.

Both plain 'as' and 'powerpc64le-linux-gnu-as' show '-mlibresoc' option for PowerPC in both host machines x86_64 and powerpc64le.

Kindly review and give feedback.
Comment 2 Luke Kenneth Casson Leighton 2022-06-10 15:58:35 BST
very cool, veera, especially the -mlibresoc.
ghostmansd, can you check it? or, let veera know
which "make test" to run.
Comment 3 Luke Kenneth Casson Leighton 2022-06-10 16:01:46 BST
(In reply to vklr@vkten.in from comment #1)

> Both plain 'as' and 'powerpc64le-linux-gnu-as' 

ah: "as" should *not* have been overwritten, that is very
important.

it should explicitly *and only* be powerpc64le-linux-gnu-as
that is the svp64 branch.

if /usr/local/bin/as has been created, that will interfere
with building of software on that machine.
Comment 4 Dmitry Selyutin 2022-06-10 16:54:21 BST
If the vanilla as shows -mlibresoc option, this is very wrong: it means vanilla as is overriden. Please post the output of `which -a as`.

What was the rationale for options? Why disable NLS? Why disable multilib? Please also under no circumstances disable `-Werror` flag; if svp64 branch has issues, this should be fixed in branch.
Comment 5 Dmitry Selyutin 2022-06-10 16:56:37 BST
> cd /home/$SUDO_USER/src/binutils-gdb

You're already here after `cd ..` two lines above. :-)


> chown -R $SUDO_USER .
> chgrp -R $SUDO_USER .

Why? If something has been created by superuser (I don't actually see a need except for apt-get install, by the way), then it should be removed by superuser. I don't see a need to change the ownership.
Comment 6 Dmitry Selyutin 2022-06-10 17:00:07 BST
> if [ "`arch`" = "x86_64" ]; then
>   export HOST="x86_64-pc-linux-gnu"
> elif [ "`arch`" = "ppc64le" ]; then
>   export HOST="powerpc64le-linux-gnu"
> fi

I don't think we can do it better than binutils. I suggest dropping HOST entirely as well as --host. It should be guessed by the configure script automatically.
Comment 7 Dmitry Selyutin 2022-06-10 17:01:50 BST
Also you likely don't need to clone the whole repo, limit the history with --depth=1.

The rest LGTM.
Comment 8 Luke Kenneth Casson Leighton 2022-06-10 17:44:17 BST
(In reply to Dmitry Selyutin from comment #7)
> Also you likely don't need to clone the whole repo, limit the history with
> --depth=1.

apparently you need --no-single-branch as well otherwise you
end up only with master branch
Comment 9 Jacob Lifshay 2022-06-10 18:19:36 BST
(In reply to Luke Kenneth Casson Leighton from comment #8)
> (In reply to Dmitry Selyutin from comment #7)
> > Also you likely don't need to clone the whole repo, limit the history with
> > --depth=1.
> 
> apparently you need --no-single-branch as well otherwise you
> end up only with master branch

a better option is:
git clone --depth 1 -b <the-branch-you-want> <url.git> <dir>

that only fetches the latest commit on the branch you want (which can be a tag rather than a branch).
Comment 10 Veera 2022-06-11 00:04:20 BST
(In reply to Luke Kenneth Casson Leighton from comment #3)
> (In reply to vklr@vkten.in from comment #1)
> 
> > Both plain 'as' and 'powerpc64le-linux-gnu-as' 
> 
> ah: "as" should *not* have been overwritten, that is very
> important.
> 
> it should explicitly *and only* be powerpc64le-linux-gnu-as
> that is the svp64 branch.
> 

In powerpc64le machine it is also installed as 'as'.

(In reply to Dmitry Selyutin from comment #4)
> (In reply to vklr@vkten.in from comment #1)
> If the vanilla as shows -mlibresoc option, this is very wrong: it means
> vanilla as is overriden. Please post the output of `which -a as`.

Vanilla 'as' does not shows '-mlibresoc'. All the files are installed separately in /usr/local/binutils-gdb
Comment 11 Veera 2022-06-11 00:09:33 BST
(In reply to Dmitry Selyutin from comment #6)
> > if [ "`arch`" = "x86_64" ]; then
> >   export HOST="x86_64-pc-linux-gnu"
> > elif [ "`arch`" = "ppc64le" ]; then
> >   export HOST="powerpc64le-linux-gnu"
> > fi
> 
> I don't think we can do it better than binutils. I suggest dropping HOST
> entirely as well as --host. It should be guessed by the configure script
> automatically.

I do not think so. Explicitly putting shows we are building a cross toolchain.
The commands are itself documentary. Anyway the build machine is going to be one of ports made for Debian Buster; x86_64, powerpc64le or arm64, etc(limited).
Comment 12 Veera 2022-06-11 00:19:06 BST
(In reply to Dmitry Selyutin from comment #4)
> What was the rationale for options? Why disable NLS? Why disable multilib?
> Please also under no circumstances disable `-Werror` flag; if svp64 branch
> has issues, this should be fixed in branch.

Do you use NLS(Native Language Support: French, German, Russian...)? If needed I will enable NLS
Multilib: 32 and 64bit libs and tools for specific target machine; Do you need 32 bit support?
I will remove --disable-werror!
Comment 13 Veera 2022-06-11 03:05:45 BST
Removed --disable-werror from configure so that warning produces error.

https://git.libre-soc.org/?p=dev-env-setup.git;a=commitdiff;
h=95e4fb465d0a4e6f652f115c353c3e942e70f77b

Also checked the automated script in my x86_64 machine, it builds, installs and works fine.

Kindly do review.
Comment 14 Dmitry Selyutin 2022-06-11 06:47:31 BST
(In reply to vklr@vkten.in from comment #11)
> 
> I do not think so. Explicitly putting shows we are building a cross
> toolchain.

It's already dead obvious since you have --target. It's already explicit enough. Please don't invent things: either configure script already uses the approach you use, or has something more complex and better.

(In reply to vklr@vkten.in from comment #12)
> 
> Do you use NLS(Native Language Support: French, German, Russian...)? If
> needed I will enable NLS
> Multilib: 32 and 64bit libs and tools for specific target machine; Do you
> need 32 bit support?

Do you immediately use plugins? Can't binutils work without --enable-threads? My point is, most users don't really bother changing these via the configure script. I'd like to see as simple workload as can be; unless stuff breaks without some option, I'd consider leaving its state to default, and don't even touch it.

I suspect that we might need multilib one day, Luke knows the better. And, if this is the case, we do --enable-multilib exactly when needed. I understand that you want to trim and optimize the toolchain, but it's like shooting the sparrows with a cannon.

I won't object to keeping the code as is, I'm simply trying to show the rationale. As long as stuff works, I'm happy.
Comment 15 Dmitry Selyutin 2022-06-11 06:51:25 BST
(In reply to Dmitry Selyutin from comment #14)
> 
> I won't object to keeping the code as is, I'm simply trying to show the
> rationale. As long as stuff works, I'm happy.

...except for --host, this looks like reinventing the wheel. --target is already explicit enough, so I don't buy this argument. :-)
Comment 16 Veera 2022-06-11 08:06:21 BST
(In reply to Dmitry Selyutin from comment #15)
> (In reply to Dmitry Selyutin from comment #14)
> > 
> > I won't object to keeping the code as is, I'm simply trying to show the
> > rationale. As long as stuff works, I'm happy.
> 
> ...except for --host, this looks like reinventing the wheel. --target is
> already explicit enough, so I don't buy this argument. :-)

Removed HOST variable and also --host option to configure. Checked the script is correct by building, installing and running the commands; as, ld, gdb.

plugins is needed for lto support, threads are used by gold linker.
For now multilib is kept disabled as it will require more work to install. It may require 32 bit libc and friends. By the way the installed programs have support for 32 bit elf powerpcle.

uri: https://git.libre-soc.org/?p=dev-env-setup.git;a=commitdiff;h=df19af7a4977dd34bf236e46a9d6592f3129a99b
Comment 17 Dmitry Selyutin 2022-06-11 08:16:37 BST
Cool! Cool cool cool. Thank you!
Comment 18 Luke Kenneth Casson Leighton 2022-06-11 09:15:23 BST
(In reply to vklr@vkten.in from comment #16)

> Removed HOST variable and also --host option to configure. Checked the
> script is correct by building, installing and running the commands; as, ld,
> gdb.

we *do not* want the commands to be created "as", "ld", "gdb".
as i said already they *must* only be powerpc64le-linux-gnu-as,
powerpc64le-linux-gnu-ld, and powerpc64le-linux-gnu-gdb

this is EXTREMELY IMPORTANT

(In reply to vklr@vkten.in from comment #10)

> In powerpc64le machine it is also installed as 'as'.

that's because it was installed from a debian package
"apt-get install binutils" which is a stable native version.

it is ABSOLUTELY CRITICAL THAT THIS COMMAND NOT BE DESTROYED
OR OVERWRITTEN OR OVERRIDDEN IN ANY WAY SHAPE OR FORM.


> All the files are installed
> separately in /usr/local/binutils-gdb

i have removed that and set it to the default /usr/local which is what
i asked to be done.  we do not need to keep adding more and more entries
to /usr/local/{command_directory}
Comment 19 Veera 2022-06-11 11:19:39 BST
(In reply to Luke Kenneth Casson Leighton from comment #18)
> (In reply to vklr@vkten.in from comment #16)
>
> we *do not* want the commands to be created "as", "ld", "gdb".
> as i said already they *must* only be powerpc64le-linux-gnu-as,
> powerpc64le-linux-gnu-ld, and powerpc64le-linux-gnu-gdb
> 
> this is EXTREMELY IMPORTANT
> 
Just referred them in short(not actual).
> (In reply to vklr@vkten.in from comment #10)
> 
> > All the files are installed
> > separately in /usr/local/binutils-gdb
> 
> i have removed that and set it to the default /usr/local which is what
> i asked to be done.  we do not need to keep adding more and more entries
> to /usr/local/{command_directory}
Thanks.
Well, for initial testing installation I like keeping them in a separate subdirectory, so that I can easily glance what is installed and where. After that we can put them in common /usr/local directory.

Anyway, can this bug be considered done?
Comment 20 Luke Kenneth Casson Leighton 2022-06-11 11:36:34 BST
(In reply to vklr@vkten.in from comment #19)

> Just referred them in short(not actual).

quite important not to do that :)

> Well, for initial testing installation I like keeping them in a separate
> subdirectory, so that I can easily glance what is installed and where.

in a chroot it shouldn't be interfering with anything else.
if it does that's important to know early.

> After
> that we can put them in common /usr/local directory.
> 
> Anyway, can this bug be considered done?

i'm happy with it.  dmitry?
Comment 21 Dmitry Selyutin 2022-06-11 11:41:30 BST
Me too. Speaking of prefix... I've been thinking that anyone sane would use chroot, and prefix doesn't matter that much.
Comment 22 Dmitry Selyutin 2022-06-13 18:15:51 BST
Cross-posting. Basically should've been there, but I first thought of IRC.

<ghostmansd> Ahem, I didn't pay attention that in 847 we made a new script, not modified ppc64-gdb-gcc... Shouldn't we drop that one?
<ghostmansd> I think we should either: a) migrate gcc and its prerequisites to some script like gcc-install; 2) create some script called toolchain-install and have all the stuff there

https://libre-soc.org/irclog/%23libre-soc.2022-06-13.log.html#t2022-06-13T18:05:29
Comment 23 Dmitry Selyutin 2022-06-13 19:13:39 BST
Veera, I have a problem with a fresh install. Upon launching as inside chroot environment on talos1, I observe the following:

$ echo | powerpc64le-linux-gnu-as -mlibresoc - -c /tmp/test.o 
powerpc64le-linux-gnu-as: error while loading shared libraries: libopcodes-2.38.50.20220525.so: cannot open shared object file: No such file or directory
$ ldd /usr/local/bin/powerpc64le-linux-gnu-as
        linux-vdso64.so.1 (0x00007fff8ef20000)
        libopcodes-2.38.50.20220525.so => not found
        libbfd-2.38.50.20220525.so => not found
        libdl.so.2 => /lib/powerpc64le-linux-gnu/libdl.so.2 (0x00007fff8ee40000)
        libc.so.6 => /lib/powerpc64le-linux-gnu/libc.so.6 (0x00007fff8ec20000)
        /lib64/ld64.so.2 (0x00007fff8ef40000)

You might opt to try it in a new chroot environment.
Comment 24 Luke Kenneth Casson Leighton 2022-06-13 20:44:55 BST
(ghostmansd)root@75-224-155-23:~# !powe
powerpc64le-linux-gnu-as
powerpc64le-linux-gnu-as: error while loading shared libraries: libopcodes-2.38.50.20220525.so: cannot open shared object file: No such file or directory
(ghostmansd)root@75-224-155-23:~# ls -altr /usr/local/lib/libopcodes-2.38.50.20220525.so 
-rwxr-xr-x 1 root root 1025488 Jun 13 18:15 /usr/local/lib/libopcodes-2.38.50.20220525.so

(ghostmansd)root@75-224-155-23:~# file !$
file /usr/local/lib/libopcodes-2.38.50.20220525.so
/usr/local/lib/libopcodes-2.38.50.20220525.so: ELF 64-bit LSB pie executable, 64-bit PowerPC or cisco 7500, version 1 (SYSV), dynamically linked, BuildID[sha1]=c50d0d637b70508b105f4264b7d727b91ba34747, with debug_info, not stripped

(ghostmansd)root@75-224-155-23:~# !ldd
ldd /usr/local/bin/powerpc64le-linux-gnu-as
        linux-vdso64.so.1 (0x00007fff960c0000)
        libopcodes-2.38.50.20220525.so => not found
        libbfd-2.38.50.20220525.so => not found
        libdl.so.2 => /lib/powerpc64le-linux-gnu/libdl.so.2 (0x00007fff95fe0000)
        libc.so.6 => /lib/powerpc64le-linux-gnu/libc.so.6 (0x00007fff95dc0000)
        /lib64/ld64.so.2 (0x00007fff960e0000)

huhn?

suggestion: screw it. compile --enable-static.
Comment 25 Veera 2022-06-13 20:59:09 BST
(In reply to Dmitry Selyutin from comment #23)
> Veera, I have a problem with a fresh install. Upon launching as inside
> chroot environment on talos1, I observe the following:
> 
> $ echo | powerpc64le-linux-gnu-as -mlibresoc - -c /tmp/test.o 
> powerpc64le-linux-gnu-as: error while loading shared libraries:
> libopcodes-2.38.50.20220525.so: cannot open shared object file: No such file
> or directory
> $ ldd /usr/local/bin/powerpc64le-linux-gnu-as
>         linux-vdso64.so.1 (0x00007fff8ef20000)
>         libopcodes-2.38.50.20220525.so => not found
>         libbfd-2.38.50.20220525.so => not found
>         libdl.so.2 => /lib/powerpc64le-linux-gnu/libdl.so.2

Find out in which directory libopcodes-2.38.50.20220525.so and libbfd-2.38.50.20220525.so are installed and add that to /etc/ld.so.conf.d/binutils-gdb.conf and run 'ldconfig' as root. Thats it.

In my installed system output of ldd:
root@75-224-155-23:~# ldd /usr/local/binutils-gdb/bin/powerpc64le-linux-gnu-as
        linux-vdso64.so.1 (0x00007fffb3d80000)
        libopcodes-2.38.50.20220609.so => /usr/local/binutils-gdb/lib/libopcodes-2.38.50.20220609.so (0x00007fffb3c20000)
        libbfd-2.38.50.20220609.so => /usr/local/binutils-gdb/lib/libbfd-2.38.50.20220609.so (0x00007fffb3a40000)
        libdl.so.2 => /lib/powerpc64le-linux-gnu/libdl.so.2 (0x00007fffb3a00000)
        libc.so.6 => /lib/powerpc64le-linux-gnu/libc.so.6 (0x00007fffb37e0000)
        /lib64/ld64.so.2 (0x00007fffb3da0000)

So the directory is /usr/local/binutils-gdb/lib and if installed prefix is /usr/local then the conf dir will be /usr/local/lib
Comment 26 Dmitry Selyutin 2022-06-13 21:02:57 BST
(In reply to vklr@vkten.in from comment #25)
> (In reply to Dmitry Selyutin from comment #23)
> Find out in which directory libopcodes-2.38.50.20220525.so and
> libbfd-2.38.50.20220525.so are installed and add that to
> /etc/ld.so.conf.d/binutils-gdb.conf and run 'ldconfig' as root. Thats it.

No, this is wrong. Any user should have things working right after launching the script. This is absolutely critical not to force user to do something on their own, that's the whole point of these install scripts.
Comment 27 Veera 2022-06-13 22:20:56 BST
(In reply to Dmitry Selyutin from comment #26)
> (In reply to vklr@vkten.in from comment #25)
> > (In reply to Dmitry Selyutin from comment #23)
> > Find out in which directory libopcodes-2.38.50.20220525.so and
> > libbfd-2.38.50.20220525.so are installed and add that to
> > /etc/ld.so.conf.d/binutils-gdb.conf and run 'ldconfig' as root. Thats it.
> 
> No, this is wrong. Any user should have things working right after launching
> the script. This is absolutely critical not to force user to do something on
> their own, that's the whole point of these install scripts.

Fixed and updated script which runs ldconfig while installation. Runs fine, checked.

Please give feedback.

https://git.libre-soc.org/?p=dev-env-setup.git;a=commitdiff;h=77236b292ab715cd5a7f9fb11ad36941b650ea74
Comment 28 Veera 2022-06-16 03:09:29 BST
Updated install script with isl support for binutils.

uri: https://git.libre-soc.org/?p=dev-env-setup.git;a=commitdiff;h=545b276042a73e99ddaa71a977e8efd461c9bfc8

Review will be helpful.
Comment 29 Luke Kenneth Casson Leighton 2022-07-19 13:24:53 BST
Dmitry i am re-submitting this via the "secret" URLs
in one single combined RFP covering #847 #857 and #867
and updating the submitted date to 2022-07-19 on all 3.