Created attachment 142 [details] Full log (everything that was sent by redirection from standard out) from clean run of "coriolis_install" Installation occured on fresh installation of Debian Buster 10.10. The latest "dev-env-setup" scripts were used (also tested before my commits changed "mk-deb-chroot", same issue). First, "mk-deb-chroot" was invoked and the chroot [libresoc] was created. Then "dev-env-setup" folder was copied from the host system to the newly created chroot (into /opt/chroot/$chrootdir/home/$SUDO_USER/). After getting into the chroot environment with schroot -c libresoc the script "coriolis_install" was called (as user, not root). The script was able to install the necessary apt packages, yosys, and coriolis-2.x, however produced an error when installing alliance. Attached is the full log (everything that was sent by redirection from standard out) from clean run of "coriolis_install". The error is related to the library libMut. Searching for the relevant files using "find . -name libMut*", I find the expected library shared objects: ./build/mbk/src/.libs/libMut.so.4.0.2 ./build/mbk/src/.libs/libMut.so.4 ./build/mbk/src/.libs/libMut.so ./build/mbk/src/.libs/libMut.a ./build/mbk/src/.libs/libMut.lai ./build/mbk/src/.libs/libMut.la ./build/mbk/src/libMut.la ./install/lib/libMut.so.4.0.2 ./install/lib/libMut.so.4 ./install/lib/libMut.so ./install/lib/libMut.la ./install/lib/libMut.a This is a blocker, because lvx and cgt fail to properly install (the example arm_chip_cts_r" is not found). I'm not sure if my system is missing certain packages/libraries, or if there's another issue.
the script does not start with "#!/bin/bash" so when executed cannot find the "source" command
Added the missing script type (#!/bin/bash), and setup a new chroot, re-ran everything again. The same error about libMut still occured. One thing I noticed is as the script kept going and tried to "make lvx": [ERROR] CParsVst() VHDL Parser: Unable to find cell "pvssick_px", please check your <./coriolis2/settings.py>. Python stack trace: #0 in <module>() at /home/rohdo/alliance-check-toolkit/bin/doChip.py:322 make: [mk/pr-coriolis.mk:65: arm_chip_cts_r.vst] Error 1 (ignored) touch *_cts.* MBK_OUT_LO=al; export MBK_OUT_LO; MBK_SEPAR='_'; export MBK_SEPAR; /home/rohdo/alliance/install/bin/cougar -c -f arm_chip_cts_r arm_chip_cts_r_ext /bin/sh: 1: /home/rohdo/alliance/install/bin/cougar: not found make: *** [mk/alliance.mk:50: arm_chip_cts_r_ext.al] Error 127 When I tested with a new chroot, I used "mk-deb-chroot" from commit e347189573501565debe9b52ce9aa87e49de183d (the one before Luke's 049b1b260f72b41a06d1e706abf013ace634dbea). Has that change with the /tmp directory changed something significant Luke?
libAbl_la_LDFLAGS = -version-info @ABL_DLL_VERSION@ -L../../aut/src \ -L../../mbk/src add extra bit to src/abl/src/Makefile.am rerun autostuff (manually) rerun make install (manually, as root) repeat until success i have encountered this before, it is a libtool f***up.
jean-paul, this is a weird one: despite the fact that there has been no discernible change in the build script, alliance has suddenly stopped building. do you remember the coriolis2 patches i submitted for building on debian, where all those libraries could not be found? and it was libtool-related, a known issue, where recursive dependencies had to be explicitly listed at the top level? i think this is the same thing. although, there may be something else, very weird, going on: a "git diff" shows a massive set of comparative changes which is not normal. many of them showing removal of code which is the same library causing issues.
jean-paul, false alarm, coriolisenv was empty, no news yet as to how or why. andrey as a temporary measure rerun coriolisEnv.py put its output into ~/coriolisenv source that.
andrey remaking coriolisenv "fixed" things. however i had to handedit the file to change /home//alliance to /home/{HOMEUSERACCOUNTNAME}/alliance without that the install library location is not found hence the reason why, obviously, the libraries cannot be found.
maje lvx of ARM cmos underway. looks all good. just script is borked not alliance not coriolis2 $SUDO_USER likely empty var. needs #!/bin/bash at top
(In reply to Luke Kenneth Casson Leighton from comment #3) > libAbl_la_LDFLAGS = -version-info @ABL_DLL_VERSION@ -L../../aut/src \ > -L../../mbk/src > > add extra bit to src/abl/src/Makefile.am > > rerun autostuff (manually) > rerun make install (manually, as root) > > repeat until success > > i have encountered this before, it is a libtool f***up. This problem did also show up on my hand, but "sometimes" only. I must confess I don't really understand why. I nevertheless made commit 720e37e that *seems* to solve it. Always wanted to migrate to CMake, but no time yet though.
> This problem did also show up on my hand, but "sometimes" only. > I must confess I don't really understand why. I nevertheless made > commit 720e37e that *seems* to solve it. > Always wanted to migrate to CMake, but no time yet though. F**k. My "end", not my "hand".
(In reply to Luke Kenneth Casson Leighton from comment #5) > andrey as a temporary measure rerun coriolisEnv.py put its output into > ~/coriolisenv > > source that. Did as you said, my coriolisenv was indeed empty. (In reply to Luke Kenneth Casson Leighton from comment #6) > /home//alliance > to > /home/{HOMEUSERACCOUNTNAME}/alliance > > without that the install library location is not found hence > the reason why, obviously, the libraries cannot be found. Also added the missing {HOMEUSERACCOUNTNAME}. (In reply to Luke Kenneth Casson Leighton from comment #7) > $SUDO_USER likely empty var. > needs #!/bin/bash at top I added "#!/bin/bash" to the top of coriolisenv, is that where you added it? Still the same issue with libMut occurred. (In reply to Luke Kenneth Casson Leighton from comment #3) > libAbl_la_LDFLAGS = -version-info @ABL_DLL_VERSION@ -L../../aut/src \ > -L../../mbk/src > > add extra bit to src/abl/src/Makefile.am I also made the change to "src/abl/src/Makefile.am" and now during "sudo make install" I get an error about libAut: /usr/bin/ld: cannot find -lAut collect2: error: ld returned 1 exit status make[2]: *** [Makefile:628: libBdd.la] Error 1 make[2]: Leaving directory '/home/rohdo/alliance/build/bdd/src' make[1]: *** [Makefile:524: install-recursive] Error 1 make[1]: Leaving directory '/home/rohdo/alliance/build/bdd' make: *** [Makefile:570: install-recursive] Error 1 > > rerun autostuff (manually) > rerun make install (manually, as root) > > repeat until success How many times should I try? I tried three times. Also at the end of autostuff, I get the following warnings: configure.ac:3: https://www.gnu.org/software/automake/manual/automake.html#Modernize-AM_005fINIT_005fAUTOMAKE-invocation configure.ac:5: installing './compile' configure.ac:3: installing './missing' abe/src/Makefile.am: installing './depcomp' ocp/src/placer/Makefile.am:5: warning: 'INCLUDES' is the old name for 'AM_CPPFLAGS' (or '*_CPPFLAGS') They are probably not important, right?
After looking at Luke's git merges for "coriolis_install" script (only came back to it today), noticed again the issue with "coriolisenv" file containing "//" in the PATHs (missing the username contained in $SUDO_USER variable of whoever runs the script). For example: LD_LIBRARY_PATH="/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64:/home//alliance/install/lib64:/home//alliance/install/lib:/home//alliance/install/lib64:/home//alliance/install/lib:"; This is caused, not by an issue in coriolisEnv.py, but because the PATH variable already had the $SUDO_USER missing. This actually happens earlier in "coriolis_install" script. The "echo -e" statement (see line 28 of the script) that writes into .bash_profile was using single quotes ('') and thus was not expanding the $SUDO_USER (which is defined at the script level, and doesn't exist at the shell level where the echo statement is executed). I'm working on a fix atm, and will have it done tomorrow. Then I'll give the coriolis installation another go, and this time (hopefully) alliance should see all the needed libraries for compilation. The reason we didn't see such an issue before, is probably because scripts were always run as root via "sudo bash" which always adds the $SUDO_USER environment variable. Running a script as a normal user means $SUDO_USER doesn't exist, and so has to be defined locally. Variable expansion would then be done within the script (using double quotes "") before the command is run in the shell. Otherwise we could define our own variable with a different name, as manually defining $SUDO_USER outside a script is probably a bad idea. The linux environment doesn't seem to define an environment variable for the current user (there is a var for user id though).
(In reply to andrey from comment #11) > After looking at Luke's git merges for "coriolis_install" script (only came welcome back > The reason we didn't see such an issue before, is probably because scripts > were always run as root via "sudo bash" which always adds the $SUDO_USER > environment variable. yes. there are checks which prevent you from proceeding without it. > Otherwise we could define our own variable with a different name, as > manually defining $SUDO_USER outside a script is probably a bad idea. > The linux environment doesn't seem to define an environment variable for the > current user (there is a var for user id though). $USER. which when calling runuser will be set correctly.
(In reply to Luke Kenneth Casson Leighton from comment #12) > (In reply to andrey from comment #11) > $USER. > > which when calling runuser will be set correctly. Thanks! I swapped the $SUDO_USER for $USER instead. Now at the start of "coriolis_install" I added a check so that only a normal user runs the script (not user id 0). Another thing I noticed (which was occuring before) was that the "coriolisEnv.py" script wasn't being executed by Python interpreter. Running the command: $ /home/$USER/coriolis-2.x/src/coriolis/bootstrap/coriolisEnv.py > /home/$USER/coriolisenv Gives: Traceback (most recent call last): File "/home/rohdo/coriolis-2.x/src/coriolis/bootstrap/coriolisEnv.py", line 133, in <module> shellBin, isBourneShell = guessShell() File "/home/rohdo/coriolis-2.x/src/coriolis/bootstrap/coriolisEnv.py", line 111, in guessShell shellPath = whichCommand.stdout.readlines()[0][:-1] IndexError: list index out of range Is it to do with the $TERM or $SHELL variables? My chroot's are set to $TERM=xterm-256color $SHELL=/bin/bash
(In reply to andrey from comment #13) > Running the command: > $ /home/$USER/coriolis-2.x/src/coriolis/bootstrap/coriolisEnv.py > > /home/$USER/coriolisenv > > Gives: > Traceback (most recent call last): > File "/home/rohdo/coriolis-2.x/src/coriolis/bootstrap/coriolisEnv.py", > line 133, in <module> > shellBin, isBourneShell = guessShell() > File "/home/rohdo/coriolis-2.x/src/coriolis/bootstrap/coriolisEnv.py", > line 111, in guessShell > shellPath = whichCommand.stdout.readlines()[0][:-1] > IndexError: list index out of range > > Is it to do with the $TERM or $SHELL variables? This is the function in question: def guessShell (): # This environement variable cannot be trusted as it is set once when # the user logs in. If aftewards it changes it that variable is *not* # affected :-(. #if os.environ.has_key('SHELL'): return os.environ['SHELL'] psCommand = subprocess.Popen ( ['ps', '-p', str(os.getppid()) ], stdout=subprocess.PIPE ) shell = psCommand.stdout.readlines()[1].decode('ascii')[:-1].split()[3].lstrip('-') whichCommand = subprocess.Popen ( ['which', shell ], stdout=subprocess.PIPE ) shellPath = whichCommand.stdout.readlines()[0][:-1] isBourneShell = True cshBins = [ '/usr/bin/tcsh' , '/bin/tcsh' , '/usr/pkg/bin/tcsh' , '/usr/local/bin/tcsh' , '/usr/bin/csh' , '/bin/csh' , '/usr/pkg/bin/csh' , '/usr/local/bin/csh' ] if shellPath in cshBins: isBourneShell = False #print( 'GUESSED SHELL: "{}"'.format( shellPath )) return shellPath, isBourneShell
remember, this all "works fine" and has done for years, so there is something wrong / missing from the new script, which needs tracking down. suggest running the actual "which" command to see its output. modify coriolisEnv.py to get it to print it out for you
I ran the coriosEnv.py script with the python debugger, and noticed that the python code inserts an error message into "coriolisenv" output: PYTHONPATH="/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/etc/coriolis2:sitePackageDir_has_been_not_found/stratus:sitePackageDir_has_been_not_found/cumulus/plugins:sitePackageDir_has_been_not_found/cumulus:sitePackageDir_has_been_not_found/crlcore:sitePackageDir_has_been_not_found:";export PYTHONPATH; In my installation of Debian Buster 10.10, PYTHONPATH has not been defined, however this is not a requirement in coriolis or alliance documentation. The coriosEnv.py on line 259 defines a placeholder string: sitePackagesDir = "sitePackageDir_has_been_not_found" Found the place where the placeholder should be overriden (lines 257-266): pyVersion = sys.version_info version = "%d.%d" % (pyVersion[0],pyVersion[1]) sitePackagesDir = "sitePackageDir_has_been_not_found" for pyPackageDir in [ "%s/python%s/site-packages" % (absLibDir,version) , "%s/python%s/dist-packages" % (absLibDir,version) , "%s/%s/site-packages" % (absLibDir,version) ]: if os.path.isdir(pyPackageDir): sitePackagesDir = pyPackageDir break The "version" variable which is returned is "3.7", and the script searches for these paths: '/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python3.7/site-packages' '/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python3.7/dist-packages' '/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/3.7/site-packages' However my coriolis path uses the directory name "python3", not "python3.7". Thus the placeholder remains and produces incorrect "coriolisenv". Do you think this is a change in coriolis or a debian issue? What's the name of the python folder you find under "/home/$USER/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/"? Modifying the "coriolisEnv.py" to only use the major version (3), the script produces "seemingly" correct output. After clearing the build directory I re-ran the compilation of alliance, which had lasted longer than last time, and it failed again with the same "-lMut" error. The full coriolisenv (after bodging to not use minor python version): echo "Using user-selected Coriolis 2 (/home/rohdo/coriolis-2.x)";echo "Switching to Coriolis 2.x (Release.Shared)";PATH="/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/bin:/usr/lib/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games";BOOTSTRAP_TOP="/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install";CORIOLIS_TOP="/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install";export PATH BOOTSTRAP_TOP CORIOLIS_TOP STRATUS_MAPPING_NAME;PYTHONPATH="/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/etc/coriolis2:/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python3/dist-packages/stratus:/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python3/dist-packages/cumulus/plugins:/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python3/dist-packages/cumulus:/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python3/dist-packages/crlcore:/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python3/dist-packages:";export PYTHONPATH;LD_LIBRARY_PATH="/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64:/home/rohdo/alliance/install/lib64:/home/rohdo/alliance/install/lib:";export LD_LIBRARY_PATH;hash -r;
(In reply to andrey from comment #16) > However my coriolis path uses the directory name "python3", not "python3.7". this will fail. coriolis2 only works with python 2.7. if that's not installed then it's a bug in earlier parts of the install process. i spent some considerable time 12 or so days ago bashing coriolis_install into shape so it would successfully install python2.7 are you re-running the script entirely from scratch after 100% deleting the entirety of the (broken) chroot, each time? l.
https://gitlab.lip6.fr/vlsi-eda/coriolis/-/tags/LS180_RC7_FINAL oo hang on. Jean-Paul is in the middle of moving to python3. it's quite important not to try to use the version he's in the middle of, but instead to use the tag LS180_RC7_FINAL. rats
(In reply to Luke Kenneth Casson Leighton from comment #17) > coriolis2 only works with python 2.7. Ok, didn't realise that, will keep in mind now. > i spent some considerable time 12 or so days ago bashing > coriolis_install into shape so it would successfully install > python2.7 > > are you re-running the script entirely from scratch after > 100% deleting the entirety of the (broken) chroot, each time? I didn't. Now created a new schroot and will make sure python2.7 is specified. (In reply to Luke Kenneth Casson Leighton from comment #18) > https://gitlab.lip6.fr/vlsi-eda/coriolis/-/tags/LS180_RC7_FINAL > > oo hang on. > > Jean-Paul is in the middle of moving to python3. it's quite important > not to try to use the version he's in the middle of, but instead to > use the tag LS180_RC7_FINAL. > > rats Ok, I'll modify "coriolis_install" to pull the LS180_RC7_FINAL tag.
Hello All, This is captain speaking... Just to add some precision to what Luke has said: * The first stage of Coriolis Python 3 migration is finished with the current latest commit. That is it works with Python 3, but using the portage of the old C-Macro style wrapper. alliance-check-toolkit has also been updated to Python 3 accordingly. But not soclayout (yet). Shouldn't be difficult. * I'm currently working to do the port on the new C++/template wrapper. The wrapper itself is complete and I'm about to start the mass migration. The two wrappers (C-Macro & C++/template, both on Python3 can cohabit). The installation process should already work with Python 3 as it is now so it may be worth investigating. Best,
(In reply to Jean-Paul.Chaput from comment #20) > Hello All, > > This is captain speaking... Just to add some precision to what Luke has said: :) > The installation process should already work with Python 3 as it is now > so it may be worth investigating. we need something that builds ls180 for the team in India, there are going to be 500+ students trying to do coriolis2 builds.
Didn't work again, same libMut error. 1. Created new environment using "mk-deb-chroot" (as root). 2. Copied the dev-env-scripts to new schroot using "cp-scripts-to-chroot.sh" (as root). 3. Get into schroot and run "coriolis_install" script as normal user. Script fails to execute "coriolisEnv.py" (always fails when run in the shell for me, same error as in comment #13). Also Alliance install (sudo make -j1 install) fails with: /usr/bin/ld: cannot find -lMut collect2: error: ld returned 1 exit status make[2]: *** [Makefile:601: libAbl.la] Error 1 make[2]: Leaving directory '/home/rohdo/alliance/build/abl/src' make[1]: *** [Makefile:524: install-recursive] Error 1 make[1]: Leaving directory '/home/rohdo/alliance/build/abl' make: *** [Makefile:570: install-recursive] Error 1 4. Generate coriolisenv manually by running: /home/$USER/coriolis-2.x/src/coriolis/bootstrap/coriolisEnv.py > /home/$USER/coriolisenv 5. Cleared "alliance" and "alliance-check-toolkit" build. 6. Modifying Makefile.am as you suggested in comment #3. 7. Re-run autostuff 9. Source coriolisenv 10. ../src/configure --prefix=/home/$USER/alliance/install --enable-alc-shared 11. Alliance install fails, now with the following: /usr/bin/ld: cannot find -lAut collect2: error: ld returned 1 exit status make[2]: *** [Makefile:628: libBdd.la] Error 1 make[2]: Leaving directory '/home/rohdo/alliance/build/bdd/src' make[1]: *** [Makefile:524: install-recursive] Error 1 make[1]: Leaving directory '/home/rohdo/alliance/build/bdd' make: *** [Makefile:570: install-recursive] Error 1 Now it looks like the src/bdd/src/Makefile.am is complaining. That file has the following lines at the end: libBdd_la_LDFLAGS = -version-info @BDD_DLL_VERSION@ -L../../abl/src -L../../mbk/src libBdd_la_LIBADD = -lAbl -lAut Might seem naive of me, but is "libBdd_la_LIBADD" wrong? The LDFLAGS make me assume LIBADD should be -lAbl and -lMut. Is it worth re-installing my host Debian? If you're able to run the same sequence of scripts, and alliance installs without problem, then there must be either: -difference in Debian versions (mine's 10.10) -missing packages that you might have that I don't (no packages other than what's in "mk-deb-chroot" and "coriolis_install" were installed. -missing/misconfigured environment vars -something else that I missed (I noticed nothing else in coriolis2 wiki or libre-soc wiki)
this is what coriolisEnv.py should be creating: it is obviously including LD_LIBRARY_PATH pointing to the correct location - the libraries. until you have LD_LIBRARY_PATH pointing to the correct location you are *not* going to get past this error. the fix *is* to have coriolisenv extend LD_LIBRARY_PATH whereupon autoconf will find it and add the correct instructions into Makefiles investigating alliance and coriolis2 source code is unnecessary: there is nothing wrong. the problem *is* in the output of coriolisEnv.py - which is also correct and has nothing wrong with it - the problem *is* in the environment variables passed to the script and with ~/.bashrc and so on. you will find a missing entry in ~/.bashrc that looks like: PATH=/home//xxxxx or other entry where the ${USER} formerly ${SUDO_USER} did not expand out correctly. source /tmp/coriolisenv echo "Using user-selected Coriolis 2 (/home/lkcl/coriolis-2.x)";echo "Switching to Coriolis 2.x (Release.Shared)";PATH="/home/lkcl/coriolis-2.x/Linux.x86_64/Release.Shared/install/bin:/usr/lib/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games";BOOTSTRAP_TOP="/home/lkcl/coriolis-2.x/Linux.x86_64/Release.Shared/install";CORIOLIS_TOP="/home/lkcl/coriolis-2.x/Linux.x86_64/Release.Shared/install";export PATH BOOTSTRAP_TOP CORIOLIS_TOP STRATUS_MAPPING_NAME;PYTHONPATH="/home/lkcl/coriolis-2.x/Linux.x86_64/Release.Shared/install/etc/coriolis2:/home/lkcl/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib/python2.7/dist-packages/stratus:/home/lkcl/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib/python2.7/dist-packages/cumulus/plugins:/home/lkcl/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib/python2.7/dist-packages/cumulus:/home/lkcl/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib/python2.7/dist-packages/crlcore:/home/lkcl/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib/python2.7/dist-packages:";export PYTHONPATH;LD_LIBRARY_PATH="/home/lkcl/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib:/home/lkcl/alliance/install/lib:";export LD_LIBRARY_PATH;hash -r;
(In reply to andrey from comment #22) > libBdd_la_LIBADD = -lAbl -lAut > Might seem naive of me, but is "libBdd_la_LIBADD" wrong? no, because one of those two libraries is built with -lMut, and consequently it is found that way (recursively) there is nothing wrong here - it is just a symptom. you are investigating symptoms of the fact that LD_LIBRARY_PATH has not been set, because the .bash_profile / .bashrc was not properly established (a missing env variable) > The LDFLAGS make me assume LIBADD should be -lAbl and -lMut. LD_LIBRARY_PATH does not have the correct entries, therefore of course -lMut, which is not installed in /usr/lib it is a user-defined path location, cannot be found. > > Is it worth re-installing my host Debian? no, not at all. > -missing/misconfigured environment vars this is the source of the problem.
(In reply to Luke Kenneth Casson Leighton from comment #21) > (In reply to Jean-Paul.Chaput from comment #20) > > The installation process should already work with Python 3 as it is now > > so it may be worth investigating. > > we need something that builds ls180 for the team in India, there are going > to be 500+ students trying to do coriolis2 builds. I will check it on my chrooted Debian 10 before this weekend to see if I can reproduce the problem and correct it.
(In reply to andrey from comment #13) > (In reply to Luke Kenneth Casson Leighton from comment #12) > > (In reply to andrey from comment #11) > > $USER. > > > > which when calling runuser will be set correctly. > Thanks! I swapped the $SUDO_USER for $USER instead. Now at the start of > "coriolis_install" I added a check so that only a normal user runs the > script (not user id 0). Normally you'd get the current user's home directory by using "$HOME/my_file", rather than "/home/$USER/my_file". Linux/Unix has a convenient shortcut for $HOME since using it is soo common: you use ~/my_file
(In reply to Jacob Lifshay from comment #26) > Normally you'd get the current user's home directory by using > "$HOME/my_file", rather than "/home/$USER/my_file". Linux/Unix has a > convenient shortcut for $HOME since using it is soo common: you use ~/my_file docs for ~: https://www.gnu.org/software/bash/manual/html_node/Tilde-Expansion.html
(In reply to Jacob Lifshay from comment #26) > Normally you'd get the current user's home directory by using > "$HOME/my_file", rather than "/home/$USER/my_file". Linux/Unix has a > convenient shortcut for $HOME since using it is soo common: you use ~/my_file right, ok, normally, yes, however in this case we don't want that because of the way it is used on uoregon's servers.
(In reply to Jean-Paul.Chaput from comment #25) > I will check it on my chrooted Debian 10 before this weekend to > see if I can reproduce the problem and correct it. thx JP. i should probably ask university of oregon if you can have access to the server.
(In reply to Luke Kenneth Casson Leighton from comment #23) > until you have LD_LIBRARY_PATH pointing to the correct location > you are *not* going to get past this error. > the problem *is* in the output of coriolisEnv.py - which is also > correct and has nothing wrong with it - the problem *is* in > the environment variables passed to the script and with ~/.bashrc > and so on. > > you will find a missing entry in ~/.bashrc that looks like: > > PATH=/home//xxxxx > > or other entry where the ${USER} formerly ${SUDO_USER} did not expand > out correctly. After checking .bash_profile, I saw that there are additional paths: export LD_LIBRARY_PATH=${ALLIANCE_TOP}/lib:${LD_LIBRARY_PATH} export LD_LIBRARY_PATH=${ALLIANCE_TOP}/lib64:${LD_LIBRARY_PATH} Running "source ~/.bash_profile" adds these additional entries on top of "coriolisenv": /home/rohdo/alliance/install/lib64: /home/rohdo/alliance/install/lib: The first of these entries points to directory "lib64" in alliance which doesn't exist for me. The second is a copy of an entry from coriolisenv (so is redundant). No other missing or broken exist in .bash_profile, .bash_rc, or .profile. (In reply to Luke Kenneth Casson Leighton from comment #24) > > -missing/misconfigured environment vars > > this is the source of the problem. Ok checked my coriolisenv against yours. LD_LIBRARY_PATH is meant to have library paths from coriolis: /home/$USER/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib and from alliance: /home/$USER/alliance/install/lib Again I re-created the schroot, re-compiled yosys, coriolis, and checked that coriolisenv is correct. Installing alliance returns the -lMut error as before. The only difference between luke's coriolisenv and mine, as that his coriolis library path uses the name "lib": /home/lkcl/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib where's mine has "lib64" /home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64 The coriolis I installed uses the folder "lib64" so that's why there's a difference ("lib" directory doesn't exist in my installation). (In reply to Jacob Lifshay from comment #26) > Normally you'd get the current user's home directory by using > "$HOME/my_file", rather than "/home/$USER/my_file". Linux/Unix has a > convenient shortcut for $HOME since using it is soo common: you use ~/my_file I do know of this shortcut as I normally use it, but in this script's case I was following the existing nomenclature (also there are other commands that require $USER anyway). (In reply to Jacob Lifshay from comment #27) > docs for ~: > https://www.gnu.org/software/bash/manual/html_node/Tilde-Expansion.html Thanks for the info! Found no other missing/broken entries in coriolisenv. > source /tmp/coriolisenv Here's my coriolisenv (added newline chars for clarity): echo "Using user-selected Coriolis 2 (/home/rohdo/coriolis-2.x)"; echo "Switching to Coriolis 2.x (Release.Shared)"; PATH="/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/bin: /usr/lib/ccache: /usr/local/bin: /usr/bin: /bin: /usr/local/games: /usr/games"; BOOTSTRAP_TOP="/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install"; CORIOLIS_TOP="/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install"; export PATH BOOTSTRAP_TOP CORIOLIS_TOP STRATUS_MAPPING_NAME; PYTHONPATH="/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/etc/coriolis2: /home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python2.7/dist-packages/stratus: /home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python2.7/dist-packages/cumulus/plugins: /home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python2.7/dist-packages/cumulus: /home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python2.7/dist-packages/crlcore: /home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python2.7/dist-packages:"; export PYTHONPATH; LD_LIBRARY_PATH="/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64: /home/rohdo/alliance/install/lib:"; export LD_LIBRARY_PATH; hash -r;
(In reply to andrey from comment #30) > The only difference between luke's coriolisenv and mine, as that his > coriolis library path uses the name "lib": > /home/lkcl/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib > where's mine has "lib64" > /home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64 > > The coriolis I installed uses the folder "lib64" so that's why there's a > difference ("lib" directory doesn't exist in my installation). Is this to do with me running this on a 64-bit machine?
(In reply to andrey from comment #31) > > The coriolis I installed uses the folder "lib64" so that's why there's a > > difference ("lib" directory doesn't exist in my installation). > Is this to do with me running this on a 64-bit machine? we all run on 64 bit machines. ok, so where are the libs *actually* getting installed? in a directory ending lib or ending lib64?
(In reply to Luke Kenneth Casson Leighton from comment #32) > ok, so where are the libs *actually* getting installed? > > in a directory ending lib or ending lib64? The library with the error (lMut) is in alliance. In alliance installation (*lib*): /home/rohdo/alliance/install/lib/ there are libAut, libMut, etc. In coriolis installation (*lib64*): /home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/ there are other libraries.
(In reply to andrey from comment #33) > (In reply to Luke Kenneth Casson Leighton from comment #32) > > ok, so where are the libs *actually* getting installed? > > > > in a directory ending lib or ending lib64? > The library with the error (lMut) is in alliance. remember: there is no error with -lMut *itself*, there is only a missing LD_LIBRARY_PATH, which is the means and method of *FINDING* that library (in non-standard locations). the library is there. > > In alliance installation (*lib*): > /home/rohdo/alliance/install/lib/ and that matches with the LD_LIBRARY_PATH: LD_LIBRARY_PATH="/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64: ------>>>>>>> /home/rohdo/alliance/install/lib:"; <<<<<<------ export LD_LIBRARY_PATH; > there are libAut, libMut, etc. excellent. so, because it is there, we know that there is no problem with the library libMut, because it is there. therefore there is a problem with the LD_LIBRARY_PATH environment variable not being created, or being corrupted, or not being set. what does the output of "env" say? what does the output of "env" say after running "source ~/coriolisenv"?
(In reply to Luke Kenneth Casson Leighton from comment #34) > remember: there is no error with -lMut *itself*, there is only a missing > LD_LIBRARY_PATH, which is the means and method of *FINDING* that library > (in non-standard locations). Sure, should've chosen a better way to phrase it, apologies. > therefore there is a problem with the LD_LIBRARY_PATH environment > variable not being created, or being corrupted, or not being set. > > what does the output of "env" say? SHELL=/bin/bash SCHROOT_CHROOT_NAME=libresoc SCHROOT_COMMAND=-bash SCHROOT_SESSION_ID=libresoc-9c210447-b8b4-4444-9a1e-e15f649bdfbc PWD=/home/rohdo LOGNAME=rohdo SCHROOT_ALIAS_NAME=libresoc SCHROOT_GROUP=rohdo SCHROOT_USER=rohdo HOME=/home/rohdo TERM=xterm-256color USER=rohdo DISPLAY=:0.0 SHLVL=1 SCHROOT_GID=1000 YOSYS_TOP=/home/rohdo/yosys LD_LIBRARY_PATH=/home/rohdo/alliance/install/lib64:/home/rohdo/alliance/install/lib: SCHROOT_UID=1000 PATH=/usr/lib/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games ALLIANCE_TOP=/home/rohdo/alliance/install _=/usr/bin/env > what does the output of "env" say after running "source ~/coriolisenv"? SHELL=/bin/bash SCHROOT_CHROOT_NAME=libresoc BOOTSTRAP_TOP=/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install SCHROOT_COMMAND=-bash SCHROOT_SESSION_ID=libresoc-9c210447-b8b4-4444-9a1e-e15f649bdfbc PWD=/home/rohdo LOGNAME=rohdo SCHROOT_ALIAS_NAME=libresoc SCHROOT_GROUP=rohdo SCHROOT_USER=rohdo HOME=/home/rohdo CORIOLIS_TOP=/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install PYTHONPATH=/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/etc/coriolis2:/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python2.7/dist-packages/stratus:/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python2.7/dist-packages/cumulus/plugins:/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python2.7/dist-packages/cumulus:/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python2.7/dist-packages/crlcore:/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/python2.7/dist-packages: TERM=xterm-256color USER=rohdo DISPLAY=:0.0 SHLVL=1 SCHROOT_GID=1000 YOSYS_TOP=/home/rohdo/yosys LD_LIBRARY_PATH=/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64:/home/rohdo/alliance/install/lib: SCHROOT_UID=1000 PATH=/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/bin:/usr/lib/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games ALLIANCE_TOP=/home/rohdo/alliance/install _=/usr/bin/env
(In reply to andrey from comment #35) > > what does the output of "env" say? > LD_LIBRARY_PATH=/home/rohdo/alliance/install/lib64:/home/rohdo/alliance/ > install/lib: > > what does the output of "env" say after running "source ~/coriolisenv"? > LD_LIBRARY_PATH=/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/ > lib64:/home/rohdo/alliance/install/lib: ahhh. so, now, what happens if you *manually* blow away the build directory and *manually* run the alliance build commands? (starting at ./autostuff, after deleting ~/alliance/build) https://libre-soc.org/HDL_workflow/coriolis2/ remember add *BOTH* export LD_LIBRARY_PATH=.... /lib and /lib64, just like it says in those build instructions.
(In reply to Luke Kenneth Casson Leighton from comment #36) > so, now, what happens if you *manually* blow away the build directory > and *manually* run the alliance build commands? > > (starting at ./autostuff, after deleting ~/alliance/build) Here's the run: 1. Get into schroot environment 2. Source coriolisenv loads the following: (libresoc)rohdo@firedragon:~$ echo $LD_LIBRARY_PATH /home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64:/home/rohdo/alliance/install/lib: 3. Source .bash_profile extends the variable: (libresoc)rohdo@firedragon:~$ echo $LD_LIBRARY_PATH /home/rohdo/alliance/install/lib64:/home/rohdo/alliance/install/lib:/home/rohdo/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64:/home/rohdo/alliance/install/lib: 4. Delete the alliance build folder 5. Go through the build sequence starting from ./autostuff. After runninng "sudo make -j1 install", end up with an error: /usr/bin/ld: cannot find -lMut collect2: error: ld returned 1 exit status make[2]: *** [Makefile:601: libAbl.la] Error 1 make[2]: Leaving directory '/home/rohdo/alliance/build/abl/src' make[1]: *** [Makefile:524: install-recursive] Error 1 make[1]: Leaving directory '/home/rohdo/alliance/build/abl' make: *** [Makefile:570: install-recursive] Error 1 The alliance building sequence in "coriolis_install": cd alliance/src ./autostuff cd /home/$USER/alliance/build ../src/configure --prefix=/home/$USER/alliance/install --enable-alc-shared cd /home/$USER/alliance/build source /home/$USER/coriolisenv sudo make -j1 install Sourcing coriolisenv just before "make -j1 install" would clear the additional LD_LIBRARY_PATH entries introduced by .bash_profile (looks like this might be an artefact of experimentation when writing the script?). So I did two runs (deleting build folder each time): first one I DIDN'T source coriolisenv before "make -j1 install" (thus preserving the additional entries); second one in which I DID source coriolisenv. In both cases I got the same error (shown above). > https://libre-soc.org/HDL_workflow/coriolis2/ > > > remember add *BOTH* export LD_LIBRARY_PATH=.... /lib and /lib64, just like > it says in those build > instructions. Checked the wiki instructions, not missing anything else.
(In reply to andrey from comment #37) > After runninng "sudo make -j1 install", end up with an error: > /usr/bin/ld: cannot find -lMut which means the compiler's not looking in the very location we've told it to look. this is irritating. can you do "dpkg -l" and put the results as an attachment? at this point we need to verify that the compiler and other tools are correct versions.
Created attachment 144 [details] Schroot Debian Package list Created attachment 144 [details] Schroot Debian Package list (In reply to Luke Kenneth Casson Leighton from comment #38) > which means the compiler's not looking in the very location we've > told it to look. > > this is irritating. Indeed, I've been going mad over it haha... > can you do "dpkg -l" and put the results as an attachment? > > at this point we need to verify that the compiler and other tools are correct > versions. Attached.
you'd put "sudo" in front of the alliance "make install". this attempts to log in as root... which of course does not have the environment variables set up. the install of alliance is done as a *user*, not as a system-wide install, and therefore does not require root.
Created attachment 145 [details] Alliance-check-toolkit make lvx log (In reply to Luke Kenneth Casson Leighton from comment #40) > you'd put "sudo" in front of the alliance "make install". > > this attempts to log in as root... which of course does not > have the environment variables set up. > > the install of alliance is done as a *user*, not as a system-wide > install, and therefore does not require root. What a mistake... should've combed the wiki more as that doesn't have "sudo" in front of it. Probably got confused when I was converting from root to normal user script. Apologies. So Alliance now installs which is great! However when doing "make lvx" as part of alliance-check-toolkit, I get several python errors (full log attached): ... Traceback (most recent call last): File "/home/rohdo/alliance-check-toolkit/bin/doChip.py", line 11, in <module> import Cfg ImportError: dynamic module does not define module export function (PyInit_Cfg) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/rohdo/alliance-check-toolkit/bin/doChip.py", line 40, in <module> showPythonTrace( __file__, e, False ) NameError: name 'showPythonTrace' is not defined ... *** mbk error *** alcloadphfig : unable to open file : arm_chip_cts_r.ap . make: *** [mk/alliance.mk:50: arm_chip_cts_r_ext.al] Error 100 My PYTHONPATH variable is loaded. Just before this error the makefile script uses coriolisEnv.py, so that might be an issue again? The alliance-check-toolkit steps are: source coriolisenv touch alliance-check-toolkit/etc/mk/users.d/user-$USER.mk cat <<EOG >>alliance-check-toolkit/etc/mk/users.d/user-$USER.mk export CORIOLIS_TOP=/home/$USER/coriolis-2.x/Linux.x86_64/Release.Shared/install export ALLIANCE_TOP=/home/$USER/alliance/install export CHECK_TOOLKIT=/home/$USER/alliance-check-toolkit export YOSYS_TOP=/home/$USER/yosys EOG source /home/$USER/.bash_profile cd alliance-check-toolkit/benchs/ARM/cmos make lvx make cgt The file user-$USER.mk (alliance-check-toolkit/etc/mk/users.d/user-$USER.mk) contains environment variables which are loaded during "make lvx".
(In reply to andrey from comment #41) > > the install of alliance is done as a *user*, not as a system-wide > > install, and therefore does not require root. > What a mistake... should've combed the wiki more as that doesn't have "sudo" > in front of it. Probably got confused when I was converting from root to > normal user script. Apologies. no problem. got there. > So Alliance now installs which is great! excellent. > However when doing "make lvx" as part of alliance-check-toolkit, I get > several python errors (full log attached): check the contents of coriolisenv.
(In reply to Luke Kenneth Casson Leighton from comment #42) > > However when doing "make lvx" as part of alliance-check-toolkit, I get > > several python errors (full log attached): > > check the contents of coriolisenv. looks fine - i'm going to "avoid" this one by simply compiling the adder benchmark (which is really quick) instead of the ARM one. (git pull)
(In reply to Luke Kenneth Casson Leighton from comment #43) > > check the contents of coriolisenv. > > looks fine - i'm going to "avoid" this one by simply compiling > the adder benchmark (which is really quick) instead of the ARM one. > > (git pull) Thanks! The adder compiles (place-n-routes? not sure which verb is relevant). Was the issue with the ARM chip files on my end, or have there been changes that break the flow? Also, now that setting up coriolis/alliance works (somewhat), shall I finish up the original script you requested me to do? I can make a new bug for it, as it is no longer quite relevant to this particular bug.
(In reply to andrey from comment #44) > Thanks! The adder compiles (place-n-routes? not sure which verb is relevant). > Was the issue with the ARM chip files on my end, or have there been changes > that break the flow? don't know, not concerned. > > Also, now that setting up coriolis/alliance works (somewhat), shall I finish > up the original script you requested me to do? > > I can make a new bug for it, as it is no longer quite relevant to this > particular bug. yes please, close this one.