Bug 291 - HDL Workflow and Coriolis2 chroot automated setup scripts
Summary: HDL Workflow and Coriolis2 chroot automated setup scripts
Status: RESOLVED FIXED
Alias: None
Product: Libre-SOC's first SoC
Classification: Unclassified
Component: Hardware Layout (show other bugs)
Version: unspecified
Hardware: PC Linux
: --- normal
Assignee: Cole Poirier
URL:
Depends on:
Blocks:
 
Reported: 2020-04-21 22:42 BST by Cole Poirier
Modified: 2020-12-02 20:36 GMT (History)
3 users (show)

See Also:
NLnet milestone: NLNet.2019.02.029.Coriolis2
total budget (EUR) for completion of task and all subtasks: 200
budget (EUR) for this task, excluding subtasks' budget: 200
parent task for budget allocation: 205
child tasks for budget allocation:
The table of payments (in EUR) for this task; TOML format:
cole={amount=200,submitted=2020-09-20,paid=2020-10-01}


Attachments
Take a correct snaphost of the Coriolis environment in ~/coriolisenv (610 bytes, patch)
2020-05-18 21:26 BST, Jean-Paul Chaput
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Cole Poirier 2020-04-21 22:42:22 BST
In service of the description of bug 178: "We need the alliance2/coriolis workflow documented, a suite of tutorials found (or written), and a "test project" to be done which gives a guide to completion time of layout." Cole is working on scripts to automate the process of installation and setup of the HDL workflow development environment. This should make the on-boarding of the LIPFR6 interns who will be doing the layout this summer easier.
Comment 1 Cole Poirier 2020-04-21 22:46:10 BST
Luke, should I amend section 6.6 of the HDL workflow to install gcc-8-powerpc64-linux-gnu as you said this is what is available in debian/10? The current instructions say to install gcc-9-powerpc64-linux-gnu but as you said that's only available in debian/testing. Since you said that there should be no need for compiling ppc code at this point, is this section meant to be a part of a different workflow?
Comment 2 Cole Poirier 2020-05-12 20:41:04 BST
Luke, I was able to get the script to work to the point of correctly installing and configuring the coriolis chroot such that it would open the coriolis window diplaying the chicken, which when clicked on a few times displayed the coriolis layout window as on the HDL workflow wiki. Unfortunately my script no longer reaches this point of successful completion, and I am now consistently getting stuck at this error:
```
-- CORIOLIS_TOP is set to /home/ace/coriolis-2.x/Linux.x86_64/Release.Shared/install
-- Components of CORIOLIS_DIR_SEARCH:
--   /home/ace/coriolis-2.x/Linux.x86_64/Release.Shared/install
-- Found CORIOLIS_PYTHON:
--   /home/ace/coriolis-2.x/Linux.x86_64/Release.Shared/install/lib64/libpycrlcore.so
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ace/coriolis-2.x/Linux.x86_64/Release.Shared/build/unittests
Make/Ninja command: ['make', '-j4', 'install']
Scanning dependencies of target unittests
[ 50%] Building CXX object src/CMakeFiles/unittests.dir/unittests.cpp.o
[100%] Linking CXX executable unittests
[100%] Built target unittests
Install the project...
-- Install configuration: "Release"
-- Installing: /home/ace/coriolis-2.x/Linux.x86_64/Release.Shared/install/bin/unittests
-- Set runtime path of "/home/ace/coriolis-2.x/Linux.x86_64/Release.Shared/install/bin/unittests" to ""
Traceback (most recent call last):
  File "./cgt", line 23, in <module>
    helpers.showPythonTrace( sys.argv[0], e )
NameError: name 'helpers' is not defined
```
Comment 3 Luke Kenneth Casson Leighton 2020-05-13 01:09:29 BST
(In reply to Cole Poirier from comment #2)
> Luke, I was able to get the script to work to the point of correctly
> installing and configuring the coriolis chroot such that it would open the
> coriolis window diplaying the chicken, which when clicked on a few times
> displayed the coriolis layout window as on the HDL workflow wiki.
> Unfortunately my script no longer reaches this point of successful
> completion, and I am now consistently getting stuck at this error:

ok so this "per se" is of no help as without knowing "which script"
there is no way to know what it is doing.  until we know where that
script is, and can examine it (and run it) there's not a lot that can
be done.
Comment 4 Cole Poirier 2020-05-13 01:15:00 BST
(In reply to Luke Kenneth Casson Leighton from comment #3)
> (In reply to Cole Poirier from comment #2)
> ok so this "per se" is of no help as without knowing "which script"
> there is no way to know what it is doing.  until we know where that
> script is, and can examine it (and run it) there's not a lot that can
> be done.

Oops, sorry, I forgot to link the script's git repo. This error occurs when running the script 'coriolis2-chroot' after running the script 'install-apt-reqs' from this repo https://git.libre-soc.org/?p=dev-env-setup.git;a=summary.
Comment 5 Cole Poirier 2020-05-13 01:16:35 BST
Note, theres a very high probability that I am making very silly bash scripting mistakes because I'm new to this. So any help and corrections are greatly appreciated :)
Comment 6 Luke Kenneth Casson Leighton 2020-05-13 02:19:01 BST
ok so there's nothing in there which says "picking a hard-coded git tag number"
so i suspect that it's as simple as jean-paul made a commit recently which
broke something.
Comment 7 Cole Poirier 2020-05-13 03:00:18 BST
(In reply to Luke Kenneth Casson Leighton from comment #6)
> ok so there's nothing in there which says "picking a hard-coded git tag
> number"
> so i suspect that it's as simple as jean-paul made a commit recently which
> broke something.

Ah, I'm realizing now that I really should have been using a hard-coded git tag number. Can you share your known-working git tag number for coriolis2 so I can continue working on the script tomorrow?
Comment 8 Luke Kenneth Casson Leighton 2020-05-13 10:19:25 BST
(In reply to Cole Poirier from comment #7)
> (In reply to Luke Kenneth Casson Leighton from comment #6)
> > ok so there's nothing in there which says "picking a hard-coded git tag
> > number"
> > so i suspect that it's as simple as jean-paul made a commit recently which
> > broke something.
> 
> Ah, I'm realizing now that I really should have been using a hard-coded git
> tag number.

ah - no, not exactly.  keep it simple for now.  the idea we came up with
is to have a new (single) repo with git submodules.

the git branch/tag info will be encoded into the submodules.  this way,
the script remains the same.

> Can you share your known-working git tag number for coriolis2 so
> I can continue working on the script tomorrow?

wait for jean-paul to investigate:
https://gitlab.lip6.fr/vlsi-eda/coriolis/-/issues/13
Comment 9 Cole Poirier 2020-05-13 18:58:39 BST
(In reply to Luke Kenneth Casson Leighton from comment #8)
> (In reply to Cole Poirier from comment #7)
> > Ah, I'm realizing now that I really should have been using a hard-coded git
> > tag number.
> 
> ah - no, not exactly.  keep it simple for now.  the idea we came up with
> is to have a new (single) repo with git submodules.
> 
> the git branch/tag info will be encoded into the submodules.  this way,
> the script remains the same.
> 
> > Can you share your known-working git tag number for coriolis2 so
> > I can continue working on the script tomorrow?
> 
> wait for jean-paul to investigate:
> https://gitlab.lip6.fr/vlsi-eda/coriolis/-/issues/13

Ok, thank you for the explanation, makes much more sense to me now. I will move on to the non-chroot part of the HDL workflow for now.
Comment 10 Jean-Paul Chaput 2020-05-18 21:26:20 BST
Created attachment 56 [details]
Take a correct snaphost of the Coriolis environment in ~/coriolisenv
Comment 11 Jean-Paul Chaput 2020-05-18 21:30:24 BST
I did miss to attach this to the previous attachement.

My guess for the problem, is that the Coriolis environment was badly set, hence the helpers import failure. The patch contains a correction. I was not able to check it in the exact same conditions as the script assumes that the host OS is already a Debian like.
Comment 12 Luke Kenneth Casson Leighton 2020-05-18 21:37:45 BST
(In reply to Jean-Paul.Chaput from comment #11)
> I did miss to attach this to the previous attachement.
> 
> My guess for the problem, is that the Coriolis environment was badly set,
> hence the helpers import failure. The patch contains a correction. I was not
> able to check it in the exact same conditions as the script assumes that the
> host OS is already a Debian like.

thank you jean-paul.
Comment 13 Cole Poirier 2020-05-18 22:02:46 BST
(In reply to Jean-Paul.Chaput from comment #11)
> I did miss to attach this to the previous attachement.
> 
> My guess for the problem, is that the Coriolis environment was badly set,
> hence the helpers import failure. The patch contains a correction. I was not
> able to check it in the exact same conditions as the script assumes that the
> host OS is already a Debian like.

Thank you very much Jean-Paul! I appreciate how busy you are, and how valuable your time is.
Comment 14 Cole Poirier 2020-05-18 23:15:35 BST
I have applied Jean-Paul's patch, committed and pushed it, and am now continuing testing of the script.
Comment 15 Cole Poirier 2020-07-28 20:35:20 BST
The script is tested and complete. The wiki docs are in sync with the script. Closing this. Will create new purpose-specific bug if there is a coriolis2 issue in the future, or if changes need to be made to the setup scripts.
Comment 16 Cole Poirier 2020-09-15 19:55:14 BST
Update budget allocations and parent task for budget allocations per http://lists.libre-soc.org/pipermail/libre-soc-dev/2020-September/000536.html