Bug 472 - tutorial and dev page needed for mesa driver
Summary: tutorial and dev page needed for mesa driver
Status: RESOLVED FIXED
Alias: None
Product: Libre-SOC's first SoC
Classification: Unclassified
Component: Documentation (show other bugs)
Version: unspecified
Hardware: Other Linux
: Low enhancement
Assignee: Cole Poirier
URL:
Depends on:
Blocks:
 
Reported: 2020-08-24 22:27 BST by Luke Kenneth Casson Leighton
Modified: 2020-12-02 20:38 GMT (History)
3 users (show)

See Also:
NLnet milestone: NLNet.2019.10.Wishbone
total budget (EUR) for completion of task and all subtasks: 100
budget (EUR) for this task, excluding subtasks' budget: 100
parent task for budget allocation: 384
child tasks for budget allocation:
The table of payments (in EUR) for this task; TOML format:
"cole"={amount=100,submitted=2020-09-20,paid=2020-10-01}


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Luke Kenneth Casson Leighton 2020-08-24 22:27:50 BST
https://libre-soc.org/3d_gpu/mesa/?updated
Comment 1 Cole Poirier 2020-08-27 21:24:22 BST
I have completed the scripts and the documentation on the wiki page for building mesa, however, as you will see in both my documentation/instructions on the wiki page as well as in my scripts I have left out `-Dvulkan-drivers=libre-soc` in the meson configuration step as using is currently fails with the error: "invalid option, valid options are: auto, amd... etc. etc." I assume the instructions specified on the wiki page  under the heading "3 .From here on is solely Vivek's instructions" are part of the way to remedy this, however, I need from you Vivek the exact way you have setup vulkan, in order to complete both the scripts and the tutorial/documentation page.

Links for quick access:

Documentation wiki page: https://libre-soc.org/3d_gpu/mesa/
Script install-mesa-apt-reqs: https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=install-mesa-apt-reqs;h=169ca51a493c9e325668f0d92f27f87837f9fd92;hb=HEAD
Script lsoc-mesa-dev-env-setup: https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=lsoc-mesa-dev-env-setup;h=1b75db4ee03348b0fe0378f826e6f99e318cef5d;hb=HEAD
Comment 2 Luke Kenneth Casson Leighton 2020-08-27 22:03:32 BST
(In reply to Cole Poirier from comment #1)
> I have completed the scripts and the documentation on the wiki page for
> building mesa, however, as you will see in both my
> documentation/instructions on the wiki page as well as in my scripts I have
> left out `-Dvulkan-drivers=libre-soc`

this is almost certainly because of a wrong driver / library / include path.

this will need to be resolved.
Comment 3 Cole Poirier 2020-08-27 23:55:52 BST
(In reply to Luke Kenneth Casson Leighton from comment #2)
> (In reply to Cole Poirier from comment #1)
> > I have completed the scripts and the documentation on the wiki page for
> > building mesa, however, as you will see in both my
> > documentation/instructions on the wiki page as well as in my scripts I have
> > left out `-Dvulkan-drivers=libre-soc`
> 
> this is almost certainly because of a wrong driver / library / include path.
> 
> this will need to be resolved.

Yes, Vivek will have to provide the full information on his installation and configuration of vulkan and mesa, as right now he has only provided his mesa configuration details, and some vulkan environment variables, what is missing is his vulkan configuration and installation details. After he responds with these necessary details I will be able to add them to the scripts and update the documentation on the wiki page.
Comment 4 Luke Kenneth Casson Leighton 2020-08-28 01:13:23 BST
or, you could go through the build logs which of course you have saved with either script or nohup and from the install locations work out for yourself why you have installed the library in a place that you have not correctly told meson to look.
Comment 5 Cole Poirier 2020-08-28 03:10:23 BST
(In reply to Luke Kenneth Casson Leighton from comment #4)
> or, you could go through the build logs which of course you have saved with
> either script or nohup and from the install locations work out for yourself
> why you have installed the library in a place that you have not correctly
> told meson to look.

Luke, that's *incredibly* unhelpful. To be explicit, it feels like you're not providing help, simply calling me stupid.

The build logs from what exactly? From mesa? I'm trying to configure mesa, it configures and installs properly without the option `-Dvulkan-drivers=libre-soc`.

Where would these libre-soc vulkan drivers be generated? Doesn't vulkan itself need to be installed and configured for this to work? In that case, as I stated above, I need vivek to tell me how he configured this.

Using the above vulkan-drivers option whether with libre-soc, or with the path that I've installed the drm drivers to `-Dvulkan-drivers=~/opt/lsoc-mesa` fails with the *exact* same error

```../meson.build:21:0: ERROR: Options "~/opt/lsoc_mesa" are not in allowed choices: "auto, amd, freedreno, intel, swrast"```

Again, Vivek needs to provide the his full process, it's a waste of my time trying to guess for hours on end how this should be done, especially while the dcache, icache, and mmu are not complete.

Is this task, which is in its earliest stages and has a *year* if not more of leeway, *really* a higher priority than those *HDL* tasks which have mere weeks to be finished *and* tested? Do we not want to be able to run linux on our Oct 31 2020 tapeout chip?
Comment 6 Luke Kenneth Casson Leighton 2020-08-28 08:15:02 BST
(In reply to Cole Poirier from comment #5)
> (In reply to Luke Kenneth Casson Leighton from comment #4)

> 
> Luke, that's *incredibly* unhelpful. To be explicit, it feels like you're
> not providing help, simply calling me stupid.

sorry.  it was really late night.  

> The build logs from what exactly? From mesa? I'm trying to configure mesa,
> it configures and installs properly without the option
> `-Dvulkan-drivers=libre-soc`.

whilst i do not know the build process in detail, this is asking to not build the libresoc vulkan driver.

> Where would these libre-soc vulkan drivers be generated?

an accurate answer to that is: in the build tree, followed by installation in the directory specified at configure time.

if this is not specified it defaults to /usr/local usually.

remember though what i said, don't install in /usr/local (not on the main system that is) specify --prefix=/opt/local or ~/libresocmesa


> Doesn't vulkan
> itself need to be installed and configured for this to work?

vulkan is an API. you don't "install a vulkan", you configure mesa to generate a library that is compatible *with* the vulkan API.

> In that case,
> as I stated above, I need vivek to tell me how he configured this.
> 
> Using the above vulkan-drivers option whether with libre-soc, or with the
> path that I've installed the drm drivers to
> `-Dvulkan-drivers=~/opt/lsoc-mesa` fails with the *exact* same error
> 
> ```../meson.build:21:0: ERROR: Options "~/opt/lsoc_mesa" are not in allowed
> choices: "auto, amd, freedreno, intel, swrast"```

*click*.  i know what it is.  did you use the master branch or the libresocdev branch?  (not exact name, you need to look it up, git branch -R i think)

find and check out the right branch and libresoc will be in the list.

> Is this task, which is in its earliest stages and has a *year* if not more
> of leeway, *really* a higher priority than those *HDL* tasks which have mere
> weeks to be finished *and* tested? Do we not want to be able to run linux on
> our Oct 31 2020 tapeout chip?

very good point.
Comment 7 vivekvpandya 2020-08-28 09:04:49 BST
Sorry Cole and Luke,
First of all thanks Cole for putting things on the doc.

I think all steps you have mentioned is correct except take source from https://gitlab.freedesktop.org/vivekvpandya/mesa/-/tree/libresoc_dev

I also apologies to Luke, I will soon request you git access with RSA key. Then we can have source on libre-soc git server and probably avoid problems in future.
Comment 8 vivekvpandya 2020-08-28 09:07:39 BST
Basically we are not at point where we can push things on upstream mesa repo. So taking source from git clone https://gitlab.freedesktop.org/mesa/mesa.git will not have libre-soc related skeleton of the driver.
Comment 9 Luke Kenneth Casson Leighton 2020-08-28 10:33:01 BST
https://gitlab.freedesktop.org/vivekvpandya/mesa/-/tree/libresoc_dev

hilarious, pretty close.

do "git pull origin libresoc_dev" followed by git checkout -b libresoc_dev
Comment 10 Cole Poirier 2020-08-28 19:38:57 BST
(In reply to Luke Kenneth Casson Leighton from comment #6)
> (In reply to Cole Poirier from comment #5)
> > (In reply to Luke Kenneth Casson Leighton from comment #4)
> 
> > 
> > Luke, that's *incredibly* unhelpful. To be explicit, it feels like you're
> > not providing help, simply calling me stupid.
> 
> sorry.  it was really late night.  

Thank you I accept your apology (explicitly accepting apologies is something I learned from friend). I understand, over six months of getting to know you, I've come to understand that you only lose your very helpful even chipper demeanor, when you're days, if not weeks past the point of initial exhaustion :)

> > The build logs from what exactly? From mesa? I'm trying to configure mesa,
> > it configures and installs properly without the option
> > `-Dvulkan-drivers=libre-soc`.
> 
> whilst i do not know the build process in detail, this is asking to not
> build the libresoc vulkan driver.
> 
> > Where would these libre-soc vulkan drivers be generated?
> 
> an accurate answer to that is: in the build tree, followed by installation
> in the directory specified at configure time.
> 
> if this is not specified it defaults to /usr/local usually.
> 
> remember though what i said, don't install in /usr/local (not on the main
> system that is) specify --prefix=/opt/local or ~/libresocmesa

Yes, I've been very careful to follow your advise on this, also noted the reason that it's important to follow this warning on the documentation page.

> > Doesn't vulkan
> > itself need to be installed and configured for this to work?
> 
> vulkan is an API. you don't "install a vulkan", you configure mesa to
> generate a library that is compatible *with* the vulkan API.

Ah, makes more sense.

> > In that case,
> > as I stated above, I need vivek to tell me how he configured this.
> > 
> > Using the above vulkan-drivers option whether with libre-soc, or with the
> > path that I've installed the drm drivers to
> > `-Dvulkan-drivers=~/opt/lsoc-mesa` fails with the *exact* same error
> > 
> > ```../meson.build:21:0: ERROR: Options "~/opt/lsoc_mesa" are not in allowed
> > choices: "auto, amd, freedreno, intel, swrast"```
> 
> *click*.  i know what it is.  did you use the master branch or the
> libresocdev branch?  (not exact name, you need to look it up, git branch -R
> i think)
> 
> find and check out the right branch and libresoc will be in the list.

Click!! Wrong branch error ;)

Should be easy to fix now :)

> > Is this task, which is in its earliest stages and has a *year* if not more
> > of leeway, *really* a higher priority than those *HDL* tasks which have mere
> > weeks to be finished *and* tested? Do we not want to be able to run linux on
> > our Oct 31 2020 tapeout chip?
> 
> very good point.

Thank you, I just want to make sure we are spending our limited resources and time optimally.
Comment 11 Cole Poirier 2020-08-28 19:45:37 BST
(In reply to vivekvpandya from comment #7)
> Sorry Cole and Luke,
> First of all thanks Cole for putting things on the doc.

No problem, happy to as it makes things easier, and is an NLNET requirment.

> I think all steps you have mentioned is correct except take source from
> https://gitlab.freedesktop.org/vivekvpandya/mesa/-/tree/libresoc_dev

Right!

(In reply to vivekvpandya from comment #8)
> Basically we are not at point where we can push things on upstream mesa
> repo. So taking source from git clone
> https://gitlab.freedesktop.org/mesa/mesa.git will not have libre-soc related
> skeleton of the driver.

Ah, thank you.

(In reply to Luke Kenneth Casson Leighton from comment #9)
> https://gitlab.freedesktop.org/vivekvpandya/mesa/-/tree/libresoc_dev
> 
> hilarious, pretty close.
> 
> do "git pull origin libresoc_dev" followed by git checkout -b libresoc_dev

Ok, awesome :)
Comment 12 Cole Poirier 2020-09-09 16:52:52 BST
Hi Vivek,

Just a quick note, your instructions worked very well, thank you, but a small modification is necessary in order to get it to compile. It appears that the vk_format* files are missing from your src/libre-soc/vulkan folder, can you run `cp src/amd/vulkan/vk_format* src/libre-soc/vulkan` on your repo to remedy this? Thank you.
Comment 13 Luke Kenneth Casson Leighton 2020-09-09 18:07:52 BST
(In reply to Cole Poirier from comment #12)
> Hi Vivek,
> 
> Just a quick note, your instructions worked very well, thank you, but a
> small modification is necessary in order to get it to compile. It appears
> that the vk_format* files are missing from your src/libre-soc/vulkan folder,
> can you run `cp src/amd/vulkan/vk_format* src/libre-soc/vulkan` on your repo
> to remedy this? Thank you.

or, is there a command that generated those files? (we have a development
rule: "output from a command" is not to be added to a repository)

if not, then a quick "git clone" in a separate working directory
(followed by a build) should show you what files are missing,
as should "git status".
Comment 14 Cole Poirier 2020-09-09 19:28:09 BST
(In reply to Luke Kenneth Casson Leighton from comment #13)
> (In reply to Cole Poirier from comment #12)
> > Hi Vivek,
> > 
> > Just a quick note, your instructions worked very well, thank you, but a
> > small modification is necessary in order to get it to compile. It appears
> > that the vk_format* files are missing from your src/libre-soc/vulkan folder,
> > can you run `cp src/amd/vulkan/vk_format* src/libre-soc/vulkan` on your repo
> > to remedy this? Thank you.
> 
> or, is there a command that generated those files? (we have a development
> rule: "output from a command" is not to be added to a repository)
> 
> if not, then a quick "git clone" in a separate working directory
> (followed by a build) should show you what files are missing,
> as should "git status".

Not an obvious or documented command, your above suggestion was the exact process I took to figure this out. Git clone. Try to configure the build. Meson complains that vk_format_x is missing. cd to main mesa folder out of build folder. `find . -name vk_format_x`. Result: `src/freedreno/vulkan/vk_format_x, src/amd/vulkan/vk_format_x`. `cp src/amd/vulkan/vk_format_x` because I recognize amd but not freedreno. Repeat an additional three times with meson complaining about a different missing file with the name format vk_format* each time. Didn't repeat again as then it configured and `ninja install`'d successfully.

There probably is something else going on there but investigating that is far beyond the scope of my task of documenting how to build vivek's mesa branch from vivek's *provided* instructions, so the task of figuring out what these files mean and how they are generated is *squarely* in the scope of vivek's task.
Comment 15 Luke Kenneth Casson Leighton 2020-09-09 19:37:58 BST
(In reply to Cole Poirier from comment #14)

> missing file with the name format vk_format* each time. Didn't repeat again
> as then it configured and `ninja install`'d successfully.

yep sounds like it's just missing files.  "git status" by vivek will help identify the full list.
Comment 16 Jacob Lifshay 2020-09-09 19:50:41 BST
(In reply to Cole Poirier from comment #14)
> because I recognize amd but not freedreno.

freedreno is the driver for Qualcomm's Adreno GPUs, used in a lot of smartphones.
Comment 17 Cole Poirier 2020-09-09 19:51:28 BST
(In reply to Jacob Lifshay from comment #16)
> (In reply to Cole Poirier from comment #14)
> > because I recognize amd but not freedreno.
> 
> freedreno is the driver for Qualcomm's Adreno GPUs, used in a lot of
> smartphones.

Oh cool! Thanks for the info Jacob :)
Comment 18 Cole Poirier 2020-09-14 20:49:50 BST
Page exists and correctly documents the setup process up to the point vivek has reached. Closing, a new bug should be opened for additions to the documentation and scripts as vivek progresses.
Comment 19 Cole Poirier 2020-09-15 20:04:03 BST
Update budget allocations and parent task for budget allocations per http://lists.libre-soc.org/pipermail/libre-soc-dev/2020-September/000536.html
Comment 20 Luke Kenneth Casson Leighton 2020-09-15 20:39:50 BST
excellent, did you also remember to subtract EUR 100 from 384 "child" budget?
(same for other bug)
Comment 21 Cole Poirier 2020-09-15 23:36:49 BST
(In reply to Luke Kenneth Casson Leighton from comment #20)
> excellent, did you also remember to subtract EUR 100 from 384 "child" budget?
> (same for other bug)

I have now,  thanks, I didn't understand properly how this works, I thought it would automatically deduct from the parent bug. See forthcoming mailing list email about problem related to this but not relevant to this bug.