hmm, let's think this through:

* nvidia and other companies have billions of dollars
* LibreSOC applied for charitable grants of EUR 50,000
* the tasks that were applied for are low-level critical and unique research
* nvidia and other companies have a vested interest in seeing their hardware be promoted (in direct competition to LibreSOC)
* we are implementing low-level ISA primitives (not high-level single-purpose hard macros) and these need to go into libavutil (etc) regardless.
* this particularly because our focus is on general purpose Video *and Audio* including Libre (non-patented) CODECs not just a specific high-monetary-value list determined by the Khronos Group.

with our primary critical focus being to complete the R&D without which we do not have hardware or low-level libraries that we have committed to NLnet that we will complete, we have nothing to put behind the Vulkan Video API in ffmpeg, and yet nvidia and other billion dollar companies would benefit and profit, *even as* we divert funds away from our critical primary research.

this would likely result in said companies spongeing, yet again, off of libre software developers.  encouraging or empowering said companies to do that seems very unwise.

therefore what i propose is that nvidia and other members of the Khronos Group be approached and asked to provide direct grant funding for the proposed ffmpeg work.

NLnet may also be contacted and would likely be interested and happy to act as the financial conduit and manager of those funds, which has the advantage for said companies that it is a tax deductible charitable gift.

once our primary research is completed, and our obligations to NLnet are met, the situation can be re-evaluated.

one point: the vast majority of existing hw vendors already have their own APIs that already work with ffmpeg, so they are less likely to fund this. Otoh, I'd expect Google to probably make Vulkan Video a requirement for some future version of Android, so they might help fund it.

Also, if we do implement Vulkan Video in our Vulkan driver(s), it may save a lot of work for us since we'd just need to implement the SimpleV accelerated codecs once in the Vulkan driver(s) and not multiple times (due to there being several video libraries we'd want to support -- off the top of my head gstreamer, ffmpeg, libav, eventually Android). I'd expect the interface between ffmpeg and Vulkan Video to be a few thousand lines of code, so much smaller than a full codec.

https://bugs.libre-soc.org/show_bug.cgi?id=137

the list includes CODECs (including Audio CODECs) which Vulkan Video has
"not got round to", and includes low-level instructions and processing
that is, again, not part of Vulkan Video.

ffmpeg, gstreamer and others - which we committed to upstream - are
common.  Vulkan Video presently has zero adoption.

implementing Vulkan Video in ffpmeg right now provides us nothing, is
an additional task for which no budget has been allocated (because it
did not exist at the time of the Grant application) and takes up precious
funding and time.

the priority and reality is that we need more people to help with the
actual NLnet Video tasks that we have promised to deliver on than we
need to *add* to those tasks.

(In reply to Luke Kenneth Casson Leighton from comment #4)
> https://bugs.libre-soc.org/show_bug.cgi?id=137
> 
> the list includes CODECs (including Audio CODECs) which Vulkan Video has
> "not got round to", and includes low-level instructions and processing
> that is, again, not part of Vulkan Video.

For the video codecs, we can easily have custom Vulkan extensions for the codecs that aren't currently supported, it's pretty easy to do. We'd then switch to the standard extensions once they're ready, I'd expect that, if designed properly, the custom extensions would be nearly identical to the later standard extensions, making it really easy to convert the code base. If we submit them, Khronos might base the standard extensions on our custom extensions.

Think of it this way: would you rather have 10kloc per codec times each of several different libraries (ending up with *50kloc per codec*), or have 1-2kloc of interfacing code (that others may write for us due to it being an industry standard) in each of several different libraries and only 10kloc per codec in one library (the Vulkan driver). The Vulkan driver *can* use libx264 (and others) in its implementation, so we don't have to write the whole codec from scratch.

Using Vulkan Video can save us a *lot* of work...

(the numbers are approximations and may be off by a decent factor)

Jacob, I think the opposite approach would work better. That is, the vulkan driver should link to ffmpeg, and use the code developed that way. Thus $hypothetical_android_app that wants video decode accel using vulkan video can get it.

However I agree with Luke it's out of scope for the current grant, and with zero adoption, not an important point in any way.