Bug 91 - Design and implement texturing opcodes for 3D graphics
Summary: Design and implement texturing opcodes for 3D graphics
Status: CONFIRMED
Alias: None
Product: Libre-SOC's first SoC
Classification: Unclassified
Component: Source Code (show other bugs)
Version: unspecified
Hardware: Other Linux
: --- enhancement
Assignee: Jacob Lifshay
URL:
Depends on:
Blocks:
 
Reported: 2019-06-05 01:46 BST by Luke Kenneth Casson Leighton
Modified: 2020-02-17 17:48 GMT (History)
1 user (show)

See Also:
NLnet milestone: NLnet.2019.02
total budget (EUR) for completion of task and all subtasks: 0
budget (EUR) for completion of task (excludes budget allocated to subtasks): 0
parent task for budget allocation:
child tasks for budget allocation:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Luke Kenneth Casson Leighton 2019-06-05 01:46:04 BST
Texture maps need to be addressed by FP numbers and interpolated.

The maps are big and regularly arranged, and the loading on memory best covered by a separate instruction path.

Also with interpolation being involved they are best done as special instructions which perform the interpolation without needing extra CPU load.
Comment 1 Luke Kenneth Casson Leighton 2019-06-05 01:51:34 BST
http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2019-June/001657.html

Background. Earlier in thread discusses how LD/ST memory AGEN misses are affected by the regularity of the Texture maps.

http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2019-June/001629.html

(Edit 12aug2019) extra context:
http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2019-August/002469.html
Comment 2 Jacob Lifshay 2019-06-05 06:09:01 BST
This is one of the bigger tasks (several months of work) due to the sheer number of different texture formats that need to be implemented, so it will definitely need some budget, which I'll let Luke allocate.

Since I'm planning on implementing all the required formats in Rust as part of the Vulkan driver (since they need to work on x86 as well), we can split that out into a separate library to use for implementing the HW decoding as well. I think I could write it in such a way that we can generate both nmigen and LLVM IR from the same code. Additionally, the Rust code can be used as a Python library for testing purposes.

I think we should support YCbCr textures, since they are commonly used in video formats.

We will need to decide which compressed texture formats to support, Vulkan requires supporting at least one of these three:

* BC formats (also called S3TC and DXTn/DXTC)
    common on desktops
    effectively required for OpenGL
    oldest, I think the patents have expired in 2018
* ETC2 and EAC formats 
    required for OpenGL
    not patented
    http://www.phoronix.com/vr.php?view=MTE1ODU
* ASTC LDR formats
    common on mobile devices
    best compression ratios
    royalty free
    http://www.phoronix.com/vr.php?view=MTE1NDk

ASTC also has support for HDR formats.

If we can, I'd like to support all four.
Comment 3 Luke Kenneth Casson Leighton 2019-06-09 01:45:09 BST
(In reply to Jacob Lifshay from comment #2)

> This is one of the bigger tasks (several months of work) due to the sheer
> number of different texture formats that need to be implemented, so it will
> definitely need some budget, which I'll let Luke allocate.

 it sounds... like, to be honest, we need to have a serious think about
 that.
 
> Since I'm planning on implementing all the required formats in Rust as part
> of the Vulkan driver (since they need to work on x86 as well), we can split
> that out into a separate library to use for implementing the HW decoding as
> well. I think I could write it in such a way that we can generate both
> nmigen and LLVM IR from the same code.

 that would be really good.

> Additionally, the Rust code can be
> used as a Python library for testing purposes.

 as long as doing so does not interfere with formal proofs (clearly yosys
 cannot be made to critically depend on either python or a rust library).

 we have sfpy as a link to softfloat: "parallel mirroring" during simulations
 and checking simulated results against outside resources *is* the entire
 point of working in python, it's extremely convenient.

> I think we should support YCbCr textures, since they are commonly used in
> video formats.

 as we're doing a VPU as well, yes.

 yes on supporting as many texture formats as possible: we may however
 need to seriously prioritise.
Comment 4 Luke Kenneth Casson Leighton 2019-08-05 07:21:41 BST
Do we have a link to the relevant Vulkan API for textures? So we know what to interpolate?
Comment 5 Jacob Lifshay 2019-08-05 07:25:34 BST
(In reply to Luke Kenneth Casson Leighton from comment #4)
> Do we have a link to the relevant Vulkan API for textures? So we know what
> to interpolate?

it's somewhat spread throughout the vulkan spec, will find relevant links.
Comment 6 Luke Kenneth Casson Leighton 2019-08-12 11:47:56 BST
https://www.khronos.org/registry/DataFormat/specs/1.1/dataformat.1.1.pdf
Comment 7 Jacob Lifshay 2019-10-03 14:00:49 BST
one thing to watch out for:
apparently ASTC may not be open source: https://www.phoronix.com/scan.php?page=news_item&px=ASTC-Restrictive-License
Comment 8 Luke Kenneth Casson Leighton 2019-10-03 14:24:21 BST
Ok good find.
Comment 9 Jacob Lifshay 2019-10-16 06:20:49 BST
(In reply to Jacob Lifshay from comment #7)
> one thing to watch out for:
> apparently ASTC may not be open source:
> https://www.phoronix.com/scan.php?page=news_item&px=ASTC-Restrictive-License

will need to confirm openness, Khronos claims it's royalty-free:
https://www.khronos.org/news/press/khronos-releases-atsc-next-generation-texture-compression-specification
Comment 10 Jacob Lifshay 2020-02-17 17:48:59 GMT
(In reply to Jacob Lifshay from comment #9)
> (In reply to Jacob Lifshay from comment #7)
> > one thing to watch out for:
> > apparently ASTC may not be open source:
> > https://www.phoronix.com/scan.php?page=news_item&px=ASTC-Restrictive-License
> 
> will need to confirm openness, Khronos claims it's royalty-free:
> https://www.khronos.org/news/press/khronos-releases-atsc-next-generation-
> texture-compression-specification

ASTC effectively is now open: https://www.phoronix.com/scan.php?page=news_item&px=Arm-ASTC-Encoder-Apache-2.0