Bug 113 - FCVT unit tests have too great a dynamic range
Summary: FCVT unit tests have too great a dynamic range
Alias: None
Product: Libre-SOC's first SoC
Classification: Unclassified
Component: ALU (including IEEE754 16/32/64-bit FPU) (show other bugs)
Version: unspecified
Hardware: PC Linux
: --- enhancement
Assignee: Luke Kenneth Casson Leighton
Depends on:
Blocks: 48
  Show dependency treegraph
Reported: 2019-07-21 10:31 BST by Luke Kenneth Casson Leighton
Modified: 2020-12-04 20:36 GMT (History)
1 user (show)

See Also:
NLnet milestone: NLnet.2019.02.012
total budget (EUR) for completion of task and all subtasks: 250
budget (EUR) for this task, excluding subtasks' budget: 250
parent task for budget allocation: 48
child tasks for budget allocation:
The table of payments (in EUR) for this task; TOML format:
lkcl={amount=250, paid=2020-03-12}


Note You need to log in before you can comment on or make changes to this bug.
Description Luke Kenneth Casson Leighton 2019-07-21 10:31:20 BST
the range on the higher-sized to lower-sized randomised unit tests
give input values that are so large, they're almost exclusively
converted to +INF (or hit the -ve limit - 0 for unsigned, -INF for signed)

better tests are needed.
Comment 1 Luke Kenneth Casson Leighton 2019-08-01 01:00:34 BST
this for:

* fcvt float2int (large FP to smaller int)
* int2float (for FP16/32/64 to FP16)
* fcvt FP to FP (large FP to smaller FP)

int input is fairly easy to deal with: test within "close" range.
FP however needs *FP* numbers to be generated that will (or will
almost) fit into the target int range.
Comment 2 Luke Kenneth Casson Leighton 2020-02-10 15:10:56 GMT

added appx-range within chosen int (-INTMAX*2 to +INTMAX*2) now need one
that is very close to the limits.
Comment 3 Luke Kenneth Casson Leighton 2020-02-11 22:40:39 GMT
also added floating-point conversion tests which convert numbers just at
the limits of accuracy (or INF) of the target number.