Bug 1171 - Improve budget-sync to generate HTML version of task_db/report
Summary: Improve budget-sync to generate HTML version of task_db/report
Status: RESOLVED FIXED
Alias: None
Product: Libre-SOC's first SoC
Classification: Unclassified
Component: Documentation (show other bugs)
Version: unspecified
Hardware: PC Linux
: High enhancement
Assignee: 2022-08E
URL: https://libre-soc.org/task_db/
Depends on:
Blocks: 1211 1212 987
  Show dependency treegraph
 
Reported: 2023-09-19 23:18 BST by Andrey Miroshnikov
Modified: 2023-12-21 18:52 GMT (History)
4 users (show)

See Also:
NLnet milestone: NLnet.2021-08-071.cavatools
total budget (EUR) for completion of task and all subtasks: 0
budget (EUR) for this task, excluding subtasks' budget: 0
parent task for budget allocation:
child tasks for budget allocation:
The table of payments (in EUR) for this task; TOML format:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andrey Miroshnikov 2023-09-19 23:18:41 BST
Currently the auto-generated report showing grants and participants lacks detail and hyperlinks:
https://libre-soc.org/task_db/report/

The proposal is to generate an HTML version of this report, where every participant's bugs are linked (allowing to easily go to the bug tracker), similar to the way individual participant's pages already allow:
https://libre-soc.org/task_db/mdwn/andrey/

Such a report will show more information allowing easier navigation:
better budget breakdown based on tasks within a grant, and participants.

the new page MUST STILL BE COMPACT (individual participants is useful
but too verbose). the new page MUST NOT CAUSE A SCROLLBAR TO OCCUR
on mobile devices.

* DONE - Generate an ADDITIONAL report identical to current, in HTML form.
* DONE - add the single line description of each bug
         (truncated if necessary).
* DONE - Add links to bugs (instead of just bug number)
* DONE - add a hyperlink to each participant's page (relative URL)

too complex, not compact enough (needs more thought):

* Add the budgets assigned to each participant for every task
         and if possibly compact a copy of the TOML (payments).
         but it should be one line per payee.

This task is probably best done as a separate branch (since budget-sync is a critical program for Libre-SOC).
Comment 1 Luke Kenneth Casson Leighton 2023-09-20 10:34:50 BST
i have carried out an Audit of the Financial Records for the cavatools
grant 2021-08-071 and i am satisfied that the records are consistent and
accurate between NLnet's RFP budgetary amounts and our corresponding
bugtracker database.

https://lists.libre-soc.org/pipermail/libre-soc-dev/2023-September/005658.html

an independent review of the tables by a second person would help
Comment 2 Luke Kenneth Casson Leighton 2023-09-20 11:36:47 BST
an apparent discrepancy has been found in the cryptoprimitives grant Financial
Records, which *pre-dates* the NLnet "online" RFP system.

the amount is for EUR 1,300 which tallies with an *EMAILed* (manual)
RFP labelled "t-1" for the date 2022-12-13 of EUR 1300, paid to Luke Leighton,
when it was actually to Jacob Lifshay.

[fpga-fund] amount = 1202 submitted = 2022-06-19 paid = 2022-07-21
[jacob] amount = 98 submitted = 2022-06-19 paid = 2022-07-21

(fpga-fund *is* jacob, so the total he received was EUR 1300)

https://lists.libre-soc.org/pipermail/libre-soc-dev/2023-September/005664.html

i think i know what happened: the entry into the manual records was *on*
2022-12-13.

i am therefore happy with this, that there is nothing amiss.
i need a second person to confirm before i can lift the freeze.
Comment 3 Andrey Miroshnikov 2023-10-02 18:54:49 BST
Luke, re-assigning the task to you, updated budget from 987.
https://lists.libre-soc.org/pipermail/libre-soc-dev/2023-October/005715.html
Comment 4 Luke Kenneth Casson Leighton 2023-12-04 16:18:02 GMT
folks (2022-08E) i am fussy about the records being consistent
between bugs.libre-soc.org and the NLnet RFP system. can this
please be updated? see comment #2?

(In reply to Luke Kenneth Casson Leighton from comment #2)

> the amount is for EUR 1,300 which tallies with an *EMAILed* (manual)
> RFP labelled "t-1" for the date 2022-12-13 of EUR 1300, paid to Luke
> Leighton,
> when it was actually to Jacob Lifshay.
> 
> [fpga-fund] amount = 1202 submitted = 2022-06-19 paid = 2022-07-21
> [jacob] amount = 98 submitted = 2022-06-19 paid = 2022-07-21
> 
> (fpga-fund *is* jacob, so the total he received was EUR 1300)
Comment 5 Luke Kenneth Casson Leighton 2023-12-04 16:19:41 GMT
commit ad6878229e39eb4c3ac70ed21c647feead7ec523 (HEAD -> master)
Author: Luke Kenneth Casson Leighton <lkcl@lkcl.net>
Date:   Mon Dec 4 16:19:26 2023 +0000

    bug #1171: output record to temp file /tmp/report.mdwn temporarily
    
    https://bugs.libre-soc.org/show_bug.cgi?id=1171
    all output going temporarily to a report.mdwn, TODO to make it an
    option on the commandline, this is so that report.mdwn (summary)
    can be joined by a *second* more detailed report (or an option
    to create one)
Comment 6 Luke Kenneth Casson Leighton 2023-12-04 16:46:14 GMT
commit ef6afbf35ef65522e273e1056e87f3202cc87bd1 (HEAD -> master, origin/master, origin/HEAD)
Author: Luke Kenneth Casson Leighton <lkcl@lkcl.net>
Date:   Mon Dec 4 16:45:55 2023 +0000

    bug#1171: output report.mdwn to output_dir, default to ./report.mdwn"
Comment 7 Jacob Lifshay 2023-12-04 21:30:58 GMT
two issues I found while reviewing the changes:
https://git.libre-soc.org/?p=utils.git;a=blob;f=Makefile;h=84b9037151b0e9c386bc213a3c126ee3a7066c46;hb=ef6afbf35ef65522e273e1056e87f3202cc87bd1#l5
both the Makefile and budget-sync independently open report.mdwn, this is highly likely to cause issues where they overwrite each other's output. since budget-sync now writes to the file directly, I think we should stop redirecting the standard output and remove nohup from the makefile. any prints that we missed will have file=f added. errors will go to stdout or stderr so people are less likely to ignore them.

https://git.libre-soc.org/?p=utils.git;a=commitdiff;h=4e4a4694d2b07037dfcab8df0d3ea825e97e5ea6
"...%" is missing a format specifier: fixed in:
https://git.libre-soc.org/?p=utils.git;a=commitdiff;h=53cc6d4a5f4ed2d360b934bd40ab0ec3c7b03963
Comment 8 Luke Kenneth Casson Leighton 2023-12-05 03:21:58 GMT
(In reply to Jacob Lifshay from comment #7)

> both the Makefile and budget-sync independently open report.mdwn, this is
> highly likely to cause issues where they overwrite each other's output.

i'm not finished yet. i had to stop and get into town to charge all my
battery operated devices.
Comment 9 Luke Kenneth Casson Leighton 2023-12-21 17:51:49 GMT
(In reply to Luke Kenneth Casson Leighton from comment #8)

> i'm not finished yet. i had to stop and get into town to charge all my
> battery operated devices.

ok that's sorted. i added a new option "--detail" and passed in an
option to save the report to a default location based on the output
subdirectory. it has to drop down one directory sigh but works fine.

the output is a bit of a mess to read (by hand) as it uses <pre> </pre> now,
rather than markdown ''' ''' as previously it was, plus it contains
<a href>s.

lots more can be done here but it is a good start.