Bug 1100 - create pytest plugin that stores each unit test output in a file and just gives a small output to pytest
Summary: create pytest plugin that stores each unit test output in a file and just giv...
Status: RESOLVED FIXED
Alias: None
Product: Libre-SOC's first SoC
Classification: Unclassified
Component: Source Code (show other bugs)
Version: unspecified
Hardware: PC Linux
: --- enhancement
Assignee: Jacob Lifshay
URL:
Depends on:
Blocks: 775
  Show dependency treegraph
 
Reported: 2023-06-02 00:28 BST by Jacob Lifshay
Modified: 2023-11-14 01:36 GMT (History)
2 users (show)

See Also:
NLnet milestone: NLnet.2021.02A.052.CryptoRouter
total budget (EUR) for completion of task and all subtasks: 600
budget (EUR) for this task, excluding subtasks' budget: 600
parent task for budget allocation: 775
child tasks for budget allocation:
The table of payments (in EUR) for this task; TOML format:
[jacob] amount = 600 submitted = 2023-10-15 paid = 2023-11-10


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jacob Lifshay 2023-06-02 00:28:40 BST
https://libre-soc.org/irclog/%23libre-soc.2023-06-01.log.html#t2023-06-01T23:51:53

to reduce pytest's memory usage, output logs will be stored in files and only the first and last few lines will be given back to pytest.
Comment 1 Jacob Lifshay 2023-06-02 05:34:08 BST
I committed an initial implementation, though didn't try running it yet. I'll add tests next.

once luke sets the repo to public, the link below should work:
https://git.libre-soc.org/?p=pytest-output-to-files.git;a=tree;hb=69a0b05097dadd2533defce80fa04235a99cce34

commit 69a0b05097dadd2533defce80fa04235a99cce34
Author: Jacob Lifshay <programmerjake@gmail.com>
Date:   Thu Jun 1 21:28:49 2023 -0700

    initial implementation -- still need to test it

commit 781bb7d5fad0e5869988f5ecf44fb3a5dfe3b5c5
Author: Jacob Lifshay <programmerjake@gmail.com>
Date:   Thu Jun 1 16:48:53 2023 -0700

    initial commit
Comment 2 Jacob Lifshay 2023-06-03 04:46:28 BST
I got it to work, it took ~18GB ram to run 32-thread pytest without SILENCELOG on the commit right before luke disabled a bunch of tests. I didn't have time to test it but I expect it to take a lot less ram at 8 threads, since the main process was a few GB and each worker was a few hundred MB to ~1GB.
https://git.libre-soc.org/?p=openpower-isa.git;a=commit;h=6a2de7d216b308182b0367b6240724e2a62b5029

it wrote out **85GB** of logs!

pytest                       7.1.2
pytest-forked                1.4.0
pytest-output-to-files       0.1.0
pytest-xdist                 2.5.0

I ran (in openpower-isa):
pytest --shorten-output-dir=test-out -n auto -v &> log.txt

https://git.libre-soc.org/?p=pytest-output-to-files.git;a=shortlog;h=0ce93414b507273cb02799dd5010683650b49dee

commit 0ce93414b507273cb02799dd5010683650b49dee
Author: Jacob Lifshay <programmerjake@gmail.com>
Date:   Fri Jun 2 20:33:57 2023 -0700

    reduce line count

commit 46d2bb04ef995f5ef2289fbdf1313f1804b2a427
Author: Jacob Lifshay <programmerjake@gmail.com>
Date:   Fri Jun 2 20:11:27 2023 -0700

    suspend default capture plugin while capturing

commit c18e34643ed6dfbc06a431fb38e3337962ee01b8
Author: Jacob Lifshay <programmerjake@gmail.com>
Date:   Fri Jun 2 19:15:49 2023 -0700

    all tests pass!

commit f7b2341b25b46edb557eeb5d8caf85cc3c497d97
Author: Jacob Lifshay <programmerjake@gmail.com>
Date:   Fri Jun 2 19:15:03 2023 -0700

    update .gitignore
Comment 3 Jacob Lifshay 2023-06-10 04:23:19 BST
as mentioned:
https://libre-soc.org/irclog/%23libre-soc.2023-06-10.log.html#t2023-06-10T02:08:57

I'd like to change the required version of pytest to 7.3.1 and pytest-xdist to 3.3.1 since we have to install from pip anyway and 3.x is ancient and ieee754fpu has required at least 6.0 for over a year and because with the plugin I wrote we don't have the running out of memory issue anymore
(those are the versions CI is currently using)
https://git.libre-soc.org/?p=ieee754fpu.git;a=blob;f=pyproject.toml;h=267ae9af7d6f7a6cb20043c7f6289b3d53213f59;hb=658d12765b103405f7965a0d3c5e399446654686#l2

I'm planning on adding a similar pyproject.toml with the pytest config to openpower-isa.git to always enable the plugin
(it's only used by pytest, so users don't need it unless they want to run tests using pytest)
that >=6.0 version requirement was committed may 2022

I added the openpower-isa side of the relevant changes on the pytest7 branch:
https://git.libre-soc.org/?p=openpower-isa.git;a=shortlog;h=refs/heads/pytest7

running pytest with SILENCELOG unset with 8 threads uses 9.46GiB ram on my system on the pytest7 branch and generates 5.8G of output log files in test-out/

commit 376be188538728f7cbd32aad38ed324dbe18e3b1
Author: Jacob Lifshay <programmerjake@gmail.com>
Date:   Fri Jun 9 20:14:44 2023 -0700

    silence log by default just in fmv/fcvt and utf-8 tests
    
    this is overridable by setting SILENCELOG to any value, which is then used

commit a1fd555dd65757371c111c7f98ce69e49b65b5b4
Author: Jacob Lifshay <programmerjake@gmail.com>
Date:   Fri Jun 9 19:02:50 2023 -0700

    Revert "disable fmv / fcvt unit tests as there are such a vast number"
    
    new pytest-output-to-files plugin mitigates memory use
    
    This reverts commit 329985466b05b821c1f02277224f445051c77bb6.

commit 9ec99a1b36236685ac80c91e9156e42cdfd3918c
Author: Jacob Lifshay <programmerjake@gmail.com>
Date:   Fri Jun 9 19:02:26 2023 -0700

    Revert "disable fmv-fcvt tests entirely"
    
    new pytest-output-to-files plugin mitigates memory use
    
    This reverts commit beadcce36dc23ffc8d599b1ac4601da9500476a4.

commit 36f03ce09789d373de58ef771e92bbc9f314ed59
Author: Jacob Lifshay <programmerjake@gmail.com>
Date:   Fri Jun 9 19:00:37 2023 -0700

    Revert "far too much memory (58 GB) being used by these unit tests,"
    
    new pytest-output-to-files plugin mitigates memory use
    
    This reverts commit e31e5486822023bf99701e410d7b5e55df355048.

commit 29a26474264682cfc311ad93e537c6c08a181006
Author: Jacob Lifshay <programmerjake@gmail.com>
Date:   Fri Jun 9 18:29:48 2023 -0700

    use pytest-output-to-files plugin with pytest==7.3.1 and pytest-xdist==3.3.1
Comment 4 Jacob Lifshay 2023-06-13 03:28:12 BST
I merged the openpower-isa.git changes and updated dev-env-setup