test: add end-to-end backend regression suite with real ASL fixtures and CI integration#22
Open
aviralsaxena16 wants to merge 3 commits intoOSIPI:mainfrom
Open
test: add end-to-end backend regression suite with real ASL fixtures and CI integration#22aviralsaxena16 wants to merge 3 commits intoOSIPI:mainfrom
aviralsaxena16 wants to merge 3 commits intoOSIPI:mainfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hello @1brahimmohamed @jan-petr,
This PR introduces a structured end-to-end regression testing framework for the ASL Method Section Generator backend.
The objective is to improve reliability and establish a deterministic validation pipeline for API behavior and real BIDS-based ASL inputs, in alignment with the GSoC project goals.
All tests pass locally and are validated under CI across multiple Python versions. Snapshot baselines are version-controlled to ensure deterministic behavior across environments.
What This PR Adds
1. End-to-End API Regression Testing
Regression tests are implemented at the FastAPI layer using
TestClientto validate:/api/report/process/bidsbehavior/api/report/process/dicombehavior/api/report/report-pdfgenerationpytest-regressionsis used to snapshot JSON responses, ensuring that future changes do not silently alter report generation logic.Volatile fields (e.g.,
nifti_slice_number) are excluded to prevent false-positive regressions.2. Real ASL Fixture-Based Testing
Added real ASL fixtures under:
tests/fixtures/real_sample/
Includes:
sub-Sub1_asl.jsonsub-Sub1_m0scan.jsonsub-Sub1_aslcontext.tsvTests simulate real BIDS uploads including:
No internal functions are mocked , the full request lifecycle is exercised:
upload → normalization → metadata extraction → report generation → serialization
This validates:
This approach ensures scientific reproducibility by protecting report text and metadata transformations from unintended drift.
3. Deterministic Sample NIfTI
A minimal valid NIfTI file is included:
app/utils/sample_nifti.nii.gz
This allows CI to exercise nibabel-based slice detection without relying on external datasets while maintaining reproducibility.
4. API Schema & Error Handling Tests
Added coverage for:
5. PDF Endpoint Validation
Verified:
application/pdfcontent-typeMinor Fixes Included
modaliy_enum.py→modality_enum.pyImpact
This PR establishes a reproducible testing baseline that enables:
New fixtures can now be added per module with minimal setup.
Test Summary
(closes #7)