diff --git a/README.md b/README.md index 96b346a..7e28be1 100644 --- a/README.md +++ b/README.md @@ -26,6 +26,7 @@ We used [MIT License](LICENSE) because it is simple, permissive and encourages d - Team Members/Developers: Chikith Rishi Maddi, Vishakha Mathur, Samuel Keun ## Contact +- Ming-Dao Chia (Ming-Dao.Chia@anu.edu.au) ## Installation and Setup @@ -63,6 +64,15 @@ The system requires two primary inputs: - Metadata CSV file - Raw hyperspectral .sig measurement files +### Input formatting + +- Example raw .sig file data can be found in [sig_files](data/example_data/example1/sig_files) + +- Example metadata csv file can be found in [example1](data/example_data/example1/metadata.csv) + +- Example Location file can also be found in [example1](data/example_data/example1/positions.csv) + + Recommended folder structure: ```text @@ -157,17 +167,104 @@ Using absolute / full file paths (supported irrespective of file location): ``` -#### Metadata File Path +## Sample and Test Data + +Example data is available in [`data/example_data`](data/example_data). It contains: + +* metadata CSV files; +* SVC `.sig` hyperspectral measurements; and +* location files used to arrange measurements in the heatmap. + +Run HyperKey with the example dataset: + +```bash +python hyperkey.py data/example_data/example1/metadata.csv \ + -r data/example_data/example1 \ + --outlier-analysis +``` + +Run the automated test suite after system changes: + +```bash +pytest --cov=scripts --cov-report=term-missing +``` + +These tests verify file matching, path handling, report generation and other expected pipeline behaviour. + +## Validation and Quality Assurance + +HyperKey currently uses automated tests and manual output inspection for functional validation. + +The following checks are performed: + +* metadata rows are matched with the expected `.sig` files; +* missing, blank and invalid file numbers are reported; +* output files are created in both default and custom locations; +* NDVI uses the spectral bands nearest to the required red and near-infrared wavelengths; +* Markdown, HTML and PDF reports contain consistent processing results; +* outlier listings are checked against the generated outlier CSV; and +* CI runs the test suite whenever changes are pushed. + + +## Technical Documentation + +HyperKey uses a modular Python workflow: + +| Component | Responsibility | +| -------------------------- | ----------------------------------------------------------------------- | +| `hyperkey.py` | User-facing entry point | +| `workflow.py` | Coordinates all processing stages | +| `pipeline.py` | Validates metadata, finds `.sig` files and creates merged spectral data | +| `visualise_heatmap.py` | Calculates the selected vegetation index and generates a heatmap | +| `visualise_measurement.py` | Generates spectral-reflectance graphs | +| `outlier_analysis.py` | Detects and exports unusual spectral measurements | +| `report.py` | Generates Markdown, HTML and PDF reports | + +The workflow follows these stages: + +1. Validate the supplied paths and metadata. +2. Match metadata rows with `.sig` measurements. +3. Create the merged spectral CSV and processing summary. +4. Generate the heatmap and spectral graph. +5. Run optional outlier analysis. +6. Generate Markdown, HTML and PDF reports. + +Output paths are resolved centrally and passed to each module. This allows the same workflow to support both the default output directory and a custom path supplied with `-o`. + +## Version History + +### Unreleased + +* Added custom output paths. +* Added optional outlier analysis. +* Added outlier listings to Markdown, HTML and PDF reports. +* Added print-friendly HTML report styling. +* Replaced Playwright and Chromium with fpdf2 for PDF generation. +* Improved unique output filenames to prevent overwriting. + + +### Initial Prototype + +* Added metadata and `.sig` file matching. +* Added merged spectral CSV generation. +* Added NDVI heatmaps and spectral-reflectance graphs. +* Added basic Markdown, HTML and PDF reports. + +For detailed development history, see the repository’s [commit history](https://github.com/appn-anu/hyperkey/commits/main) and [releases](https://github.com/appn-anu/hyperkey/releases). + +## References -## Input formatting +1. Rouse, J. W., Haas, R. H., Schell, J. A., and Deering, D. W. (1974). *Monitoring Vegetation Systems in the Great Plains with ERTS*. Third Earth Resources Technology Satellite Symposium, NASA SP-351. -Example raw .sig file data can be found in [sig_files](data/example_data/example1/sig_files) +2. Montero, D., Aybar, C., Mahecha, M. D., Martinuzzi, F., Söchting, M., and Wieneke, S. (2023). spyndex: A Python package for computing spectral indices. *SoftwareX, 23*, 101341. https://doi.org/10.1016/j.softx.2023.101341 -Example metadata csv file can be found in [example1](data/example_data/example1/metadata.csv) +3. spyndex documentation: https://spyndex.readthedocs.io/ -Example Location file can also be found in [example1](data/example_data/example1/positions.csv) +4. SVC HR-series spectroradiometer documentation: [add the exact manual or manufacturer URL used by the project]. +5. fpdf2 documentation: https://py-pdf.github.io/fpdf2/ +6. Outlier-detection method: [add the paper or statistical source corresponding to the method implemented in `outlier_analysis.py`].