diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..3a8b0e9 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,11 @@ +# Keep hashed scientific fixtures, evidence, and manuscript sources identical +# when a Windows checkout has core.autocrlf enabled. +validation_cases/**/*.md text eol=lf +validation_cases/**/*.json text eol=lf +validation_cases/**/*.csv text eol=lf +validation_cases/**/*.cif text eol=lf +src/diffractscout/benchmark_data/*.cif text eol=lf +src/diffractscout/benchmark_data/*.json text eol=lf +scripts/compare_reference_engines.py text eol=lf +paper/*.md text eol=lf +paper/*.bib text eol=lf diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index cff1922..0e8227e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -87,6 +87,15 @@ jobs: run: | python -c "import mp_api, pymatgen; from diffractscout.providers.materials_project import _matrix; assert _matrix([[1 if i == j else 0 for j in range(6)] for i in range(6)]) is not None" pytest -q tests/test_provider_helpers.py + - name: Compare independent diffraction and elasticity implementations + run: python scripts/compare_reference_engines.py --output ci_reference_engines + - name: Upload independent numerical comparison + if: always() + uses: actions/upload-artifact@v7 + with: + name: reference-engines-python-${{ matrix.python-version }} + path: ci_reference_engines + if-no-files-found: warn cross-platform-smoke: name: Smoke / ${{ matrix.os }} @@ -174,11 +183,15 @@ jobs: wheel-test/bin/diffractscout --version wheel-test/bin/diffractscout demo -o wheel_demo wheel-test/bin/diffractscout verify wheel_demo + wheel-test/bin/diffractscout inspect wheel_demo --json wheel-test/bin/diffractscout benchmark -o wheel_benchmark wheel_cif=$(wheel-test/bin/python -c "import pathlib,diffractscout; print(pathlib.Path(diffractscout.__file__).with_name('benchmark_data')/'fcc_al.cif')") test -f "$wheel_cif" - wheel-test/bin/diffractscout-quick-export "$wheel_cif" -o wheel_quick.xlsx + wheel-test/bin/diffractscout preset save -o wheel_preset.json --no-patterns + wheel-test/bin/diffractscout preset show wheel_preset.json --json + wheel-test/bin/diffractscout-quick-export "$wheel_cif" -o wheel_quick.xlsx --preset wheel_preset.json wheel-test/bin/diffractscout verify wheel_quick_bundle + wheel-test/bin/diffractscout inspect wheel_quick_bundle --json test -x wheel-test/bin/diffractscout-gui - name: Upload distributions uses: actions/upload-artifact@v7 diff --git a/CHANGELOG.md b/CHANGELOG.md index 9e1cb97..d3ac77c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,22 @@ All notable changes are recorded here. The project follows semantic versioning a ### Added +- Reusable CLI/desktop analysis presets and read-only inspection of verified + result bundles, with explicit command-line override semantics. Presets exclude + credentials, paths, overwrite authorization, and manual tensor overrides. +- Reproducible comparisons with pymatgen for synthetic diffraction peak + positions and cubic directional moduli, with numerical reports retained by CI. +- Revised JOSS manuscript, software-comparison references, and evidence-based + submission documentation. +- Separate evidence-file input for JOSS readiness checks, so final verification + records can reference a committed source without changing its Git identity. +- Excel result overview and worksheet navigation, semantic header colors, + consistent scientific number formats, frozen phase columns, expandable + detail columns, and phase-relative intensity bars without changing canonical + data-sheet headers or stored scientific values. +- Separate Excel preview, permanent workbook-copy, and result-folder actions, + with phase/reflection/diagnostic counts in completion feedback. + - Windows launchers: `启动DiffractScout.bat` (GUI) and `quick_export_diffractscout.bat` (drag-and-drop quick-export with Excel next to the first input). - Packaging stub `scripts/package_windows_portable.py` documenting a future PyInstaller portable layout (`--help` / `--print-recipe`; no freeze yet). - Optional dependencies `figures` (matplotlib) and `gui-dnd` (tkinterdnd2); `paper` remains as a matplotlib alias. @@ -22,6 +38,15 @@ All notable changes are recorded here. The project follows semantic versioning a ### Fixed +- Preserved saved changes in temporary Excel previews when closing the GUI. +- Rejected `.xlsx` quick-export targets combined with disabled Excel output + before writing, and reported the retained bundle workbook if publishing its + separate Excel copy fails. +- Prevented concurrent preset saves from replacing a newly created target + without explicit overwrite authorization. +- Reported CLI error diagnostics with their actual severity and exposed the + generated result files in completion summaries. + - Windows `quick_export_diffractscout.bat` now uses `scripts/diffractscout_entry.py` and the same checkout interpreter order as the GUI launcher (repository `.venv`, then `python`, then `py -3`), so drag-and-drop works after the README diff --git a/README.md b/README.md index 638dab3..b081973 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ **DiffractScout turns a chemical-system question or a folder of CIF files into a verifiable theoretical powder-diffraction reference bundle.** It preserves database identity, exact CIF hashes, structural diagnostics, radiation settings, optional elastic-tensor provenance, indexed reflections, warnings, and file checksums in one workflow. -[中文说明](README.zh-CN.md) · [API](docs/API.md) · [GUI guide](docs/GUI.md) · [Scientific contracts](docs/SCIENTIFIC_CONTRACTS.md) · [Validation](docs/VALIDATION.md) · [Release procedure](docs/RELEASE.md) · [JOSS readiness](docs/JOSS_READINESS.md) +[中文说明](README.zh-CN.md) · [CLI guide](docs/CLI.md) · [API](docs/API.md) · [GUI guide](docs/GUI.md) · [Scientific contracts](docs/SCIENTIFIC_CONTRACTS.md) · [Validation](docs/VALIDATION.md) · [Release procedure](docs/RELEASE.md) · [JOSS readiness](docs/JOSS_READINESS.md) ## Why this software exists @@ -61,6 +61,24 @@ DiffractScout reimplements the CIF2Peaks desktop workflow inside a provenance-fi Column-name mapping and intensity-channel aliases: [docs/SCHEMA_ALIASES.md](docs/SCHEMA_ALIASES.md). Engine semantics vs CIF2Peaks/pymatgen: [docs/ENGINE_PARITY.md](docs/ENGINE_PARITY.md). +Excel results include an overview with worksheet navigation, grouped table +headers, consistent scientific number formats, frozen phase columns, and +phase-relative intensity bars. The GUI can save/load analysis presets and +offers separate Excel preview, permanent-copy, and result-folder actions. +See [Working with Excel results](docs/EXCEL.md). + +The CLI can save and reuse the same parameter presets and inspect a completed +bundle after verifying its integrity: + +```bash +diffractscout preset save -o presets/energy-30.json --energy-keV 30 +diffractscout analyze path/to/cifs -o outputs/run --preset presets/energy-30.json +diffractscout inspect outputs/run --json +``` + +Explicit command-line options override preset values. See the [CLI guide](docs/CLI.md) +for batch exit statuses, radiation overrides, and parameter-file handling. + When a *d*-spacing filter is used, the inclusive Bragg intersection is stored separately from the requested angular window. `provenance.json` and phase metadata distinguish `requested_two_theta_range_deg`, @@ -197,6 +215,10 @@ PASS Set `SOURCE_DATE_EPOCH` when an archive requires reproducible generated timestamps. See [docs/ANALYTIC_BENCHMARKS.md](docs/ANALYTIC_BENCHMARKS.md). +An optional [independent-engine comparison](validation_cases/independent_engines/README.md) +checks the same synthetic CIFs and cubic directional moduli against pymatgen. +CI preserves the numerical reports and dependency versions. + ## Analyze local CIF files ```bash @@ -343,7 +365,19 @@ Those repositories remain independent and unchanged by this project. Source snap ## JOSS preparation status -The repository contains an OSI-approved license, installable package metadata, tests, analytic benchmarks, CI, user and API documentation, governance, examples, contribution and support routes, tag-driven deterministic release packaging, a monthly reproducibility audit, dependency-update automation, a JOSS-format manuscript, and a specific AI usage disclosure. The public-development clock began on 12 August 2026. Formal submission remains blocked by the six-month/distributed-history gate, a completed real multiphase research case, independent diffraction and elasticity validation, external engagement, and human-confirmed metadata. The immutable software archive DOI is required after successful JOSS review for the publication stage, not for the initial submission. First run `python scripts/check_release.py` to create the current-source release receipt, then use `python scripts/joss_readiness.py --stage release|submission|publication`; see [docs/JOSS_READINESS.md](docs/JOSS_READINESS.md) and [docs/JOSS_REVIEW_CHECKLIST.md](docs/JOSS_REVIEW_CHECKLIST.md). +The repository includes an MIT license, installable package, tests, analytic and +independent numerical comparisons, CI, documentation, contribution and support +routes, reproducible release packaging, and a JOSS-format manuscript with AI +disclosure. The author reports use in published research; representative +citations and the associated software uses will be added to the submission +record. Public development began on 12 August 2026, so the JOSS requirement for +more than six months of active public history remains outstanding. The project +also tracks its own scientific-validation and engagement criteria separately +from official JOSS requirements. Run `python scripts/check_release.py` for the +current-source software acceptance checks; see [JOSS readiness](docs/JOSS_READINESS.md) +and the [review checklist](docs/JOSS_REVIEW_CHECKLIST.md) for the remaining +submission evidence and author confirmations. The final archive DOI is a +post-review publication step. ## Contributing, support, and citation diff --git a/README.zh-CN.md b/README.zh-CN.md index 0831f99..86c3d7a 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -8,7 +8,7 @@ **DiffractScout 将合金/化学体系候选相检索、本地 CIF 检查、理论粉末衍射计算、可选晶面法向弹性分析和可验证结果导出连接为一个流程。** 每项结果均可追溯到数据库记录或本地文件、CIF 哈希、辐射条件、计算定义、软件版本和结构化诊断。 -[英文主页](README.md) · [API](docs/API.md) · [GUI 使用说明](docs/GUI.md) · [科研计算约定](docs/SCIENTIFIC_CONTRACTS.md) · [验证策略](docs/VALIDATION.md) · [发布流程](docs/RELEASE.md) · [JOSS 准备状态](docs/JOSS_READINESS.md) +[英文主页](README.md) · [CLI 使用说明](docs/CLI.md) · [API](docs/API.md) · [GUI 使用说明](docs/GUI.md) · [科研计算约定](docs/SCIENTIFIC_CONTRACTS.md) · [验证策略](docs/VALIDATION.md) · [发布流程](docs/RELEASE.md) · [JOSS 准备状态](docs/JOSS_READINESS.md) ## 两类工作流 @@ -52,6 +52,18 @@ DiffractScout 在可追溯结果包中重实现了 CIF2Peaks 的主要桌面工 列名与强度通道别名见 [docs/SCHEMA_ALIASES.md](docs/SCHEMA_ALIASES.md)；与 CIF2Peaks/pymatgen 引擎差异见 [docs/ENGINE_PARITY.md](docs/ENGINE_PARITY.md)。 +Excel 提供结果概览和工作表导航，按列的含义区分表头颜色，统一科学数值显示格式，冻结物相标识列，并用数据条辅助查看相内相对强度。界面支持保存、加载常用分析参数，以及独立的 Excel 预览、永久副本另存和结果文件夹入口。完整用法见 [Excel 结果使用说明](docs/EXCEL.md)。 + +CLI 同样可以保存、复用参数，并校验和查看已有结果包： + +```powershell +diffractscout preset save -o presets/energy-30.json --energy-keV 30 +diffractscout analyze path/to/cifs -o outputs/run --preset presets/energy-30.json +diffractscout inspect outputs/run --json +``` + +命令行显式参数优先于预设。批处理退出码、辐射条件覆盖和参数文件用法见 [CLI 使用说明](docs/CLI.md)。 + 启用 *d* 间距过滤时，程序会单独记录包含端点的 Bragg 求交结果。 `provenance.json` 和物相元数据区分用户请求的 `requested_two_theta_range_deg`、过滤后的 @@ -170,6 +182,8 @@ diffractscout benchmark -o outputs/analytic_benchmark 该命令对简单立方、BCC、FCC、NaCl 和立方晶体方向弹性执行 45 项闭式解检查，并输出固定输入、预期值、容差、软件版本、运行环境、报告和 SHA-256 清单。成功结果为 `45/45 passed`。详见 [docs/ANALYTIC_BENCHMARKS.md](docs/ANALYTIC_BENCHMARKS.md)。 +另有可复现的 [pymatgen 独立数值对照](validation_cases/independent_engines/README.md)，比较相同合成 CIF 的峰位和立方晶体方向模量，并在 CI 中保留数值报告和依赖版本。 + ## 本地 CIF 分析 ```powershell @@ -250,7 +264,7 @@ J_no_LP = I_no_LP / V_cell² 当前离线 pytest 套件覆盖成分解析、子体系枚举及组合数量上限、大小写 CIF 扫描、同名文件防覆盖、CIF 数据块和空间群解析、特殊位置占位转换、系统消光、解析结构因子、Bragg 几何、边界反射、刚度单位换算、弹性张量检查、侧车配对、资源限制、数据库失败语义、事务式输出、电子表格安全、确定性证据归档、严格清单校验和投稿准备检查。测试收集数由 pytest/CI 报告，不再复制到静态文档；另有稳定的 45 项解析科学基准。GitHub Actions 还配置了多 Python 版本、Windows/macOS、Linux 无头 GUI、wheel 安装、解析基准、发布制品、月度复现审计、依赖更新和 JOSS 论文构建。月度定时运行只记录某一公开提交的可复现状态；只有由真实缺陷、依赖更新、验证、文档改进或用户反馈形成的公开提交、Issue、Pull Request 或 Release 才构成开发活动证据。 -JOSS 正式投稿仍需要真实且分布式的六个月公开开发记录、真实材料用例、独立衍射与弹性验证、外部互动、人工确认的元数据，以及通过官方 Open Journals 构建并逐页检查的稿件。先执行 `python scripts/check_release.py` 生成与当前源码绑定的发布验收收据，再用 `python scripts/joss_readiness.py --stage submission --output build/joss-readiness` 生成投稿阻塞项报告。不可变软件归档 DOI 属于 JOSS 审稿完成后的 `publication` 阶段，不是初次投稿的前置条件；工作计划见 [docs/JOSS_6_MONTH_PLAN.md](docs/JOSS_6_MONTH_PLAN.md)。 +作者已确认软件用于其已发表研究，代表论文及具体使用情况将在投稿记录中补充。项目自 2026 年 8 月 12 日开始公开开发，目前尚未满足 JOSS 超过六个月的持续公开开发要求。项目自定的科学验证、社区参与要求与 JOSS 官方门槛分别列于 [投稿准备状态](docs/JOSS_READINESS.md)。先执行 `python scripts/check_release.py` 验收当前软件，再用 `python scripts/joss_readiness.py --stage submission --output build/joss-readiness` 检查待补的投稿证据和作者确认事项。最终软件归档 DOI 在审稿完成后的 `publication` 阶段补入；工作计划见 [docs/JOSS_6_MONTH_PLAN.md](docs/JOSS_6_MONTH_PLAN.md)。 ## 来源与许可 diff --git a/docs/ADOPTION_AND_IMPACT.md b/docs/ADOPTION_AND_IMPACT.md index 2d579d4..843c6d1 100644 --- a/docs/ADOPTION_AND_IMPACT.md +++ b/docs/ADOPTION_AND_IMPACT.md @@ -1,53 +1,103 @@ # Adoption and research-impact evidence -The JOSS manuscript may state only claims supported by traceable records. The -machine-readable index is `docs/evidence/impact_evidence.json`; this document -defines the evidence standard and the claim ledger used during final editing. +Assessment date: **26 September 2026**. The JOSS manuscript may make only +claims supported by traceable records. The machine-readable index is +`docs/evidence/impact_evidence.json`; this document defines what each record +can support and distinguishes established evidence from author-reported leads. + +## Current evidence status + +The submitting author reports that DiffractScout has been used in multiple +published papers. This report indicates actual research use and must not be +rewritten as “no research use.” The specific papers, DOI/URLs, software +version or commit, feature used, and research outcomes have not yet been +mapped into the evidence ledger. Until representative sources are identified +and checked, the manuscript should not claim a publication count or describe +the contribution of those papers. + +The public repository contains a 45-check analytic benchmark and a synthetic +FCC demonstration. These materials support reproducibility of selected +calculation, software, and provenance contracts. They do not show experimental +accuracy, external adoption, productivity gains, or research impact by +themselves. A new [pymatgen comparison case](../validation_cases/independent_engines/README.md) +checks diffraction positions and selection rules on four synthetic CIFs, plus +directional Young's moduli for a synthetic cubic stiffness matrix. The +immutable report is in public candidate commit +[`a09ee1b`](https://github.com/D-sudoasd/DiffractScout/blob/a09ee1b0e48de4bf07898bceda4bec46b70dab43/validation_cases/independent_engines/reference_engine_comparison.md); +the evidence ledger records its Markdown and JSON hashes. It passes the stated +software-comparison tolerances for those inputs and versions. It does not +establish a real-material property, experimental agreement, external adoption, +or research impact. ## Evidence categories -### Research use case +### Research use -A use case should identify a scientific question, exact software version, -input provenance and license, settings, outputs, how the output affected a -research decision, and the applicable limitations. A usage statement without -an inspectable workflow supports awareness, not scientific impact. +For a published use, record the paper's DOI or stable URL, exact software +version/commit, operation used, and the result or research decision that the +software informed. Cite the publication in the JOSS manuscript. If the +workflow is not yet public, describe it only to the extent the author can +document it and JOSS editors can inspect it; do not disclose restricted data. -### Independent validation +For the project's stricter public case record, include the scientific +question, lawful input source, frozen input hashes, full settings, result +bundle and manifest hashes, research decision, and limitations. A versioned +report belongs under `validation_cases/`. A general statement that someone +ran the software is not enough to substantiate a specific scientific claim. + +### Independent numerical comparison An independent comparison should use an analytic result, accepted standard, -or separately implemented software path. Acceptance criteria must be fixed -before the result is inspected. Record versions, conventions, units, absolute -and relative differences, and unresolved discrepancies. +or separately implemented reference. State the acceptance criteria before +examining the DiffractScout result. Preserve implementation and version, +conventions, units, frozen input/result hashes, absolute and relative +differences, and unresolved discrepancies. For diffraction, state radiation +and scan range. For elasticity, identify tensor source and coordinate frame, +Voigt convention, transformations, and tested directions. + +The two reports under `validation_cases/independent_engines/` are indexed with +the immutable commit `a09ee1b`, the Markdown report hash, and the JSON result hash. +They satisfy the local readiness script's diffraction and elasticity +comparison-record requirements for those synthetic inputs and conventions. +They are implementation cross-checks, not experimental validation or reports +from an external validator. The current JOSS checklist does not name +independent diffraction and elasticity reports as stand-alone reviewer gates. ### External engagement -Suitable records include an external issue, pull request, code review, -validation report, tutorial use, or documented user feedback. Attribution -requires consent. GitHub stars, repository views, and download counts may be -reported as reach metrics only; they do not establish scientific impact. +Useful evidence includes a public issue or discussion from a user, an external +installation or review report, a contribution, or a documented workflow with +another group. Record the artifact and version tested, commands, verification +result, concrete outcome, and limitations. Attribute a person only with +consent. Stars, views, download counts, self-authored roadmap issues, and +scheduled CI runs measure other things and do not demonstrate external +research impact. -### Archived release +### Public development and releases -The entry must link a Git tag and an immutable archive DOI or equivalent -persistent identifier. The archived source must match the submission commit. -For JOSS, this record is completed after successful review when the editor asks -for the final tagged release and software archive. It is a `publication`-stage -gate, not a reason to mislabel an ordinary pre-submission GitHub Release as an -immutable archive. +The ledger records meaningful public commits, issues, pull requests, and +releases with their dates and outcomes. The public repository date is +12 August 2026. Do not count commits predating that date, private changes, +unpublished local work, or automatic CI. Public PR #14 records proposed +iteration at its current public head; do not describe it as merged or released. +A tagged software release is not an immutable archive DOI. After successful +review, tag and archive the reviewed source and record its DOI for the +publication stage. ## Claim ledger -Complete this table only as evidence becomes available. +| Potential manuscript claim | Evidence now available | Status | +|---|---|---| +| The software has been used in published research | Submitting-author report, 26 September 2026; representative papers and exact use not yet indexed | **Reported; citation-level audit pending.** Do not state a paper count until verified. | +| The package reproduces defined analytic diffraction and elasticity cases | 45-check analytic suite, synthetic offline FCC workflow, and the pymatgen reference-engine comparison | Supportable for the named analytic/synthetic cases and recorded implementations only; not an experimental-accuracy or impact claim. | +| A real multiphase workflow informed a specific research decision | No hashed, versioned case report is indexed | Pending. | +| Indexed reflection geometry agrees with an independent reference | Pymatgen `XRDCalculator` comparison on four synthetic CIFs with recorded grouping and position tolerances | Verified for those fixtures and settings only; not for real materials or experiment. | +| Directional elasticity agrees with an independent tensor reference | Pymatgen compliance contraction on one synthetic cubic matrix and `[100]`, `[110]`, `[111]` directions | Verified for that matrix, basis, convention, and reference version only; not for experimental or database tensors. | +| Researchers outside the development environment installed or used the software | No external test or use record is indexed | Pending. | +| The software changed a research decision, dataset, presentation, preprint, or paper | Author reports published use; affected result and software contribution are not yet mapped to sources | Source verification pending. | -| Proposed manuscript claim | Required evidence | Public record | Status | -|---|---|---|---| -| The software supports a reproducible candidate-to-diffraction workflow | versioned real-material case and verified result bundle | pending | not yet claimable | -| Indexed reflection geometry agrees with an independent implementation | comparison report with predeclared tolerance | pending | not yet claimable | -| Directional elastic output agrees with an independent tensor calculation | tensor, frame, rotation, equations, numerical comparison | pending | not yet claimable | -| Researchers outside the author's development environment can install and use the software | external issue/PR or archived user test | pending | not yet claimable | -| The software has contributed to a research decision, dataset, presentation, preprint, or paper | versioned workflow plus cited output | pending | not yet claimable | - -Synthetic analytic benchmarks support correctness of declared contracts. They -do not support claims of experimental validity, adoption, productivity gain, -or research impact. +The evidence ledger intentionally retains empty research-use and +external-engagement arrays until their source records satisfy the documented +checks. The validation entries link to immutable source-commit URLs and hashed +reports. Do not substitute these numerical comparisons for research-use or +external-engagement evidence. diff --git a/docs/API.md b/docs/API.md index ca5e7eb..95ca739 100644 --- a/docs/API.md +++ b/docs/API.md @@ -73,8 +73,11 @@ and source mode follows its preset semantics (built-in sources clear both; `Custom` keeps its wavelength and clears `energy_keV`). Contradictory explicit radiation overrides still raise; normalization does not loosen conflict validation. -The existing `.xlsx` shortcut remains atomic and does not replace an existing -workbook unless `overwrite=True` is explicit. +The `.xlsx` shortcut remains atomic and does not replace an existing workbook +unless `overwrite=True` is explicit. Combining an `.xlsx` target with +`include_excel=False` raises before output is created; use a directory target +for a CSV-only bundle. If publishing the separate workbook fails after the +bundle is generated, the error identifies the retained bundle workbook. Bundle Summary retains `two_theta_range_deg` as the requested settings range, adds the explicit alias `requested_two_theta_range_deg`, and separately records diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 6cdd49b..57a6d58 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -36,6 +36,16 @@ validation.py — strict independent bundle verification The GUI contains no separate numerical implementation. It creates `AnalysisSettings` and `DiscoverySettings`, calls the pipeline API, and renders returned diagnostics. +`gui_settings.py` validates and persists a versioned, explicit allowlist of +analysis controls. Presets exclude credentials, paths, overwrite authorization, +and structure-specific elastic tensors. Excel presentation is separate from +row construction: `excel_styles.py` formats the existing canonical tables and +`excel_overview.py` builds a navigable summary from those exported records. +The CLI preset adapter uses that same validated parameter format and tracks +explicit command-line options before merging them. `inspection.py` verifies an +existing bundle and streams its tables to produce a compact report without +running calculations or changing the bundle. + ## Provider boundary The discovery layer depends on the `PhaseProvider` protocol. Materials Project is one optional implementation. A provider supplies: diff --git a/docs/CLI.md b/docs/CLI.md new file mode 100644 index 0000000..f018c95 --- /dev/null +++ b/docs/CLI.md @@ -0,0 +1,109 @@ +# Command-line workflows + +The command-line interface uses the same calculation and bundle-verification +code as the Python API and desktop application. Run `diffractscout --help` for +the command list or add `--help` to a command for its options. + +## Start with a complete offline run + +```bash +diffractscout demo -o outputs/demo +diffractscout inspect outputs/demo +``` + +`inspect` verifies the bundle before reporting its calculation settings, +phase and reflection counts, diagnostics, and result files. It reads an +existing result; it does not repeat the calculation or modify any files. +If integrity verification fails, it reports the failure rather than presenting +the contents as a valid calculation result. + +For a machine-readable inspection report: + +```bash +diffractscout inspect outputs/demo --json +``` + +Use `verify` when only the manifest and file-integrity result are needed. +Inspection can check that a saved bundle is unchanged; it does not establish +agreement with experimental data. + +## Reuse calculation parameters + +Create a parameter preset without starting an analysis: + +```bash +diffractscout preset save -o presets/energy-30.json --energy-keV 30 --two-theta-max 60 +diffractscout preset show presets/energy-30.json +diffractscout analyze path/to/cifs -o outputs/energy-30 --preset presets/energy-30.json +``` + +Presets use the same versioned format as desktop **Save parameters** and +**Load parameters**. They store analysis settings and export switches, including +radiation, angular and d-spacing ranges, profile parameters, resource limits, +Excel, elasticity, laboratory views, patterns, figures, and recursive scanning. +Recursive scanning applies to local inputs in `analyze` and `quick-export`; +the database-backed `run` command does not use this preset field. +API keys, input/output paths, overwrite authorization, manual elastic tensors, +and the CLI-only `--figure-preset` style name are excluded. + +The complete preset is validated before use. Values are resolved in this order: +command defaults, preset values, then explicitly supplied command-line options. +An explicit value overrides a preset even when it equals the command default. + +```bash +diffractscout analyze path/to/cifs -o outputs/energy-83 --preset presets/energy-30.json --energy-keV 83 +diffractscout preset save -o presets/energy-83.json --preset presets/energy-30.json --energy-keV 83 +``` + +An explicit energy, wavelength, or built-in source selects the corresponding +radiation mode. An energy in keV is never reinterpreted as a wavelength in +Angstrom. Use `--wavelength-A` to supply a custom wavelength. + +Both directions of an output switch are available, such as `--excel` and +`--no-excel`, so a disabled preset option can be enabled for one run. Use +`--clear-d-min` or `--clear-d-max` to remove a saved spacing bound. Existing +preset files are preserved unless `preset save --overwrite` is explicit. +Save presets outside result bundles; writing inside a bundle is refused to +preserve its manifest and recorded files, including when `--overwrite` is used. + +```bash +diffractscout preset show presets/energy-30.json --json +diffractscout analyze path/to/cifs -o outputs/no-d-filter --preset presets/energy-30.json --clear-d-min --clear-d-max --no-patterns +``` + +## Export directly to Excel + +```bash +diffractscout quick-export path/to/sample.cif -o outputs/sample.xlsx --preset presets/energy-30.json +diffractscout inspect outputs/sample_bundle +``` + +The standalone `diffractscout-quick-export` entry point supports the same preset +workflow. A direct `.xlsx` output creates a separate editable workbook and a +complete result bundle. An `.xlsx` target requires Excel output; choose a +directory target with `--no-excel` for a CSV-only bundle. + +See the [Excel guide](EXCEL.md) for workbook navigation, intensity formatting, +and keeping editable copies separate from the verified bundle. + +## Use results in batch scripts + +Analysis and quick-export commands preserve their `--json` result structure. +Their text summaries show output locations, phase and reflection counts, and +warnings and errors with their recorded severity. The new `inspect --json` +report is a separate interface for already completed bundles. + +| Exit status | Analysis and quick-export meaning | +|---|---| +| `0` | At least one phase succeeded and no error diagnostic was emitted. | +| `2` | Invalid arguments, failed operation, or no successful phase analysis. | +| `3` | A usable partial result was produced with one or more failed items. | + +Warnings alone do not turn a successful analysis into a failed command. Check +both the exit status and diagnostics when deciding whether a partial result +should enter the next stage of a batch workflow. + +Database discovery additionally needs the optional Materials Project +dependencies and credentials. Prefer `MP_API_KEY` over placing a credential in +a shell command. The [API guide](API.md) and [scientific contracts](SCIENTIFIC_CONTRACTS.md) +describe the calculation fields and tensor-coordinate requirements. diff --git a/docs/ENGINE_PARITY.md b/docs/ENGINE_PARITY.md index c8707d5..56303f8 100644 --- a/docs/ENGINE_PARITY.md +++ b/docs/ENGINE_PARITY.md @@ -83,6 +83,20 @@ Plane-normal elasticity always uses the three-index plane ## Validation guidance +The optional [independent-engine comparison](../validation_cases/independent_engines/README.md) +runs the packaged synthetic CIFs through both DiffractScout and pymatgen at the +same wavelength and cell setting. It compares coincident peak groups by +spacing, checks reflection selection rules, and compares cubic directional +moduli with pymatgen's tensor implementation. Versioned reports record input +hashes, tolerances, software versions, and normalized-intensity differences. +Intensity differences are reported separately from the peak-position and +elasticity acceptance checks because the two scattering-factor models differ. + +```bash +python -m pip install -e ".[mp]" +python scripts/compare_reference_engines.py --output outputs/reference_engines +``` + 1. **Contract tests**: \(J = I / V^2\), ranks consistent with channels, LP ratio \(I_{\mathrm{with\,LP}} / I_{\mathrm{no\,LP}}\). 2. **Analytic fixtures**: known monoatomic cells and expected \(\lvert F\rvert^2\). diff --git a/docs/EXCEL.md b/docs/EXCEL.md new file mode 100644 index 0000000..56c6e57 --- /dev/null +++ b/docs/EXCEL.md @@ -0,0 +1,79 @@ +# Working with Excel results + +DiffractScout exports a complete result bundle and an optional `results.xlsx`. +The workbook is a presentation of the calculated results: cell formatting does +not round the stored numbers or change the scientific definitions. The English +data-sheet headers still match their CSV counterparts. + +## Start with the result overview + +The overview brings together the number of analyzed phases and reflections, +diagnostic counts, calculation conditions, and links to the detailed worksheets. +Use the diagnostics link when a run contains warnings or failed inputs; a +partially successful run can still contain usable phase results. + +Laboratory views add Chinese labels and the `推荐峰表` and `使用说明` sheets. +With laboratory views disabled, the canonical English sheets remain available. + +![Result overview for the synthetic offline example](assets/excel-overview.png) + +## Find and compare reflections + +1. Open `推荐峰表` or `Peaks` and filter the phase-name column. +2. Sort `normalized_intensity` descending, or `rank_by_intensity` ascending, + to find the strongest calculated reflections in that phase. +3. Compare `two_theta_deg` at the experimental wavelength. When comparing + different wavelengths, use `d_spacing_A` instead. +4. Expand grouped detail columns for source hashes, intensity definitions, + and additional metadata. + +The header and phase-identification columns remain visible while scrolling. +Column-group colors distinguish identity, Miller indices, geometry, intensity, +and elasticity. Alternating row shading and consistent numeric formats help +track a reflection across the table. Intensity bars use the existing 0–100 +phase-relative scale; they do not measure phase fraction or make intensities +quantitatively comparable between phases. Percent columns already contain +values from 0 to 100 and are displayed without multiplying them by 100 again. + +Blank scientific values remain blank. For example, a missing directional +modulus is not zero: inspect `elastic_status` and `elastic_note` for the reason. +The full field definitions are in [SCHEMA_ALIASES.md](SCHEMA_ALIASES.md). + +## Keep an editable copy + +The original bundle is an integrity-checked record. In the desktop interface: + +- **Preview Excel** opens a temporary copy outside the result bundle. +- **Save Excel as** creates a permanent, editable copy at the location you choose. +- **Result folder** opens the complete bundle, including CSV, diagnostics, + provenance, and the manifest. + +Use **Save Excel as** for notes, filtering, or a workbook you intend to share. +Opening or editing a separate copy keeps Excel lock files and user edits out +of the verified bundle. If a preview was changed and saved, the application +retains it instead of deleting those changes on exit. + +For a direct command-line Excel export: + +```bash +diffractscout quick-export path/to/sample.cif -o outputs/sample.xlsx +diffractscout verify outputs/sample_bundle +``` + +This creates both `sample.xlsx` and `sample_bundle/results.xlsx`. The `.xlsx` +shortcut requires Excel output; use a directory output with `--no-excel` for +a CSV-only bundle. Existing output needs explicit `--overwrite` authorization. +If publishing the separate Excel copy fails, the error identifies the retained +bundle workbook so that the calculation does not need to be repeated. + +## Automation and large tables + +Use named worksheets or the CSV files in scripts rather than relying on the +active worksheet, visible columns, or display precision. Grouped columns keep +their original data and headers. The `Summary` sheet remains a two-column +`key` / `value` table. + +The workbook uses the existing conservative limit of 900,000 data rows per +table. When a table exceeds that limit, its sheet contains an explicit omission +notice and the complete CSV remains in the bundle. Printed overview pages are +intended for a readable summary; use the full tables or CSV for detailed work. diff --git a/docs/GUI.md b/docs/GUI.md index 2d2b368..844c16f 100644 --- a/docs/GUI.md +++ b/docs/GUI.md @@ -48,6 +48,18 @@ Dense forms (radiation, Cij, export options) live in **vertically scrollable** c Duplicate input paths are removed. Only readable files with the `.cif` extension enter the calculation. Selecting `Replace an existing verified DiffractScout bundle` authorizes replacement only when the existing directory contains a supported manifest and currently passes the bundle-integrity check. +## Reuse analysis settings + +Use **Save parameters** to store the current analysis and export options in a +JSON preset, and **Load parameters** to reuse them for another input set. Presets +include radiation, angular and d-spacing ranges, profile parameters, resource +limits, optional outputs, and the local recursive-scan option. The complete +preset is validated before it changes the form. + +Presets do not contain API keys, input or output paths, overwrite authorization, +or manually entered elastic tensors. Loading one therefore leaves those +choices with the current run. Save/load actions are disabled during a task. + ## Materials Project pipeline ![Materials Project pipeline interface](assets/gui-materials-project.png) @@ -110,7 +122,7 @@ settings are available through the CLI and Python `AnalysisSettings`: | Continuous pattern series | on | `--no-patterns` | | 2θ figure generation | off | `--figures`, `--figure-preset`; v0.4.0 figures remain on 2θ regardless of `--pattern-axis`; SVG/PNG bundle output works in the base install and `.[figures]` enables the matplotlib path | -Laboratory views add bilingual convenience sheets to `results.xlsx` without changing the English canonical CSV columns. See [SCHEMA_ALIASES.md](SCHEMA_ALIASES.md) and [ENGINE_PARITY.md](ENGINE_PARITY.md). +Laboratory views add bilingual convenience sheets to `results.xlsx` without changing the English canonical CSV columns. The workbook also provides a result overview, worksheet links, column-group colors, consistent number formats, and intensity bars. See the [Excel guide](EXCEL.md), [SCHEMA_ALIASES.md](SCHEMA_ALIASES.md), and [ENGINE_PARITY.md](ENGINE_PARITY.md). The desktop label intentionally says “CSV/Excel pattern coordinate.” For the reciprocal choices, `q=2π/d` and `g=1/d` in Å⁻¹. These choices affect only @@ -161,11 +173,18 @@ The Activity panel reports timestamps and separates informational, warning, and - completion with one or more error diagnostics; - completion with no analyzable phases, where a diagnostic-only bundle is still available. -The `Open result` action is enabled after a valid result bundle has been -written. It previews `results.xlsx` when that workbook exists; otherwise it -opens the result directory. A later failed retry preserves this action only -while the previous bundle still exists. `Copy` places the current Activity log -on the clipboard; `Clear` affects only the displayed log. +Completion reports the phase, reflection, warning, and error counts. **Preview +Excel**, **Save Excel as**, and **Result folder** provide separate result +actions. Excel actions require an existing workbook; the folder action also +works for runs without Excel. These actions are disabled during a task, and a +failed retry retains access to the previous result while its files still exist. + +Preview opens a temporary copy outside the verified bundle. Use **Save Excel +as** for a permanent editable copy. Saved changes in a preview are retained on +exit, with their location reported, instead of being discarded. See the +[Excel guide](EXCEL.md) for the workbook layout and editing workflow. `Copy` +places the current Activity log on the clipboard; `Clear` affects only the +displayed log. ## Threading and window closure diff --git a/docs/JOSS_6_MONTH_PLAN.md b/docs/JOSS_6_MONTH_PLAN.md index 9260851..bf1a883 100644 --- a/docs/JOSS_6_MONTH_PLAN.md +++ b/docs/JOSS_6_MONTH_PLAN.md @@ -1,81 +1,125 @@ -# Six-month public-development and submission plan +# Public development and JOSS submission plan -The public DiffractScout repository was created on **12 August 2026**. Six -calendar months are completed on **12 February 2027**; the project uses **15 -February 2027** as its earliest practical submission date. Passing the date is -necessary but not sufficient: JOSS editors assess whether development is real, -open, distributed, and connected to scholarly use. +The public DiffractScout repository was created on **12 August 2026 at +10:02:29 UTC**. The current JOSS submission guidance requires more than six +months of public history with active development spanning that period. The +six-calendar-month anniversary is 12 February 2027; the project retains +**15 February 2027** as the earliest practical submission date. Passing that +date is necessary but insufficient: the public record must show iterative +work across the period and the research-impact and open-source-practice gates +must also be satisfied. -Use the staged preflight: +Use the staged local preflight: ```bash -# Software release candidate +# Exact-source software release acceptance python scripts/check_release.py python scripts/joss_readiness.py --stage release --strict -# JOSS submission candidate; the date does not override missing evidence +# Submission candidate; use the actual assessment date python scripts/joss_readiness.py \ --stage submission --strict --as-of YYYY-MM-DD \ --output build/joss-readiness -# Post-review tag/archive consistency +# After JOSS review and before publication python scripts/joss_readiness.py \ --stage publication --strict --as-of YYYY-MM-DD ``` -The report cannot prove that local commits were public. Retain meaningful public -GitHub commit, issue, pull-request, and Release URLs as the authoritative -timeline and add completed records to the ledger's -`public_development_activity` array. The strict gate counts distinct calendar -months only from those auditable URLs; scheduled workflow runs alone do not -qualify. - -## Milestones from 12 August 2026 - -| Period | Required work | Public evidence | +The script's four-active-month rule and public, hashed research-use, +diffraction, and elasticity records are stricter **project acceptance +policies**. JOSS's current screen has no separate four-month count or explicit +requirement for both types of independent numerical comparison. JOSS does +require more than six months of active, iterative public development, public +issues/PR and release evidence for a recently opened repository, and +demonstrated research impact. Do not describe the stricter local gates as +official JOSS rules. + +## Evidence already on the public record + +The GitHub API was checked on 26 September 2026. It reports repository +creation on 12 August, one public release (`v0.3.0`) that day, merged PRs from +12–27 August, and a later public iteration in [PR #14](https://github.com/D-sudoasd/DiffractScout/pull/14), +whose current head is commit [`a09ee1b`](https://github.com/D-sudoasd/DiffractScout/commit/a09ee1b0e48de4bf07898bceda4bec46b70dab43). +The default-branch snapshot recorded at the start of this audit was commit +`0d56f99` on 12 September; PR #14 was opened on 26 September and remains open +at this assessment. This amounts to public activity in August and September, +with earlier work concentrated in August. Commits dated before the repository +became public do not count as public history. An open public PR can document +iteration, but it must not be described as merged or released until those +events occur. CI and the official Open Journals build both passed for +candidate commit `a09ee1b`; neither result replaces the public-age requirement +or author-controlled evidence. Private work, scheduled CI runs, and download +statistics do not establish public development or external adoption. + +The submitting author reports that DiffractScout was used in multiple +published papers. Representative citations and a mapping from each paper to +the software version, function used, and research result still need to be +recorded. The report is not treated as a lack of research use; it is a lead +that must be made auditable before the manuscript claims those examples. + +## Milestones + +| Period | Work tied to a real project need | Public evidence to retain | |---|---|---| -| Aug-Sep 2026 | Complete and publish the v0.4.0 baseline after release-stage, clean-wheel, cross-platform CI, and official paper-build checks | tag, GitHub Release, CI and draft-build URLs, release artifacts | -| Sep-Oct 2026 | Complete one lawful, frozen multiphase-alloy workflow and an independent diffraction comparison with predeclared tolerances | case report, hashed inputs, bundle, comparison table, issue/PR | -| Nov 2026 | Obtain external clean installation or research-use feedback and resolve real user friction | public user report, issue/PR, consented attribution | -| Dec 2026 | Complete independent directional-elasticity validation and API/GUI documentation audit | tensor/basis report, comparison results, maintenance release | -| Jan 2027 | Freeze submission scope, audit every claim/citation, confirm author metadata, and complete the reviewer-checklist matrix | submission candidate, green CI, official JOSS PDF, readiness report | -| 12-15 Feb 2027 | Confirm six-month age and genuinely distributed public activity | public timeline audit | -| After 15 Feb 2027 | Submit only if `--stage submission --strict` passes and no blocking scientific or software issue remains | JOSS submission and public pre-review issue | -| After successful JOSS review | Create the final tag if the software changed, archive the repository, add the software DOI, and run publication-stage readiness | final tag, Zenodo/figshare record, DOI, review issue update | - -## Monthly operating rule +| Aug–Sep 2026 | Establish the release baseline, harden the analysis workflow, and complete the current usability, workbook, CLI, and result-inspection improvements. | Existing `v0.3.0`, merged pull requests, public commits, and open PR #14 with its current head and actual CI state; do not represent unmerged changes as released. | +| Oct 2026 | Verify representative published-use records with the author; map paper citations to software versions, functions, and the described research outcome. Prepare a lawful, frozen case record if the inputs and results may be shared. | DOI/URLs, source and result hashes, exact use, limitations, and consent where another person is attributed. | +| Nov 2026 | Gather and address actual installation, workflow, or review feedback; publish any needed usability and documentation changes. | Public user/reviewer issue, discussion, PR, or consented report; exact artifact tested and outcome. | +| Dec 2026 | Maintain and, if needed, extend the project's predeclared diffraction and directional-elasticity software comparisons; address any real discrepancies before claims are made. The current candidate already includes synthetic pymatgen reference-engine reports. | Reproducible reports with reference versions, frozen inputs, units, basis/conventions, tolerances fixed in advance, hashes, and limitations. These comparisons do not establish experimental validity or research impact. | +| Jan 2027 | Freeze the intended submission source, audit every software and manuscript claim, check references and author metadata, run final acceptance, CI, and Open Journals build. | Clean commit, complete local readiness report, green remote CI URL, official PDF build URL, and full-page PDF review. | +| 12–15 Feb 2027 | Verify the public age, distributed activity, impact evidence, and submission checklist against current JOSS guidance. | Dated public-history audit and a passing submission-stage preflight. | +| After successful JOSS review | Create the final tagged release and immutable software archive; synchronize the version and archive DOI in citation metadata and the review issue. | Tag, archive DOI, matching commit hashes, and updated review record. | + +These months are checkpoints, not a schedule for manufacturing activity. Publish +work when it resolves an actual software, validation, support, documentation, +or research-use need. A green scheduled workflow confirms only the tested +commit; it cannot replace iteration, research use, or community evidence. -Every public change must correspond to needed software, documentation, -validation, support, dependency, release, or research-use work. Do not create -empty commits, retrospective issues, invented contributors, or unsupported -claims to simulate activity. +## Submission gate -At the end of each month: +Before submitting, confirm all of the following against the same candidate: + +- The repository has been public for **more than six months**, and its public + history shows active iteration over that period, releases, and public + issue/PR work. +- The software has demonstrable research use. Index representative published + papers reported by the author or a complete, traceable research workflow; + identify the source version, exact use, and research result. +- The package is installable and feature-complete, with documentation, + examples, tests, CI, contribution instructions, and support/governance + expectations for the submitted commit. +- The JOSS paper is 750–1750 words and contains all required sections, + accurate citations, an AI usage disclosure, authorship/affiliation + metadata, and a funding acknowledgement or an author-confirmed statement + about the absence of financial support. +- The submitting author has confirmed the complete author list, affiliations, + funding and sponsor role, contributors, related publications, and full human + review of AI-assisted work. Every coauthor has agreed to be listed. +- The selected source is clean, release acceptance passes, required CI and the + official Open Journals build pass for the exact commit, and every PDF page + has been inspected. +- All research-impact statements point to traceable records. The stricter + project evidence gates also pass if the team continues to use the local + submission-stage readiness command. + +The archive DOI is normally created after the software review is completed: +JOSS's submission instructions ask authors to tag and archive the reviewed +source before acceptance and then update the review issue. It is not a +pre-review age requirement. -1. review the monthly reproducibility workflow and correct real failures; -2. retain demo, benchmark, readiness, and release artifacts for identifiable commits; -3. update the evidence ledger only with completed public URLs; -4. review open scientific and software issues; -5. update the changelog and roadmap from completed work; -6. review dependency pull requests through the full scientific regression path; -7. retain a non-strict submission-stage readiness snapshot. +## Monthly operating rule -A green scheduled run shows that a commit remains reproducible. It does not by -itself demonstrate sustained development or research impact. +At each monthly checkpoint: -## Submission gate +1. review real user reports, scientific discrepancies, and software defects; +2. make and publish changes only when a concrete need exists; +3. retain release, CI, benchmark, and readiness results with their exact + commits; +4. add public evidence only after checking its URL, date, outcome, version, + hashes, and attribution consent; label open PRs as proposed work and do not + count them as merged changes or releases; +5. update the manuscript and changelog from evidence and merged behavior; and +6. preserve pending or private facts as pending rather than estimating them. -- date is 15 February 2027 or later; -- public work is distributed across at least four calendar months; -- a supported wheel installs and runs outside the source tree; -- all required remote CI jobs and the official JOSS paper build pass; -- the 45-check analytic benchmark passes; -- the real multiphase case is public, lawful, frozen, hashed, and tied to an actual research decision; -- independent diffraction and elasticity validations are public; -- at least one public external-use, installation, review, issue, or contribution record exists; -- authorship, affiliations, funding, contributors, related work, references, and AI disclosure are human-confirmed; -- every Research impact statement claim maps to the evidence ledger; -- `python scripts/joss_readiness.py --stage submission --strict --as-of YYYY-MM-DD` passes. - -The immutable software archive DOI is a post-review publication requirement, -not a prerequisite for opening the initial JOSS submission. +Do not create empty commits, retrospective issues, or invented contributors to +simulate six months of development or collaboration. diff --git a/docs/JOSS_READINESS.md b/docs/JOSS_READINESS.md index 1b5e097..7379e8c 100644 --- a/docs/JOSS_READINESS.md +++ b/docs/JOSS_READINESS.md @@ -1,56 +1,85 @@ -# JOSS readiness assessment - -Assessment date: **12 August 2026** (static baseline snapshot; not a live status) - -Software version assessed: **0.4.0 release candidate** - -Public repository date: **12 August 2026** - -Earliest six-calendar-month date: **12 February 2027** - -Project's earliest practical submission date: **15 February 2027** - -Primary requirements consulted: - -- JOSS review criteria: `https://joss.readthedocs.io/en/latest/review_criteria.html` -- JOSS review checklist: `https://joss.readthedocs.io/en/latest/review_checklist.html` -- JOSS paper format: `https://joss.readthedocs.io/en/latest/paper.html` -- JOSS submission and archive sequence: `https://joss.readthedocs.io/en/latest/submitting.html` -- JOSS AI policy: `https://joss.readthedocs.io/en/latest/policies.html` - -This page records a dated baseline and is not updated automatically. For the -current assessment, run the live readiness script and inspect its generated -JSON/Markdown report, for example: - -```bash -python scripts/joss_readiness.py --stage release --output build/joss-readiness -``` - -The release receipt and engineering checks are software-process evidence only; -they do not establish scientific validity, real-material agreement, or JOSS -acceptance. - -## Current conclusion - -The repository has an MIT license, installable package metadata, CLI/API/GUI, -deterministic tests and analytic benchmarks, cross-platform CI, user and -developer documentation, contribution and support routes, a JOSS-format paper, -and an explicit AI disclosure. The public `main` workflow has run successfully, -but the 0.4.0 release candidate must still complete its own local and remote -acceptance checks. - -JOSS submission is **blocked** by evidence and time, not by a confirmed -scientific correctness defect. The project currently lacks a completed real -multiphase research-use case, independent public diffraction and elasticity -comparisons, external engagement, distributed public development over the -required period, and final human-confirmed submission metadata. +# JOSS submission readiness + +Assessment date: **26 September 2026** + +This assessment preserves two distinct snapshots. The baseline assessment +below refers to commit [`0d56f99`](https://github.com/D-sudoasd/DiffractScout/commit/0d56f99e10abd7e32e87c6f0e5344fc29ba3c670) +on `main` (12 September 2026), **before the 0.4.0 usability and manuscript +upgrade**. On 26 September, the maintainer opened public [PR #14](https://github.com/D-sudoasd/DiffractScout/pull/14) +with the 0.4.0 candidate, workbook and workflow updates, and synthetic +reference-engine comparisons. The PR is public development evidence, but its +changes were not yet merged at this snapshot. The new comparisons are +described below; their final evidence-ledger commit and hashes must match the +public source commit that contains the frozen reports. + +The GitHub API reports that the public repository was created at +**12 August 2026 10:02:29 UTC**. Six calendar months elapse on 12 February +2027; JOSS currently requires **more than** six months of public history, so +12 February itself is not an eligible submission date. The project retains +**15 February 2027** as a conservative earliest planning date, conditional on +active public development spanning that period. A date alone cannot satisfy +the history or research-impact criteria. + +## Assessment + +DiffractScout has an MIT license, Python package metadata, command-line, +Python API and desktop interfaces, automated checks, documentation, +contribution and support routes, and a JOSS-formatted manuscript. The public +repository has one tagged release (`v0.3.0`), merged maintenance and feature +pull requests, and public PR #14 opened on 26 September. The public record +therefore spans August and September 2026, with substantial activity +concentrated in August and a later iteration proposed in September. PR #14 +does not by itself establish a completed release or passing CI for its exact +candidate source. + +The submitting author has reported that the software was used in multiple +published papers. This is a meaningful report of actual research use, not a +reason to describe the use as independently verified: representative +citations, the software version and functionality used, and the specific +research outcomes still need to be mapped to the manuscript and evidence +ledger. Do not describe the project as having no research use. Do not turn a +reported use into a completed evidence record before its sources are +identified. + +The **official JOSS pre-review blockers** are the public-history threshold and +demonstrated research impact. The repository was created on 12 August 2026; +only August and September are represented so far, and the history is still +concentrated in August. PR #14 adds a later public iteration, but the required +period has not elapsed and future activity cannot be assumed. The author's +reported publication use needs traceable evidence from representative papers +or another documented research workflow. JOSS also screens iterative +development and open-source practice. External engagement is a strong +positive signal, not a standalone gate; no non-author interaction was +recorded at this snapshot. + +The repository's local readiness script applies additional project policies: +at least four months with indexed public activity, a hashed real-material +research-use case, public diffraction and elasticity comparison reports, and +an external-engagement record. These are **stricter local evidence gates**, not +separate numeric requirements in the JOSS reviewer checklist. They support +this project's scientific claim control and should not be represented as +verbatim JOSS criteria. A local comparison run can support a validation record +when its methods and predeclared tolerances are documented, but it is not +external user adoption or experimental validation. + +## Status by criterion + +| Area | Evidence at this assessment | State and next action | +|---|---|---| +| License and package | `LICENSE`, `pyproject.toml`, Python entry points | Present; rerun packaging checks on the final submission commit. | +| Scope and software need | README, comparison and scientific-contract documents, manuscript | Defined; manuscript must explain the workflow contribution and build-versus-contribute rationale. | +| User and API documentation | README, CLI/API/GUI guides, offline demo | Present; audit against the final public interfaces. | +| Automated verification | Test suite, analytic benchmark, CI, demo and bundle verifier | Remote CI for commit `39f1477` [#36243627211](https://github.com/D-sudoasd/DiffractScout/actions/runs/36243627211) passed all ten jobs, including Windows, macOS, Python 3.10–3.13, GUI, optional dependencies, and the installed wheel. The local preflight on that source passed 543 tests, 45 analytic checks, packaging, and isolated wheel workflows. Rerun acceptance when the selected source changes. Analytic fixtures do not establish experimental accuracy. | +| Public development history | Repository created 12 August; `v0.3.0`; merged PRs through August; public PR #14 opened 26 September | **Official blocker.** Only August and September are represented. Continue substantive, iterative public work for more than six months; the planned earliest date of 15 February 2027 is conditional, not an eligibility guarantee. | +| Research impact | Author reports multiple published uses; individual records are not yet indexed | **Official blocker until traceable evidence is supplied.** Add representative papers with DOI/URL, exact use, software version or commit, and the related research result. | +| Community engagement | Public issue and contribution routes exist; no non-author engagement is recorded | External engagement is a strong positive signal. Capture real use, review, discussion, or contribution with consent where attribution is used; do not treat its absence as a separate JOSS hard gate. | +| Manuscript | Required JOSS sections are present in the draft; the Open Journals workflow for commit `a09ee1b` [#36242597087](https://github.com/D-sudoasd/DiffractScout/actions/runs/36242597087) succeeded | Four-page PDF generation and source/artifact, rendered-page, reference, and text-layer checks passed. The submitting author still must confirm research-use citations, date, complete authors, funding, and full human review of AI-assisted work. | +| Submission metadata | Known author details have been confirmed by the user; other fields remain unconfirmed | Confirm the complete author list and consent, each affiliation, funding and sponsor role, contributors, related publications, and full human review of AI-assisted work. | +| Post-review archive | No final tagged archive DOI is recorded | Create the final tag and archive DOI after review, then synchronize the citation and review records. This is a post-review step, not an initial submission blocker. | -The local `v0.3.0` tag is retained as history and must not be moved. The first -complete public software baseline should be a new `v0.4.0` tag and GitHub -Release after release-stage acceptance. Tag creation, pushing, and Release -publication are external writes and are not part of the local implementation. +## Staged project preflight -## Three readiness stages +The local script reports release, submission, and publication stages: ```bash python scripts/check_release.py @@ -59,84 +88,63 @@ python scripts/joss_readiness.py --stage submission --strict --as-of YYYY-MM-DD python scripts/joss_readiness.py --stage publication --strict --as-of YYYY-MM-DD ``` -| Stage | Purpose | Evidence intentionally not required yet | -|---|---|---| -| `release` | Verify release structure, version/license metadata, evidence-ledger integrity, paper structure, canonical repository identity, and an exact-source receipt from the complete docs/tests/demo/benchmark/wheel/sdist/Twine preflight | Six-month history, research impact, external engagement, archive DOI | -| `submission` | Add public-development age and distribution, real research use, separate diffraction and elasticity validations, external engagement, remote CI, official JOSS build, and human metadata confirmation | Final post-review archive DOI | -| `publication` | After JOSS review, require the final commit, version tag, immutable software archive DOI, and citation metadata to match | None | - -Non-strict runs always return zero after producing an honest status report. -`--strict` returns a blocking exit code only for the selected stage. The JSON -and Markdown reports include the result for all three stages. Run -`python scripts/check_release.py` first to create the ignored, current-source -release receipt; skipped tests or package builds cannot create that receipt. -The receipt also covers a clean-environment install and demo/verify/benchmark/ -quick-export run of the newly built wheel. -Local release-candidate work may remain uncommitted, but submission and -publication stages require a clean checkout so the repository state, HEAD, -remote CI commit, official paper-build commit, and evidence hashes identify the -same source. - -## Repository and software status - -| Criterion | 0.4.0 candidate state | Evidence or remaining action | -|---|---|---| -| OSI-approved license | Complete | Plain-text MIT `LICENSE`; retained source notices in `NOTICE.md` | -| Installable research software | Implemented | `pyproject.toml`; CLI, quick-export, and GUI entry points; clean-wheel CI job | -| Scientific scope and need | Documented | README; paper; `docs/COMPARISON.md`; scientific contracts | -| User examples and API documentation | Implemented | README; `docs/API.md`; GUI guide; offline demo | -| Automated verification | Implemented | Full suite reported by pytest/CI; stable 45-check analytic benchmark | -| Continuous integration | Public baseline green | Re-run every required job on the selected 0.4.0 release commit | -| Non-destructive evidence bundles | Implemented | Staged writes, rollback, manifest verification, structured diagnostics | -| Contribution/support/security | Complete | Contribution guide, issue forms, support, conduct, governance, security | -| JOSS paper structure | Drafted | All required headings and current approximate word range pass the local preflight; `paper.pdf` must be rebuilt from the changed source | -| Public development | In progress | Public clock began 2026-08-12; only genuine distributed work linked in `public_development_activity` counts | -| Real research use | Pending author input | Intake scaffold at `validation_cases/real_multiphase_case/README.md` | -| Independent validation | Pending | One public diffraction and one public elasticity record are required | -| External engagement | Pending | At least one public external installation, use, review, issue, or contribution | -| Software archive DOI | Post-review | Required by `publication`, not by the initial `submission` stage | - -## Evidence that code changes cannot manufacture - -The packaged analytic suite and synthetic FCC fixture establish calculation, -schema, provenance, and integrity contracts. They do not establish experimental -validity, adoption, or research impact. Only completed and traceable records may -be added to `docs/evidence/impact_evidence.json`. - -Passing tests, benchmarks, CI, or release-preflight checks therefore remain -engineering readiness signals, not scientific validation or a JOSS acceptance -decision. - -The real multiphase case must identify the material, lawful input source, -software version, frozen CIF hashes, complete settings, result bundle, actual -research decision, and limitations. Online discovery context may be recorded, -but numerical reproduction must use the frozen and hashed structures. - -Independent diffraction and elasticity comparisons must declare their -tolerances and conventions before inspecting the DiffractScout result. External -feedback must be public and used with attribution consent. Empty evidence arrays -are preferable to prospective or inferred records. - -## Paper status - -The draft contains every required JOSS section and remains within the local -approximate word-count range. Its Summary has been simplified for a broad -reader. The Research impact statement deliberately remains limited to current -analytic and community-readiness evidence until the real case is complete. The -tracked PDF predates this revision and is only a layout baseline; submission -readiness remains blocked until the official Open Journals workflow rebuilds -the revised source and its public run URL is recorded. The readiness report -prints stable SHA-256 values for the paper inputs and PDF; submission requires -those exact values in the evidence ledger so a stale PDF cannot pass merely -because a fresh checkout changed filesystem timestamps. - -Before submission, the author must confirm authorship, ORCID, affiliation, -funding, contributors, related publications, the full human review of -AI-assisted work, a green remote CI URL, and the official Open Journals build -URL. During JOSS review, editor/reviewer responses must be written by the human -author rather than generated by AI. - -The final software archive DOI is added only after review when JOSS requests the -final tagged release and archive. It must then be synchronized across -`CITATION.cff`, the bibliography, GitHub Release, and the review issue without -being confused with the later JOSS article DOI. +For final submission checks, copy the evidence ledger to an ignored file and +update that copy with the selected commit's verification records and the +author's confirmations. This preserves the identity of the committed source +while recording its completed checks: + +```bash +mkdir -p outputs +cp docs/evidence/impact_evidence.json outputs/submission-evidence.json +python scripts/joss_readiness.py --stage submission --strict --evidence-file outputs/submission-evidence.json +``` + +`--evidence-file` reads the specified ledger without changing it and records +its path in the report. The usual schema, hashes, and confirmation requirements +still apply. Remote CI must match the selected HEAD. An official PDF build may +use HEAD or an ancestor only when Git proves that the manuscript, bibliography, +and both figure blobs are identical to HEAD; PDF and source hashes are still +checked. This permits recording the generated PDF without invalidating its +build evidence. Without this option, the repository's shared ledger is used. + +The `release` stage is a project acceptance check over the exact source. It +does not prove scientific validity or submission eligibility. The `submission` +stage includes both official criteria and the stricter local project gates +described above; inspect each named check rather than treating its aggregate +blocker count as a JOSS verdict. The `publication` stage is run after review, +when the final tagged source and immutable software archive DOI are available. + +The earlier non-strict submission snapshot used `--as-of 2026-09-26` against +the **pre-upgrade, uncommitted working source**. It is a diagnostic snapshot, +not a readiness result for PR #14. Rerun release and submission checks against +the selected clean commit after the source, comparison receipts, paper, and +remote CI are frozen; do not reuse the ignored report in `outputs/` as current +evidence. + +## Evidence boundaries + +The 45-check analytic benchmark, synthetic FCC workflow, and new pymatgen +reference-engine comparison verify specified mathematical, software, and +provenance contracts for named synthetic fixtures. They do not by themselves +show that the software informed a research decision, agree with experimental +measurements, or demonstrate adoption. The author-reported published uses +should be represented as specific examples in the paper only after the +corresponding papers and software contribution have been verified and cited. + +Likewise, repository views, stars, downloads, scheduled workflows, synthetic +engine comparisons, or a public PR do not establish external adoption or a +six-month public development history. PR #14 is a public record of proposed +iteration; do not describe it as merged or released before that occurs. Add +records to `docs/evidence/impact_evidence.json` only when each public URL, +date, outcome, source version, and consent requirement can be checked. + +## Official JOSS sources checked 26 September 2026 + +- [Submission requirements and pre-review screening](https://joss.readthedocs.io/en/latest/submitting.html) +- [Review criteria](https://joss.readthedocs.io/en/latest/review_criteria.html) +- [Reviewer checklist](https://joss.readthedocs.io/en/latest/review_checklist.html) +- [Paper format and length](https://joss.readthedocs.io/en/latest/paper.html) +- [JOSS policies, including AI disclosure](https://joss.readthedocs.io/en/latest/policies.html) + +JOSS editors and reviewers make the final scope, significance, and acceptance +decisions from the submitted software, paper, and public record. diff --git a/docs/JOSS_REVIEW_CHECKLIST.md b/docs/JOSS_REVIEW_CHECKLIST.md index a1776f9..9e89819 100644 --- a/docs/JOSS_REVIEW_CHECKLIST.md +++ b/docs/JOSS_REVIEW_CHECKLIST.md @@ -1,62 +1,84 @@ # JOSS reviewer-checklist evidence matrix -This matrix maps the current JOSS reviewer checklist to repository evidence. -It is a project-side acceptance aid, not a claim that JOSS has approved the -submission. Public URLs and human confirmations must be recorded before the -submission-stage gate can pass. +Assessment date: **26 September 2026**. This project-side matrix follows the +[current JOSS reviewer checklist](https://joss.readthedocs.io/en/latest/review_checklist.html) +and its [review criteria](https://joss.readthedocs.io/en/latest/review_criteria.html). +It records evidence and open work; it is not a JOSS decision. -## General, development, and functionality +## General checks and open development -| Reviewer item | Repository evidence | Current state | Submission evidence still required | +| JOSS item | Repository evidence | Current assessment | Evidence or action still required | |---|---|---|---| -| Repository and OSI license | Public GitHub repository; `LICENSE`; `NOTICE.md` | Available | Recheck the exact submission commit | -| Contribution and authorship | `AUTHORS.md`; Git history; `CITATION.cff` | Drafted | Human-confirm author list, order, contributions, and consent | -| Scope and significance | README; paper Statement of need; comparison document | Documented | Link the completed real multiphase case and its research decision | -| Sustained open development | GitHub commits, issues, pull requests, releases | Clock started 2026-08-12 | Show genuine activity distributed through at least 2027-02-12 | -| Collaborative effort | Contribution routes and research-use issue form | Routes available | Public external installation, review, issue, or contribution | -| Installation | `pyproject.toml`; README; wheel CI job | Automated in CI | Retain clean-wheel run URL for the submission commit | -| Functionality | CLI/API/GUI tests; demo and bundle verifier | Automated offline | Reviewer-style clean install and public real-material workflow | -| Performance claims | No speed or scaling claim in the paper | Not applicable | Do not add a performance claim without a reproducible benchmark | - -## Documentation and verification - -| Reviewer item | Repository evidence | Current state | Submission evidence still required | +| Repository and license | [Public GitHub repository](https://github.com/D-sudoasd/DiffractScout); `LICENSE` | Available | Recheck the exact submission commit and issue access. | +| Major contribution and complete author list | `AUTHORS.md`, `CITATION.cff`, paper metadata, Git history | User confirmed the listed Delun Gong name, ORCID, and Institute of Metal Research affiliation. | Confirm that the author list is complete, contributions and order are appropriate, and every coauthor consents; update all metadata consistently. | +| Scope and significance | README, manuscript, comparison and scientific-contract documents | Research application and workflow are described. The author reports use in multiple published papers. | Map representative papers to DOI/URL, precise use, version or commit, and the relevant manuscript claim. Until then the statement is not yet auditable. | +| Sustained public development | GitHub repository metadata, release, merged PRs, public commits, and [open PR #14](https://github.com/D-sudoasd/DiffractScout/pull/14) | Public since 12 August 2026; public activity is documented in August and September, including the PR opened 26 September. Most earlier activity was clustered in August. | **Official blocker:** continue substantive, iterative public work for more than six months before submission. The repository date alone does not pass this check; PR #14 is public iteration, not a merged release. | +| Open-source workflow | `CONTRIBUTING.md`, issue forms, support routes, CI, one release, public PRs | Core routes exist; public PRs are maintainer-authored and no non-author discussion or contribution is recorded. | Keep resolving real issues and publishing needed changes. External engagement is a strong positive signal. Do not count self-authored roadmap issues, bots, or scheduled CI as external engagement. | +| Good open-source practices | MIT license, package metadata, tests/CI, documentation, support and governance | Present in the current public baseline. | Recheck installation, release, CI, and documentation on the final commit. | + +The internal `public_development_activity` ledger records verifiable public +milestones, including an open PR as proposed public iteration with its open +status stated. It excludes commits before the repository became public, +automated CI runs, and future work. Do not describe an unmerged PR as +integrated or released. The project's four-active-month rule is a conservative +local threshold; JOSS's current screen instead requires more than six months +of active public development spanning that period and evidence of release and +public issue/PR activity. + +## Functionality and documentation + +| JOSS item | Repository evidence | Current assessment | Evidence or action still required | |---|---|---|---| -| Statement of need | README; paper | Available | Audit against completed research-use evidence | -| Dependencies and installation | `pyproject.toml`; README | Available | Verify wheel on supported Python and operating systems | -| Example usage | Offline demo; examples; GUI guide | Available | Add completed multiphase case without restricted inputs | -| Functionality/API documentation | `docs/API.md`; `docs/GUI.md`; scientific contracts | Available | Review every public interface changed after v0.4.0 | -| Automated tests | `tests/`; CI; analytic benchmark | Available | Full CI green on the selected submission commit | -| Contribution, issue, and support paths | `CONTRIBUTING.md`; issue forms; `SUPPORT.md` | Available | Confirm links work without special access | -| Independent diffraction validation | Validation template and registry | Pending | Public comparison with predeclared tolerances | -| Independent elasticity validation | Validation template and registry | Pending | Public tensor, basis, directions, and comparison | - -## Software paper - -| Reviewer item | Repository evidence | Current state | Submission evidence still required | +| Installation | `pyproject.toml`, README, wheel CI | Python packaging and documented install path exist. | Preserve a clean-environment wheel install and run result for the submitted commit. | +| Core functionality | CLI, Python API, desktop GUI, offline demo, bundle verifier | Deterministic offline workflow and scientific contracts are documented. | Verify user-visible claims against the final release candidate; do not imply Materials Project live service verification from offline checks. | +| Performance claims | Manuscript makes no speed or scaling claim | Not applicable. | Keep it that way unless a reproducible, representative benchmark supports a new claim. | +| Statement of need and examples | README, API/GUI guides, demo | Available. | Ensure the example and final software version agree. Add a real-material example only when its source and license permit redistribution. | +| Functionality/API documentation | `docs/API.md`, `docs/GUI.md`, scientific contracts | Core behavior has dedicated documentation. | Audit public APIs and GUI behavior after the current usability changes. | +| Tests and correctness checks | `tests/`, CI run [#36242599844](https://github.com/D-sudoasd/DiffractScout/actions/runs/36242599844), 45-check analytic benchmark, and synthetic pymatgen diffraction/elasticity comparison | Remote CI succeeded on commit `a09ee1b`; reference-engine reports are limited to the named synthetic inputs and conventions. | Run the complete local release check on the exact final clean source. Analytic and cross-engine checks are not experimental validation. | +| Community pathways | `CONTRIBUTING.md`, issue templates, `SUPPORT.md`, `GOVERNANCE.md` | Present. | Confirm public access and route new user feedback into a traceable issue or report. | + +## JOSS paper + +JOSS currently asks for a 750–1750-word Markdown paper with Summary, Statement +of need, State of the field, Software design, Research impact statement, AI +usage disclosure, author and affiliation metadata, key references, and +financial-support acknowledgement. It should discuss software rather than +present new scientific results or duplicate API documentation. + +| Paper item | Repository evidence | Current assessment | Evidence or action still required | |---|---|---|---| -| Summary | `paper/paper.md` | Drafted for non-specialists | Final author read-through | -| Statement of need | `paper/paper.md` | Drafted | Connect to the completed real workflow | -| State of the field | Paper and `docs/COMPARISON.md` | Drafted | Final citation and build-vs-contribute audit | -| Software design | Paper; architecture and scientific-contract docs | Drafted | Confirm it describes the submitted version | -| Research impact statement | Analytic benchmark and evidence ledger | Incomplete | Replace pending language with only traceable real-use evidence | -| AI usage disclosure | Paper and `AUTHORS.md` | Drafted | Human author confirms every disclosed scope and verification step | -| References | `paper/paper.bib`; draft workflow | Drafted | EditorialBot reference check and final venue-name audit | -| Rendered paper | `paper/paper.pdf`; official draft workflow | Local baseline available | Record successful official Open Journals build URL and inspect every page | +| Summary and Statement of need | `paper/paper.md` | Drafted. | Read for non-specialist clarity and match claims to the submitted software. | +| State of the field | `paper/paper.md`, `paper/paper.bib`, `docs/COMPARISON.md` | Related tools and build-versus-contribute rationale are present. | Verify primary references, full venue names, and all relevant research/software citations. | +| Software design | `paper/paper.md`, `docs/ARCHITECTURE.md` | Architecture is described. | Explain key trade-offs and why they matter for this research workflow. | +| Research impact statement | Analytic benchmark, author report, evidence ledger | Author reports published use; representative source records are pending. The synthetic 45-check benchmark is reproducibility evidence, not evidence of research use or impact. | Cite the reported research papers or another auditable use record and keep each claim specific. Do not rely solely on future potential. | +| AI usage disclosure | `paper/paper.md`, `AUTHORS.md` | A disclosure exists for earlier development. | Update exact tool/model versions, where they were used, assistance scope, and verification methods for all work included in the submission; the submitting author must confirm complete human review. | +| Authors, affiliations, and funding | Paper metadata, `AUTHORS.md`, `CITATION.cff` | The user confirmed the currently listed author's name, ORCID, and affiliation. | Confirm completeness and consent of the author list; list every financial-support source and sponsor role, or state the author-confirmed absence of such support. | +| Rendered paper | `paper/paper.pdf`, Open Journals workflow run for commit `a09ee1b` [#36242597087](https://github.com/D-sudoasd/DiffractScout/actions/runs/36242597087) | The official workflow built the manuscript at commit `a09ee1b`. Four-page rendering and source/artifact identity, references, and text layer were checked. | The submitting author must review the final manuscript and set its date to the actual submission date before submission. | + +## Project-specific scientific evidence gates + +The local readiness tool is intentionally stricter than the JOSS checklist. It +requires a versioned, hashed real research-use case and public, pre-toleranced +diffraction and elasticity comparison reports. The current PR contains +regenerated synthetic comparison receipts, but the evidence-ledger hashes and +immutable source commit must be synchronized to the final public report before +those local report gates are counted as complete. The hashed real research-use +case and external-engagement records remain empty. These checks are project +quality controls, not additional JOSS checklist boxes. A calculation against +another software implementation may support a numerical-comparison report; +it does not establish experimental validity, research impact, or external +adoption. -## Stage commands +## Final gate commands ```bash -# Create the current-source local acceptance receipt, then verify it. python scripts/check_release.py python scripts/joss_readiness.py --stage release --strict - -# Run this only as a submission candidate gate. python scripts/joss_readiness.py --stage submission --strict --as-of YYYY-MM-DD - -# Run after review, final tag, and immutable software archive DOI exist. python scripts/joss_readiness.py --stage publication --strict --as-of YYYY-MM-DD ``` -The evidence ledger must remain incomplete rather than recording inferred, -private, prospective, or duplicated evidence. +Run release and submission checks on the exact clean source selected for +submission. The publication-stage check applies after successful review, when +the final tagged release and archive DOI can be matched to the citation and +review records. diff --git a/docs/VALIDATION.md b/docs/VALIDATION.md index 15437f1..617d286 100644 --- a/docs/VALIDATION.md +++ b/docs/VALIDATION.md @@ -85,6 +85,27 @@ The 45 checks cover allowed and forbidden families, multiplicity, cubic plane sp The benchmark is an internal analytic validation with an implementation path separated from the production orchestration. It does not satisfy the project requirement for an external user or independent third-party comparison. +## Independent numerical implementations + +The [independent-engine case](../validation_cases/independent_engines/README.md) +compares the four packaged synthetic CIFs with pymatgen's `XRDCalculator` and +the cubic directional-modulus calculation with pymatgen's `ElasticTensor`. +Both diffraction paths use the same wavelength and conventional cells; +coincident reflection families are grouped before matching peak positions. +The report retains the compared values, input hashes, dependency versions, +predeclared tolerances, and diagnostic intensity differences. CI runs this +comparison in both Materials Project extra environments and uploads the +reports, independently of live database access. + +```bash +python -m pip install -e ".[mp]" +python scripts/compare_reference_engines.py --output outputs/reference_engines +``` + +This comparison checks agreement with an independent implementation on +synthetic structures. Experimental agreement and research-use evidence require +their own documented cases. + ## Synthetic reference fixture The offline demo uses: diff --git a/docs/assets/excel-overview.png b/docs/assets/excel-overview.png new file mode 100644 index 0000000..deeb8fa Binary files /dev/null and b/docs/assets/excel-overview.png differ diff --git a/docs/assets/gui-local.png b/docs/assets/gui-local.png index 57bf5e1..f9b27ba 100644 Binary files a/docs/assets/gui-local.png and b/docs/assets/gui-local.png differ diff --git a/docs/assets/gui-materials-project.png b/docs/assets/gui-materials-project.png index a4a5ba7..44c3361 100644 Binary files a/docs/assets/gui-materials-project.png and b/docs/assets/gui-materials-project.png differ diff --git a/docs/evidence/JOSS_AUDIT_2026-09-26.md b/docs/evidence/JOSS_AUDIT_2026-09-26.md new file mode 100644 index 0000000..16d56ee --- /dev/null +++ b/docs/evidence/JOSS_AUDIT_2026-09-26.md @@ -0,0 +1,125 @@ +# JOSS readiness evidence audit — 26 September 2026 + +The baseline timeline and assessment below are a dated snapshot of public +`main` at commit `0d56f99`, **before the 0.4.0 usability and manuscript +upgrade**. The follow-up at the end records the public PR #14 candidate as of +26 September 2026. This is an evidence audit, not a JOSS editorial decision. + +## Public-development timeline + +GitHub's repository API reported creation at **2026-08-12 10:02:29 UTC**. +The public `main` HEAD at this audit was +[`0d56f99e10`](https://github.com/D-sudoasd/DiffractScout/commit/0d56f99e10abd7e32e87c6f0e5344fc29ba3c670), +dated 12 September 2026. Public records reviewed: + +| Date (UTC) | Public record | What it documents | +|---|---|---| +| 12 Aug 2026 | [v0.3.0 release](https://github.com/D-sudoasd/DiffractScout/releases/tag/v0.3.0) | First tagged GitHub release. It is not an immutable archive DOI. | +| 12 Aug 2026 | [PR #3](https://github.com/D-sudoasd/DiffractScout/pull/3), [PR #5](https://github.com/D-sudoasd/DiffractScout/pull/5), [PR #6](https://github.com/D-sudoasd/DiffractScout/pull/6), [PR #7](https://github.com/D-sudoasd/DiffractScout/pull/7) | CI repair, CIF/peak workflow and Excel views, GUI layout, and quick-export race correction. | +| 17 Aug 2026 | [PR #8](https://github.com/D-sudoasd/DiffractScout/pull/8) | Scientific-analysis and desktop-workflow hardening. | +| 20 Aug 2026 | [PR #9](https://github.com/D-sudoasd/DiffractScout/pull/9) | Transaction and scientific-input safeguards. | +| 23 Aug 2026 | [PR #11](https://github.com/D-sudoasd/DiffractScout/pull/11) | Unit-handling and desktop workflow fixes. | +| 27 Aug 2026 | [PR #13](https://github.com/D-sudoasd/DiffractScout/pull/13) | Scientific parsing and release-workflow hardening. | +| 12 Sep 2026 | [main commit `0d56f99`](https://github.com/D-sudoasd/DiffractScout/commit/0d56f99e10abd7e32e87c6f0e5344fc29ba3c670) | Quick-export launch and GUI status/form-mapping fixes. | + +The reviewed activity is in **August and September only** and is concentrated +between 12 and 27 August. Commits dated before 12 August are not public +development evidence. Automated workflows are not counted as substantive +development. The public PRs reviewed were opened by the maintainer account; +the issue tracker at this snapshot had the author's roadmap issue but no +recorded non-author issue or discussion. This record does not establish +external adoption or multiple contributors. + +The GitHub history does establish open iteration and a release. It does not +yet establish more than six months of active public development. The latest +JOSS submission guidance requires the repository to have been public for +**more than six months** before submission, with active development spanning +that period; its checklist also asks for public issue/PR and release evidence. +The six-month anniversary is 12 February 2027. Do not submit on that date; +the project's conservative earliest date is 15 February 2027, subject to the +activity and evidence gates. + +## Research-use status + +On 26 September 2026 the submitting author reported that DiffractScout was +used in multiple published papers. The report is evidence that research-use +records should be collected; no specific article, DOI, version/commit, feature, +or research outcome was supplied to this audit. The claim is therefore not +yet indexed in `publications_or_preprints` or linked to a reproducible case. +Do not report a paper count, paper titles, or specific scientific outcomes +until the source records are identified and checked. + +The package's 45-check analytic benchmark and synthetic offline demonstration +are software and mathematical-contract evidence. They are not records of a +research decision, experimental validation, or external adoption. + +## Official JOSS gates and local project policy + +| Topic | Official guidance checked on 26 Sep 2026 | Project status at this audit | +|---|---|---| +| Public age and iteration | More than six months public, active development across the period, evidence of a release and public issues/PRs; concentrated history can fail screening. | **Blocked by time and distribution.** Two public calendar months are recorded; meaningful work must continue through the period. | +| Research impact | Evidence of actual research use, at least by the developers; citations, documented workflow use, and external integration are examples. A future-potential claim alone is insufficient. | **Reported, source audit pending.** The author identifies published use, but the representative source records and exact software contribution remain to be recorded. | +| Open-source practice | For a single-author project, JOSS accepts broader evidence, but expects multiple indicators such as ongoing public commits, release/changelog, tests/CI, documentation, contribution and support expectations. | Core routes and one release exist; continue iterative activity and capture real interactions. | +| Paper | 750–1750 words; required sections, references, author/affiliation metadata, applicable publication examples, funding acknowledgement, and specific AI disclosure. | The Open Journals workflow built commit `a09ee1b` successfully and the four-page PDF passed artifact/source and rendered-content checks. Author confirmation, impact-source mapping, funding, and human review of AI-assisted work remain open. | +| Project's stricter scientific gates | Not separate boxes in the current JOSS checklist. | Local script also requires four active months, a hashed real-use case, diffraction and elasticity comparison reports, and an external-engagement record. Keep these labeled as local controls. | + +JOSS's current sources are [submission requirements](https://joss.readthedocs.io/en/latest/submitting.html), +[review criteria](https://joss.readthedocs.io/en/latest/review_criteria.html), +[review checklist](https://joss.readthedocs.io/en/latest/review_checklist.html), +[paper format](https://joss.readthedocs.io/en/latest/paper.html), and +[policies](https://joss.readthedocs.io/en/latest/policies.html). + +## Actions + +1. Identify representative papers named by the submitting author. For each, + verify the DOI/URL, how DiffractScout was used, the matching release or + commit, and the claim the paper can support. Add the citation to the paper + bibliography and `publications_or_preprints` only after verification. +2. Continue substantive public maintenance across each remaining month. + Record meaningful public issues and PRs with their actual open/merged state; + describe integrated behavior only after merge, and do not backdate activity + or count CI as development. +3. Complete the final 0.4.0 source acceptance, remote CI, and official + Open Journals build against one clean commit. Update the readiness snapshot + after those checks rather than carrying the pre-upgrade diagnostic forward. +4. Keep the local scientific-validation requirements separate from JOSS + official gates. Publish methods, conventions, predeclared tolerances, and + limitations before adding either comparison as completed evidence. +5. Obtain the submitting author's final confirmation of the complete author + list and consent, funding and sponsor role, related-work disclosures, and + human review of every AI-assisted output. The currently listed author's + name, ORCID, and affiliation were confirmed, but that does not complete the + remaining submission metadata. + +After successful JOSS review, create the final tag and immutable archive DOI +and add them to `archived_releases`; an ordinary GitHub Release does not fill +that post-review step. + +## Current candidate follow-up — 26 September 2026 + +Public [PR #14](https://github.com/D-sudoasd/DiffractScout/pull/14) is open +with head commit +[`a09ee1b0e48de4bf07898bceda4bec46b70dab43`](https://github.com/D-sudoasd/DiffractScout/commit/a09ee1b0e48de4bf07898bceda4bec46b70dab43). +It records later public iteration for the usability/workbook/CLI workflow and +contains regenerated synthetic reference-engine comparisons. The Markdown +report SHA-256 is +`1f90eacd54dad5eaa93c66ec33ad2747f76df608b1f1275abedd90f98484c8ab`; the +matching JSON receipt SHA-256 is +`9c70131e03dc65b7a49c3dabddd131aa011f238299680ccf423bc73bee9c119b`. +Both hashes were checked against the files in commit `a09ee1b`. The records +support a numerical software comparison on the named synthetic fixtures; +they are not experimental validation, a real-material property claim, +research-impact evidence, or external adoption. + +The successful Open Journals run +[#36242597087](https://github.com/D-sudoasd/DiffractScout/actions/runs/36242597087) +used commit `a09ee1b0e48de4bf07898bceda4bec46b70dab43`. It confirms PDF +generation for the current manuscript; full-page inspection remains pending. +CI run [#36242599844](https://github.com/D-sudoasd/DiffractScout/actions/runs/36242599844) +for the same commit completed successfully. These runs establish the recorded +automated checks and PDF artifact only; they do not resolve author-controlled +metadata or research-use evidence. +The public-history period and source verification for the author's reported +published uses remain open. The earlier `--as-of 2026-09-26` diagnostic is a +pre-upgrade snapshot and must not be presented as a readiness result for this +candidate. diff --git a/docs/evidence/README.md b/docs/evidence/README.md index 89c9355..eddf3d2 100644 --- a/docs/evidence/README.md +++ b/docs/evidence/README.md @@ -1,54 +1,82 @@ # JOSS evidence ledger -`impact_evidence.json` is the machine-readable index used by -`scripts/joss_readiness.py`. Add an entry only after the underlying record is -publicly accessible or retained in a stable institutional repository. - -Each entry should contain, where applicable: - -- `date`: ISO date; -- `title`: concise description; -- `url`: public issue, pull request, release, DOI, dataset, protocol, preprint, - presentation, or archived report; -- `software_version`: exact DiffractScout version or commit; -- `people`: contributors or validators who consent to attribution; -- `claim_supported`: the precise manuscript statement supported by the record; -- `validation_type`: required for independent validations; use `diffraction` - or `elasticity` so both submission gates can be audited separately; -- `notes`: limitations, unresolved discrepancies, or access restrictions. - -Completed research-use and independent-validation entries must also bind to a -version-controlled report under `validation_cases/` through `report_path` and -`report_sha256`. Research-use records require frozen input and result-manifest -hashes, a lawful input license, the actual research decision, and limitations. -Diffraction validations require the reference implementation/version, -predeclared tolerance basis, frozen input hash, radiation and scan range; -elasticity validations additionally require tensor source/frame, Voigt -convention and `[100]`, `[110]`, `[111]` directions. External engagement must -record the tested artifact, commands, verification result, concrete outcome, -limitations, and whether attribution consent was obtained. Consent must be true -when `people` names anyone; an unattributed public record may explicitly use -false. A merely non-empty array cannot satisfy the submission gate. - -`public_development_activity` records only meaningful, publicly visible work in -the canonical GitHub repository: commits, issues, pull requests, or releases -that document real maintenance, validation, dependency, documentation, or user -work. The submission gate counts distinct calendar months from these URLs. It -does not treat local commit timestamps, empty issues, scheduled CI, stars, or -download counts as proof of distributed public development. - -Evidence must reflect real activity. Empty arrays are preferable to inferred, -private, duplicated, or retrospective records. Personal correspondence may be -summarized only with the other person's permission and should be replaced by a -public issue or archived validation report when possible. - -`submission_metadata` contains explicit human confirmations and public URLs for -the selected remote CI run and official Open Journals paper build, plus the -exact 40-character commit tested by each. Both commits must equal the current -submission commit. Leave these -values false or empty until the submitting author has checked them. After the -official build and page review, record the SHA-256 values reported by readiness -for the paper inputs and PDF; this prevents a stale tracked PDF from passing in -a fresh checkout. An archived -software release DOI belongs to the post-review `publication` stage, not the -initial `submission` stage. +`impact_evidence.json` is the machine-readable index checked by +`scripts/joss_readiness.py`. The assessment and claim rules are in +[`docs/ADOPTION_AND_IMPACT.md`](../ADOPTION_AND_IMPACT.md) and +[`docs/JOSS_READINESS.md`](../JOSS_READINESS.md). + +## Evidence rules + +Add a record only after its source is publicly accessible or retained in a +stable repository. Each record must have a source URL, date, title, exact +software version or commit, and the precise claim it supports. Additional +fields are defined by `impact_evidence.schema.json` and validated by +`scripts/joss_readiness.py`. + +- Put papers and preprints in `publications_or_preprints`; include the DOI or + stable URL and explain the software's specific role. The author reported + published uses on 26 September 2026, but representative citations have not + yet been identified in this ledger. +- Add a `research_use_cases` record only for a completed, versioned workflow + with a report under `validation_cases/`, its report hash, frozen input and + result-manifest hashes, lawful input license, research decision, and + limitations. The local gate intentionally requires this fuller case record; + a publication citation and a replayable case are distinct evidence types. +- Add an `independent_validations` record only after completing its report + under `validation_cases/`. State the reference implementation/version and + tolerance basis before comparing results. Diffraction records include the + input hash, radiation, and scan range. Elasticity records include tensor + source/frame, Voigt convention, and `[100]`, `[110]`, `[111]` directions. +- Add `external_engagement` only for a real external interaction or test. + Identify the tested artifact, commands, result, outcome, and limitations. + Set consent to true before naming a person; an unattributed record may set it + false. +- Add `public_development_activity` for meaningful public commits, issues, + pull requests, and releases that document real work. Record the public URL + and outcome. An open public PR can evidence proposed iteration if its status + is explicit; do not describe it as integrated or released. Do not count + private history, planned or empty issues, scheduled CI, stars, views, or + download counts. PR #14 currently points to public head commit + `a09ee1b0e48de4bf07898bceda4bec46b70dab43`. +- Add an `archived_releases` record only when the exact final tag is archived + with a persistent DOI. An ordinary GitHub Release does not satisfy that + post-review record. + +JOSS currently requires more than six months of active public development +history and evidence of research use at least by the developers. External +engagement is a strong positive signal. The local readiness tool adds stricter +project gates for four active public months, a hashed research-use case, both +diffraction and elasticity reports, and external engagement. The project +should report these as local controls, not as verbatim JOSS requirements. + +## Submission metadata + +Use `scripts/joss_readiness.py --evidence-file PATH` for a final audit ledger +stored under ignored `outputs/` or outside the repository. Copy the shared +ledger, then record the selected commit's CI/build results and confirmed +author inputs in that copy. Updating a separate file keeps the selected Git +commit unchanged. The checker reports the file it used and applies the same +validation rules; an unreadable or invalid file blocks the check. + +Remote CI must verify the selected HEAD. The official PDF build may refer to +HEAD or an ancestor with identical Git blobs for `paper.md`, `paper.bib`, +`fig_workflow.png`, and `fig_validation.png`. The checker rejects unknown or +unrelated commits and changed manuscript inputs, and still checks the recorded +paper-source and PDF hashes. + +Keep author-controlled confirmation booleans false until the submitting +author confirms them for the selected source. As of 26 September 2026, the +user confirmed that the currently listed Delun Gong name, ORCID, and Institute +of Metal Research affiliation are correct. The complete author list, any +coauthor consent and affiliations, funding/sponsor role, contributor list, +related publications, and human review of all AI-assisted work still need to +be confirmed; a current identity confirmation does not complete that review. + +The ledger records successful CI run `36242599844` and official Open Journals +build `36242597087` on candidate commit +`a09ee1b0e48de4bf07898bceda4bec46b70dab43` as a historical snapshot. A final +audit must update the CI record to the selected HEAD; the paper build remains +usable only while its input blobs match that HEAD. The four-page PDF passed artifact +identity and rendered-content checks, while author review of all AI-assisted +work remains pending. Following successful JOSS review, create the final tag +and software archive DOI and enter them for the `publication` stage. diff --git a/docs/evidence/impact_evidence.json b/docs/evidence/impact_evidence.json index 8197490..29035ad 100644 --- a/docs/evidence/impact_evidence.json +++ b/docs/evidence/impact_evidence.json @@ -3,12 +3,157 @@ "public_repository": { "url": "https://github.com/D-sudoasd/DiffractScout", "public_since": "2026-08-12", - "note": "GitHub reports that the public repository was created on 2026-08-12. This date starts the project-side public-development clock." + "note": "GitHub repository metadata checked 2026-09-26 reports creation at 2026-08-12T10:02:29Z. This timestamp starts the public-development clock; private history before it is excluded." }, "archived_releases": [], - "public_development_activity": [], + "public_development_activity": [ + { + "date": "2026-08-12", + "title": "Publish the v0.3.0 GitHub release", + "url": "https://github.com/D-sudoasd/DiffractScout/releases/tag/v0.3.0", + "software_version": "v0.3.0", + "claim_supported": "The public repository had a tagged software release at the start of its public-development period.", + "activity_type": "release", + "outcome": "Published GitHub release v0.3.0 at the public repository; this is not an immutable archive DOI." + }, + { + "date": "2026-08-12", + "title": "Repair CI lint and release reruns", + "url": "https://github.com/D-sudoasd/DiffractScout/pull/3", + "software_version": "commit:99f0fa6ef8ea09d0814c4e92f34bebbfde71bb6f", + "claim_supported": "The maintainer publicly corrected CI and release-workflow behavior.", + "activity_type": "pull_request", + "outcome": "Merged CI and release-rerun fixes." + }, + { + "date": "2026-08-12", + "title": "Restore CIF-to-peaks workflow and laboratory exports", + "url": "https://github.com/D-sudoasd/DiffractScout/pull/5", + "software_version": "commit:14ad9f1788296b7637d3ccb1a769591205751995", + "claim_supported": "The maintainer publicly integrated CIF analysis, spreadsheet lab views, figures, and bilingual GUI workflow changes.", + "activity_type": "pull_request", + "outcome": "Merged the CIF2Peaks product-surface workflow, lab Excel views, figures, and bilingual GUI." + }, + { + "date": "2026-08-12", + "title": "Improve GUI layout and long-table Excel analysis", + "url": "https://github.com/D-sudoasd/DiffractScout/pull/6", + "software_version": "commit:fbcddef4f05de9cccebb2ad01ee764679bb45f97", + "claim_supported": "The maintainer publicly improved long-running GUI usability and spreadsheet access to long tables.", + "activity_type": "pull_request", + "outcome": "Merged a scrollable GUI layout and Excel tables for long analyses." + }, + { + "date": "2026-08-12", + "title": "Prevent concurrent quick-export workbook overwrite", + "url": "https://github.com/D-sudoasd/DiffractScout/pull/7", + "software_version": "commit:3eac2470835df8483011ac6ea1c60710cb263a77", + "claim_supported": "The maintainer publicly fixed a quick-export race that could overwrite another workbook.", + "activity_type": "pull_request", + "outcome": "Merged a fix that preserves concurrent quick-export outputs without implicit overwrite." + }, + { + "date": "2026-08-17", + "title": "Harden scientific analysis and desktop workflows", + "url": "https://github.com/D-sudoasd/DiffractScout/pull/8", + "software_version": "commit:0fa5ba5a0bc310824001aca8f7b00dbb64f6c851", + "claim_supported": "The maintainer publicly refined scientific checks and desktop workflows.", + "activity_type": "pull_request", + "outcome": "Merged scientific-analysis and desktop-workflow hardening." + }, + { + "date": "2026-08-20", + "title": "Harden upgrade transactions and scientific guards", + "url": "https://github.com/D-sudoasd/DiffractScout/pull/9", + "software_version": "commit:312afcf5d8f5b830e58fe3362148e3af230f0f77", + "claim_supported": "The maintainer publicly strengthened transactional upgrades, scientific input checks, and desktop workflows.", + "activity_type": "pull_request", + "outcome": "Merged transaction, scientific-guard, and desktop-workflow changes." + }, + { + "date": "2026-08-23", + "title": "Correct scientific units and desktop workflow gaps", + "url": "https://github.com/D-sudoasd/DiffractScout/pull/11", + "software_version": "commit:1ab397da34a3128312d1c846a8df9f0c127e230d", + "claim_supported": "The maintainer publicly corrected unit handling and desktop workflow behavior.", + "activity_type": "pull_request", + "outcome": "Merged scientific unit-handling and desktop-workflow fixes." + }, + { + "date": "2026-08-27", + "title": "Harden scientific parsing and release workflows", + "url": "https://github.com/D-sudoasd/DiffractScout/pull/13", + "software_version": "commit:b1b799c747ffe1021b4a124529559b9d33f60eba", + "claim_supported": "The maintainer publicly strengthened scientific parsing and release checks.", + "activity_type": "pull_request", + "outcome": "Merged scientific-parsing and release-workflow hardening." + }, + { + "date": "2026-09-12", + "title": "Fix checkout quick-export launch and GUI status/form mapping", + "url": "https://github.com/D-sudoasd/DiffractScout/commit/0d56f99e10abd7e32e87c6f0e5344fc29ba3c670", + "software_version": "commit:0d56f99e10abd7e32e87c6f0e5344fc29ba3c670", + "claim_supported": "The latest public main commit at the 2026-09-26 audit corrected quick-export launch and GUI status/form mapping.", + "activity_type": "commit", + "outcome": "Published the quick-export launch and GUI status/form-mapping fixes on main." + }, + { + "date": "2026-09-26", + "title": "Usability, workbook, reusable CLI workflow, and manuscript upgrade (PR #14)", + "url": "https://github.com/D-sudoasd/DiffractScout/pull/14", + "software_version": "0.4.0 candidate (commit a09ee1b0e48de4bf07898bceda4bec46b70dab43)", + "claim_supported": "The maintainer publicly proposed workflow and workbook improvements and a reproducible synthetic reference-engine comparison.", + "activity_type": "pull_request", + "outcome": "Public PR #14 is open with head commit a09ee1b0e48de4bf07898bceda4bec46b70dab43. This records visible proposed work, not a merge, release acceptance, passing full CI, research use, or external-user engagement." + } + ], "research_use_cases": [], - "independent_validations": [], + "independent_validations": [ + { + "date": "2026-09-26", + "title": "Synthetic powder-diffraction geometry and selection-rule comparison with pymatgen", + "url": "https://github.com/D-sudoasd/DiffractScout/blob/a09ee1b0e48de4bf07898bceda4bec46b70dab43/validation_cases/independent_engines/reference_engine_comparison.md#diffraction-peak-positions-and-selection-rules", + "software_version": "0.4.0 candidate (commit a09ee1b0e48de4bf07898bceda4bec46b70dab43)", + "claim_supported": "For four named synthetic CIF fixtures, DiffractScout/Gemmi and pymatgen XRDCalculator agree on the tested d-spacing groups, 2θ positions, and stated selection rules within the reported numerical tolerances.", + "validation_type": "diffraction", + "status": "complete", + "report_path": "validation_cases/independent_engines/reference_engine_comparison.md", + "report_sha256": "1f90eacd54dad5eaa93c66ec33ad2747f76df608b1f1275abedd90f98484c8ab", + "reference_implementation": "pymatgen XRDCalculator", + "reference_version": "2026.5.4", + "tolerance_basis": "Constants in scripts/compare_reference_engines.py are fixed before comparison: d-spacing grouping/matching tolerance 1e-6 Å; every matched group must satisfy |Δd| <= 1e-6 Å and |Δ2θ| <= 1e-5 degrees; analytic selection-rule expectations are read from the hashed benchmark expectations file.", + "result_sha256": "9c70131e03dc65b7a49c3dabddd131aa011f238299680ccf423bc73bee9c119b", + "diffractscout_commit": "a09ee1b0e48de4bf07898bceda4bec46b70dab43", + "input_sha256": "c82f082d2c1ed45c2219713cf972ffce67a7df78d00eb8b953541363c8f0ffdf", + "radiation": "Cu Kα, effective single wavelength 1.5406 Å", + "scan_range": "2θ = 5–120°", + "limitations": "Synthetic CIF fixtures only; this is a software-engine cross-check, not experimental validation, a real-material property claim, research-impact evidence, or external-user adoption. Intensity differences are diagnostic and have no acceptance threshold.", + "notes": "The input_sha256 field identifies the shared synthetic fcc_al.cif fixture. The machine-readable receipt also records separate SHA-256 values for bcc_fe.cif, nacl.cif, and simple_cubic_al.cif, plus the expectations file, software versions, and per-peak results. The immutable GitHub URL points to the report section at the exact public candidate commit." + }, + { + "date": "2026-09-26", + "title": "Synthetic cubic directional Young's-modulus comparison with pymatgen", + "url": "https://github.com/D-sudoasd/DiffractScout/blob/a09ee1b0e48de4bf07898bceda4bec46b70dab43/validation_cases/independent_engines/reference_engine_comparison.md#cubic-directional-youngs-modulus", + "software_version": "0.4.0 candidate (commit a09ee1b0e48de4bf07898bceda4bec46b70dab43)", + "claim_supported": "For one explicitly synthetic cubic stiffness matrix and three declared directions, DiffractScout and an independently contracted pymatgen compliance tensor agree within the reported tolerance.", + "validation_type": "elasticity", + "status": "complete", + "report_path": "validation_cases/independent_engines/reference_engine_comparison.md", + "report_sha256": "1f90eacd54dad5eaa93c66ec33ad2747f76df608b1f1275abedd90f98484c8ab", + "reference_implementation": "pymatgen ElasticTensor compliance-tensor contraction", + "reference_version": "2026.5.4", + "tolerance_basis": "Constants in scripts/compare_reference_engines.py are fixed before comparison: |ΔE| <= max(1e-8 GPa, 1e-10 × |E_pymatgen|) for [100], [110], and [111].", + "result_sha256": "9c70131e03dc65b7a49c3dabddd131aa011f238299680ccf423bc73bee9c119b", + "diffractscout_commit": "a09ee1b0e48de4bf07898bceda4bec46b70dab43", + "input_sha256": "c82f082d2c1ed45c2219713cf972ffce67a7df78d00eb8b953541363c8f0ffdf", + "tensor_source": "Explicit synthetic 6x6 cubic stiffness matrix in validation_cases/independent_engines/reference_engine_comparison.json at elasticity.synthetic_stiffness_GPa", + "tensor_frame": "Conventional cubic Cartesian frame defined by the synthetic Fm-3m fcc_al.cif (a = 4 Å); no coordinate transformation is tested", + "voigt_convention": "Voigt order [11, 22, 33, 23, 13, 12] with engineering shear strain", + "directions": ["[100]", "[110]", "[111]"], + "limitations": "Synthetic stiffness matrix only; this is a numerical convention/implementation cross-check, not validation of an experimental or database tensor, a real-material property claim, research-impact evidence, or external-user adoption.", + "notes": "The input_sha256 identifies the shared synthetic fcc_al.cif fixture. The result JSON records the tensor, per-direction values, versions, and tolerances. The immutable GitHub URL points to the report section at the exact public candidate commit." + } + ], "external_engagement": [], "publications_or_preprints": [], "presentations_or_training": [], @@ -19,14 +164,14 @@ "contributors_confirmed": false, "related_publications_confirmed": false, "human_review_confirmed": false, - "remote_ci_confirmed": false, - "remote_ci_url": "", - "remote_ci_commit": "", - "official_joss_build_confirmed": false, - "official_joss_build_url": "", - "official_joss_build_commit": "", - "paper_source_sha256": "", - "paper_pdf_sha256": "", - "note": "Set each field to true only after the submitting author completes that review." + "remote_ci_confirmed": true, + "remote_ci_url": "https://github.com/D-sudoasd/DiffractScout/actions/runs/36242599844", + "remote_ci_commit": "a09ee1b0e48de4bf07898bceda4bec46b70dab43", + "official_joss_build_confirmed": true, + "official_joss_build_url": "https://github.com/D-sudoasd/DiffractScout/actions/runs/36242597087", + "official_joss_build_commit": "a09ee1b0e48de4bf07898bceda4bec46b70dab43", + "paper_source_sha256": "3a78d172508cb1db87f1bc28d5732748a84c0f3a2427f105cac00f2b0529e675", + "paper_pdf_sha256": "2acee1fd0d014f22fbd219bd0035a11c4ea453c051bfcbd82532ce16f0ac2386", + "note": "As of 2026-09-26, the user confirmed the currently listed Delun Gong name, ORCID, and Institute of Metal Research affiliation. CI run 36242599844 and official JOSS PDF run 36242597087 succeeded on commit a09ee1b0e48de4bf07898bceda4bec46b70dab43. The generated four-page PDF was checked for source/artifact identity, rendered layout, references, and text layer. This does not substitute for author review of all AI-assisted contributions. Keep author-list completeness, coauthor consent, full affiliation, funding and sponsor role, contributor, related-publication, and human-review confirmations pending until the submitting author confirms them for the selected source." } } diff --git a/paper/README.md b/paper/README.md index ebce5dc..fb2d499 100644 --- a/paper/README.md +++ b/paper/README.md @@ -1,19 +1,38 @@ # JOSS paper sources -- `paper.md`: manuscript source using current JOSS section headings; -- `paper.bib`: bibliography; -- `fig_workflow.*`: workflow figure in raster and editable vector formats; -- `fig_validation.*`: executable offline-validation figure in raster and editable vector formats; -- `make_figures.py`: deterministic figure-generation source; -- `paper.pdf`: locally reviewed layout baseline; after `paper.md`, `paper.bib`, - or either figure changes, it is stale until rebuilt. The Open Journals action - remains the authoritative JOSS build. +This directory contains the JOSS manuscript and its reproducible figures: + +- **paper.md**: Markdown manuscript with the required JOSS sections and YAML metadata. +- **paper.bib**: cited crystallography, software, data-service, and source-project records. +- **fig_workflow.png** and **fig_validation.png**: manuscript figures; their editable SVGs are retained beside them. +- **make_figures.py**: deterministic source for the two figures. +- **paper.pdf**: a generated preview only. It is stale after any manuscript, bibliography, or figure change. The Open Journals draft build is authoritative. + +The current four-page A4 preview was generated by the [Open Journals build on 26 September 2026](https://github.com/D-sudoasd/DiffractScout/actions/runs/36242597087) from [commit a09ee1b](https://github.com/D-sudoasd/DiffractScout/commit/a09ee1b0e48de4bf07898bceda4bec46b70dab43). Its source files and PDF bytes were checked against the workflow artifact, and all four rendered pages and the extracted text were inspected. The DRAFT watermark and placeholder journal DOI, volume, and page fields belong to the official preview template. + +The manuscript describes a bounded workflow contribution: linking candidate records, source CIFs, calculation settings, diagnostic states, theoretical references, and their exported result bundle. It does not claim a new diffraction or elasticity algorithm. The developer reports prior use in several published materials-science papers; those records and their exact software-version links have not yet been provided for citation-level traceability. + +## Submission inputs still required + +Complete these items before submitting to JOSS: + +1. **Public development history.** The repository became public at 10:02:29 UTC on 12 August 2026. The current JOSS guidance asks for more than six months of public development history when a repository is recent, with sustained development and supporting release, issue, or pull-request evidence. On the manuscript date of 26 September 2026, this period is not yet met; submission should be after 10:02:29 UTC on 12 February 2027, and 15 February 2027 is a practical earliest date if development and the other criteria remain adequate. +2. **Published-use evidence.** Provide representative paper titles and DOI or stable links, identify where DiffractScout was used in each study, and give the software version or commit where possible. The manuscript currently attributes this use to the developer's report and does not cite unidentified papers. +3. **Funding and acknowledgements.** Provide any funding body and grant identifiers and state whether funders influenced software design, data collection, analysis, or publication. If there was no external financial support, confirm that explicitly. Add any additional acknowledgements that belong in the manuscript. +4. **Author list.** Delun Gong's name, ORCID, and affiliation were confirmed for this draft. Confirm that the author list is complete and that all listed authors agree to authorship. +5. **Human review of AI-assisted work.** Earlier repository work and this revision used OpenAI GPT-5.6 Pro, OpenAI Codex GPT-6, and cooperating assistant agents across software, tests, figures, documentation, and manuscript work. Automated checks are described in the manuscript, but no assertion that the human author has completed review is made. The responsible author must review, edit, and validate every AI-assisted contribution and make the core design decisions before the JOSS AI disclosure can assert compliance. +6. **Submission date.** Update the JOSS metadata date to the actual submission date if the submission happens after this draft. +7. **Archived software citation.** JOSS requests a tagged, archived software release with a DOI after successful review. At that stage, replace or supplement the source-repository citation in paper.bib with the DOI for the reviewed release; do not invent a DOI beforehand. + +The independent analytic benchmark and synthetic fixtures are software checks. They are not experimental validation and should not be presented as research-use evidence. + +## Figures Regenerate figures from the current package: -```bash -python -m pip install -e ".[paper]" -SOURCE_DATE_EPOCH=1786492800 python paper/make_figures.py -``` + python -m pip install -e ".[paper]" + SOURCE_DATE_EPOCH=1786492800 python paper/make_figures.py + +## Paper build -Build the authoritative draft with `.github/workflows/draft-pdf.yml` or Docker through `scripts/build_paper.sh`. Render every page to PNG and inspect it before release. +Build the draft with .github/workflows/draft-pdf.yml or Docker through scripts/build_paper.sh. Render and inspect every page after the paper source is frozen. The checked-in paper.pdf is not updated as part of a manuscript-only revision. diff --git a/paper/paper.bib b/paper/paper.bib index d38300f..c7ab1dd 100644 --- a/paper/paper.bib +++ b/paper/paper.bib @@ -98,9 +98,29 @@ @online{materialsproject_elasticity author = {{Materials Project}}, title = {Elastic Constants}, url = {https://docs.materialsproject.org/methodology/materials-methodology/elasticity}, - urldate = {2026-07-30} + urldate = {2026-09-26} } -% After successful JOSS review, add the DOI for the final archived software -% release requested by the editor. Do not invent it before the archive exists, -% and do not confuse it with the later JOSS article DOI. +@software{diffractscout_software, + author = {Gong, Delun}, + title = {{DiffractScout}}, + version = {0.4.0}, + year = {2026}, + url = {https://github.com/D-sudoasd/DiffractScout} +} + +@software{phasescout_2026, + author = {{D-sudoasd}}, + title = {{PhaseScout}}, + version = {source snapshot 10e4cd4a7ae1317b4368ae4cce905e643d471584}, + year = {2026}, + url = {https://github.com/D-sudoasd/PhaseScout/tree/10e4cd4a7ae1317b4368ae4cce905e643d471584} +} + +@software{cif2peaks_2026, + author = {{D-sudoasd}}, + title = {{CIF2Peaks}}, + version = {source snapshot 36e7e31419fa7ba2752e3e1bdf82e42b400e3b83}, + year = {2026}, + url = {https://github.com/D-sudoasd/CIF2Peaks/tree/36e7e31419fa7ba2752e3e1bdf82e42b400e3b83} +} diff --git a/paper/paper.md b/paper/paper.md index d143e01..3fe2781 100644 --- a/paper/paper.md +++ b/paper/paper.md @@ -1,12 +1,12 @@ --- -title: 'DiffractScout: An auditable workflow from candidate phases to indexed powder diffraction and elasticity references' +title: 'DiffractScout: An auditable Python workflow for generating powder diffraction references' tags: - Python - materials science - - powder diffraction + - powder X-ray diffraction - crystallography - phase screening - - elasticity + - research software authors: - name: Delun Gong orcid: 0000-0001-7877-7707 @@ -15,60 +15,54 @@ authors: affiliations: - name: Institute of Metal Research, Chinese Academy of Sciences, Shenyang 110016, China index: 1 -date: 12 August 2026 +date: 26 September 2026 bibliography: paper.bib --- # Summary -Materials researchers often need a trustworthy set of calculated powder-diffraction peaks before they can plan measurements or interpret an unfamiliar pattern. `DiffractScout` makes that preparatory workflow reproducible. From an alloy name, element set, database identifier, or local crystal-structure file, it can assemble candidate structures, check that they are usable, calculate indexed theoretical reflections, and, when compatible elastic data exist, attach the Young's modulus normal to each reflecting plane. Instead of returning an isolated spreadsheet, the software records the exact structures, calculation settings, data sources, warnings, software versions, and file hashes behind every result. It packages these records with tables, diagnostics, plots, and a verifiable manifest. `DiffractScout` is available as a command-line tool, Python API, and desktop application, with a deterministic offline example that reviewers can run without database credentials. It builds auditable theoretical references; it does not identify phases automatically or refine experimental diffraction data. +Researchers compare measured powder X-ray diffraction (PXRD) patterns with reflections calculated from candidate crystal structures. Preparing these references requires the structure setting, radiation, and calculation parameters to remain linked. DiffractScout is an open-source Python package that converts composition queries or local Crystallographic Information Files (CIFs) into indexed theoretical reflections and display profiles [@diffractscout_software]. Its command-line, Python, and desktop interfaces write source structures, settings, data, diagnostics, and tables to a result bundle with a verifiable manifest. The formatted workbook presents an overview and can be previewed before saving. DiffractScout supports experimental planning and interpretation; it does not identify phases in measured patterns or refine experimental data. # Statement of need -Candidate-phase screening in alloy and diffraction research commonly spans several independent tools. A researcher may translate an engineering alloy name into an element set, enumerate chemical subsystems, query a computed-materials database, download CIF records, inspect stability and symmetry metadata, calculate theoretical peaks, locate an elastic tensor, and then transfer selected values into a spreadsheet. Each individual operation is well supported, yet the hand-offs can separate a result from the structure setting, query conditions, radiation definition, tensor basis, warnings, or software version that produced it. This creates practical risks: duplicated candidates, ambiguous CIF-to-tensor pairing, coordinate-frame errors in directional properties, service failures recorded as missing data, and theoretical intensities compared with experimental peak areas under incompatible corrections. +Powder-diffraction screening links candidate compositions and structures to expected reflection positions and intensities. Preparing references spans alloy parsing, database queries, CIF validation, diffraction calculations, optional tensor matching, and spreadsheet transfer. Across separate tools, candidate identifiers, structure settings, radiation, warnings, and calculation parameters can become detached from the final table. -`DiffractScout` addresses this workflow-level problem for materials researchers, beamline users, and students who need auditable theoretical references before experimental phase identification or refinement. The package preserves CIF identity [@hall1991], labels Materials Project structures and elastic tensors as computed records [@jain2013], and carries missing or unusable quantities forward as explicit states. Its scientific scope is deliberately bounded. Database candidates are hypotheses for evaluation, not experimentally confirmed phases. The diffraction output is a kinematic theoretical reference and does not perform Rietveld, Le Bail, or Pawley refinement, quantitative phase analysis, instrument calibration, background modelling, or preferred-orientation correction. This boundary allows the software to support experimental planning and interpretation without presenting a reference-generation workflow as an experimental inverse solution. +DiffractScout connects these steps for materials researchers and students preparing references before or alongside experiments. Runs start from a composition, provider ID, or local CIF; the software records source structures and data interpretation, then exports machine-readable tables and a reviewable workbook in one bundle. Local-CIF processing needs no database credentials. This workflow makes theoretical references repeatable while leaving experimental interpretation to the researcher. # State of the field -The Materials Project provides computed structures and materials properties [@jain2013], and pymatgen supplies general materials-analysis objects and diffraction utilities [@ong2013]. Gemmi provides CIF handling, crystallographic symmetry operations, and structure-factor calculations [@wojdyr2022], while spglib provides an independent symmetry-search implementation [@togo2024]. `Dans_Diffraction` offers broad crystal-property and diffraction simulation capabilities from CIF files [@porter2023]. Elasticipy provides general linear-elasticity and tensor analysis, including directional properties [@depriester2025]. Experimental diffraction reduction and refinement are addressed by packages such as GSAS-II [@toby2013], and pyFAI specializes in detector geometry and azimuthal integration [@ashiotis2015]. +The Materials Project provides computed structures and properties [@jain2013], and pymatgen supplies materials-analysis objects and calculations [@ong2013]. Gemmi parses CIFs and performs crystallographic operations and structure-factor calculations [@wojdyr2022]; spglib offers an independent symmetry search [@togo2024]. Dans_Diffraction simulates diffraction from supplied structures [@porter2023], and Elasticipy analyzes linear elasticity and tensors [@depriester2025]. For measured data, GSAS-II supports reduction and refinement [@toby2013], while pyFAI provides detector calibration and azimuthal integration [@ashiotis2015]. -`DiffractScout` uses these projects as data sources, numerical foundations, comparison tools, or downstream environments. Its contribution is the cross-stage contract that joins composition interpretation, subsystem queries, candidate deduplication, source acquisition, CIF identity, tensor pairing, theoretical reflection calculation, diagnostics, and export in one versioned research object. Direct diffraction simulators generally begin with a structure already selected by the user. Elasticity packages generally begin with a tensor whose coordinate basis is already known. Database clients return records without defining how a later diffraction table, directional modulus, and exported spreadsheet remain bound to the same cell setting. Refinement suites solve a different experimental problem. - -General crystallographic, symmetry, database-client, and tensor algorithms remain in their established upstream projects. `DiffractScout` implements the workflow semantics that span those projects: fail-closed property pairing, explicit data states, stable output schemas, bounded numerical work, and non-destructive bundle creation. This separation provides a clear build-versus-contribute rationale while retaining upstream libraries for the domain algorithms they already implement well. +DiffractScout introduces no new diffraction or elasticity algorithm. Instead, it adds an application layer that links database or local-CIF inputs, candidate identities, user-selected settings, calculations, diagnostics, and exports in one result bundle—relations that extend beyond any one numerical library. It restructures workflow components from PhaseScout and CIF2Peaks [@phasescout_2026; @cif2peaks_2026] around shared records and provenance. Database access is optional, so local-CIF analysis runs offline. # Software design -The package separates source providers, typed domain records, numerical calculation, orchestration, and export (Figure 1). A provider protocol returns normalized candidate records and source artifacts. The optional Materials Project adapter records query context and requests conventional-standard unit cells. Local CIF analysis remains available without a database client or network connection. This architecture keeps the offline scientific core deterministic and allows additional structure providers to be added without changing the diffraction engine. - -![DiffractScout converts a composition request or local CIF collection into a verified evidence bundle. Each stage preserves identity, source, assumptions, warnings, and missing-data states.](fig_workflow.png){#fig:workflow width="100%"} +A provider supplies normalized candidate records and their source artifacts. The Materials Project adapter records query context and source identifiers; the local-CIF path requires no network service. Candidates are deduplicated before download. Each structure record preserves the CIF hash, selected data block, cell, space-group resolution, occupancy state, and diagnostics. Conflicting space-group declarations and partial occupancies remain visible for review rather than being silently repaired [@hall1991]. -CIF validation requires finite positive cell geometry and atomic sites. The software records whether a space-group assignment came from an explicit symbol, an International Tables number, Gemmi inference, or a warned fallback; spglib can provide an independent cross-check. Gemmi expands unit-cell sites, evaluates systematic absences, enumerates unique Miller-index families, and calculates X-ray structure factors. A dedicated structure-factor copy receives Gemmi's crystallographic-occupancy conversion so that special-position multiplicity is not counted twice. The indexed table reports $hkl$, $d$, $2\theta$, $q=2\pi/d$, multiplicity, $|F|^2$, and separate intensity channels with and without the explicitly defined Lorentz-polarization factor. A pseudo-Voigt profile is generated for visualization only. Profile-grid size and a conservative reciprocal-space search estimate are checked before large allocations. +The diffraction engine uses Gemmi to enumerate symmetry-related reflection families, apply systematic-absence rules, and calculate X-ray structure factors. Results include representative Miller indices, interplanar spacing, scattering-vector magnitude, multiplicity, and separately identified intensity channels with and without the stated Lorentz–polarization correction. A Gaussian, Lorentzian, or pseudo-Voigt profile is generated for display; it is not an instrument-response fit. Configured limits on profile-grid points and reciprocal-space candidate counts are checked before allocation. -Elasticity records contain a finite $6\times6$ stiffness matrix in GPa, source identity, nature-of-data label, and coordinate-frame declaration. Matrices are checked for symmetry, invertibility, conditioning, and positive definiteness. Under the engineering-shear Voigt convention, the directional modulus is +Directional Young's moduli can be attached when an elastic tensor is uniquely associated with the CIF and its coordinate frame is usable. The Materials Project's elastic constants are computed data, not experimental measurements [@materialsproject_elasticity]. DiffractScout validates the matrix, unit, conditioning, and positive definiteness before evaluation along a reflection-plane normal. When a downloaded tensor lacks a verified transformation into the CIF Cartesian frame, the software retains its provenance and emits a frame-transformation status without a directional value. This fail-closed policy avoids making a numerically plausible but unsupported tensor-to-structure pairing. -$$ -E(n)=\left[q(n)^T S q(n)\right]^{-1}, \qquad S=C^{-1}, -$$ +The command-line and desktop interfaces call the same analysis pipeline. The command-line interface accepts reusable parameter presets and can inspect and verify an existing result bundle without rerunning the analysis. The desktop application offers separate result preview, workbook saving, and result-folder actions; versioned parameter presets exclude credentials and file paths. The Excel exporter formats existing result values rather than recalculating them. It adds an overview, readable number formats, filters, frozen phase-identity columns, and grouped technical fields while preserving the corresponding tabular records. CSV and JSON outputs remain available for scripted use. -where $n$ is the reciprocal-lattice normal of the reflection. Automatic sidecar pairing requires a unique CIF or material identifier. For Materials Project downloads, directional coupling uses the raw/POSCAR-format tensor paired with the conventional-standard CIF [@materialsproject_elasticity]. IEEE-only records are retained with a `frame_transform_required` state and produce no directional modulus until a verified rotation is supplied. +Each run is first written to a staging directory. Before publication to the requested output path, a verifier checks the manifest's SHA-256 digests and file sizes, path safety, symbolic links, and unlisted files. Replacing an existing result requires explicit authorization and verification of that result. This design makes the completed directory a portable evidence bundle while keeping recovery behavior explicit. -The output directory is treated as a verifiable research bundle. A run writes into a sibling staging directory, generates all tables and metadata, creates a manifest, and verifies file hashes, byte sizes, safe paths, symbolic-link absence, and unlisted files before moving the bundle into place. Replacement of an existing bundle requires explicit authorization and a successful integrity check of the current bundle. A failed export or verification leaves the previous valid result untouched. Structured diagnostics preserve phase-specific failures while allowing independent phases to complete. +An offline analytic benchmark tests simple-cubic, body-centered cubic, face-centered cubic, and NaCl selection rules, reflection geometry, structure factors, and cubic elasticity against closed-form expectations. A separate pymatgen XRDCalculator comparison [@ong2013] matched 40 d-spacing groups from four synthetic CIFs using Cu Kα (1.5406 Å) over 5–120° 2θ. Every group met the specified spacing ($10^{-6}$ Å) and angle ($10^{-5}\,^{\circ}$) tolerances, and all tested allowed and forbidden reflection rules passed. Directional moduli along [100], [110], and [111] for a synthetic cubic tensor passed the defined absolute and relative tolerances in the pymatgen compliance-tensor comparison [@ong2013]. The automated suite checks analysis and export contracts; continuous integration runs it on Linux for Python 3.10–3.13 and selected checks on Windows and macOS. These comparisons are reproducible without a database service. -# Research impact statement +![DiffractScout workflow from candidate request or local CIFs to a verified result bundle. Source identity, assumptions, diagnostics, and missing-data states are carried across the pipeline.](fig_workflow.png){#fig:workflow width="95%"} -The repository provides executable evidence for the package's current scholarly significance and reviewer readiness. A packaged analytic suite performs 45 checks using simple-cubic, BCC, FCC, NaCl, and cubic-elasticity fixtures with closed-form selection rules, multiplicities, plane spacings, lattice structure factors, and directional moduli. The complete offline workflow uses a self-identifying synthetic $Fm\bar{3}m$ cell with $a=4$ Å and an isotropic cubic test tensor. For Cu K$\alpha$ radiation over $5$--$120^\circ$, it produces eight allowed reflection families, excludes the forbidden FCC $(100)$ and $(110)$ families, verifies $d$ spacings, multiplicities, $q=2\pi/d$, and $|F_{111}|^2=(4f_{\mathrm{Al}})^2$, and returns $E(hkl)=110$ GPa for every direction. It writes a 5751-point profile and a 12-artifact bundle protected by independent SHA-256 entries (Figure 2). The fixtures are explicitly labelled as synthetic and carry no experimental material-property claim. +![Analytic offline benchmark for the synthetic FCC fixture. The figure shows allowed reflection families and the selected geometry and bundle-integrity checks. The fixture is for software verification, not a measured material.](fig_validation.png){#fig:validation width="95%"} -![Offline verification output distributed with DiffractScout. (a) The indexed Cu K$\alpha$ powder profile for the synthetic FCC fixture, with all eight allowed reflection families labelled. (b) Selected invariants and bundle-integrity results checked by the automated suite.](fig_validation.png){#fig:validation width="100%"} +# Research impact statement -The pytest suite covers composition parsing, candidate selection, CIF and tensor validation, systematic absences, structure factors, intensity definitions, coordinate-frame boundaries, resource limits, transactional replacement, spreadsheet safety, and manifest tamper detection; its collected-test count is recorded by continuous integration rather than copied into the paper. CI targets Python 3.10--3.13, Windows, macOS, a Linux graphical-interface smoke test, wheel installation, and Open Journals draft compilation. These materials let reviewers reproduce the declared numerical and provenance contracts without database credentials. They do not establish experimental accuracy, realized research impact, or external adoption. The project therefore requires a public multiphase research-use case, independent diffraction and elasticity comparisons, and external installation or use evidence before submission. The final Research impact statement will include only claims indexed by the machine-readable public evidence ledger. +The developer reports using DiffractScout in published materials-science research. Each result bundle preserves candidate identity, source structures, calculation settings, diagnostics, and file checksums, making calculated references traceable. # AI usage disclosure -OpenAI GPT-5.6 Pro assisted integration planning, test scaffolding, documentation, figure production, and manuscript editing. The human author directed the scientific scope and retains responsibility for the architecture, numerical definitions, licensing, references, and manuscript claims. AI-assisted outputs were checked through source inspection, cited primary documentation, analytic fixtures, automated tests, bundle-integrity verification, and rendered-PDF review. Final submission requires the author's explicit confirmation of complete human review. +OpenAI GPT-5.6 Pro assisted earlier repository development, testing, documentation, figure production, and manuscript drafting from 30 July to 12 August 2026. In this revision, OpenAI Codex GPT-6 and cooperating assistant agents supported code changes, test scaffolding and review, documentation, citation discovery, manuscript restructuring, and automated verification. Software claims were checked against project source and scientific contracts, references against publisher or upstream records, and automated tests and the analytic benchmark were run. Human author review of the AI-assisted contributions remains pending. # Acknowledgements -The author acknowledges the maintainers and contributors of the Materials Project, pymatgen, Gemmi, spglib, GSAS-II, pyFAI, Dans_Diffraction, and Elasticipy projects, whose software and documentation support this workflow. +The author acknowledges the developers and maintainers of the open-source software and data services cited above. # References diff --git a/paper/paper.pdf b/paper/paper.pdf index c212fbd..6aad116 100644 Binary files a/paper/paper.pdf and b/paper/paper.pdf differ diff --git a/scripts/check_release.py b/scripts/check_release.py index afd7374..4dd0600 100755 --- a/scripts/check_release.py +++ b/scripts/check_release.py @@ -243,6 +243,7 @@ def run_installed(name: str, *args: str) -> None: run_installed("diffractscout", "--version") run_installed("diffractscout", "demo", "-o", str(demo), "--no-excel") run_installed("diffractscout", "verify", str(demo)) + run_installed("diffractscout", "inspect", str(demo), "--json") run_installed("diffractscout", "benchmark", "-o", str(benchmark)) packaged_cif = ( "import diffractscout,pathlib; " @@ -258,15 +259,19 @@ def run_installed(name: str, *args: str) -> None: ).stdout.strip() if not Path(located).is_file(): raise SystemExit(f"Installed benchmark CIF is missing: {located}") - run_installed("diffractscout-quick-export", located, "-o", str(quick)) + preset = root / "analysis-preset.json" + run_installed("diffractscout", "preset", "save", "-o", str(preset), "--no-patterns") + run_installed("diffractscout", "preset", "show", str(preset), "--json") + run_installed("diffractscout-quick-export", located, "-o", str(quick), "--preset", str(preset)) run_installed("diffractscout", "verify", str(root / "quick_bundle")) + run_installed("diffractscout", "inspect", str(root / "quick_bundle"), "--json") return { "mode": "system-site-packages" if reuse_dependencies else "isolated-dependencies", "package": str(wheel.resolve()), "source_tree_import": "rejected", "commands": ( "pip-check,entrypoint-launchers,entrypoint-metadata,installed-entrypoints," - "version,demo,verify,benchmark,quick-export,verify" + "version,demo,verify,inspect,benchmark,preset-save,preset-show,quick-export,verify,inspect" ), } diff --git a/scripts/compare_reference_engines.py b/scripts/compare_reference_engines.py new file mode 100644 index 0000000..73f53f8 --- /dev/null +++ b/scripts/compare_reference_engines.py @@ -0,0 +1,803 @@ +#!/usr/bin/env python3 +"""Compare synthetic diffraction and cubic-elasticity fixtures with pymatgen. + +This is an optional, offline validation tool. It does not participate in the +DiffractScout analysis path and does not modify production calculations. +""" + +from __future__ import annotations + +import argparse +from collections.abc import Iterable, Mapping, Sequence +from datetime import datetime, timezone +import hashlib +import importlib.metadata +import json +import math +from pathlib import Path +import sys +import warnings +from typing import Any + +import numpy as np + + +REPOSITORY_ROOT = Path(__file__).resolve().parents[1] +BENCHMARK_ROOT = REPOSITORY_ROOT / "src" / "diffractscout" / "benchmark_data" +DEFAULT_OUTPUT = REPOSITORY_ROOT / "validation_cases" / "independent_engines" +EXPECTATIONS_SCHEMA = "diffractscout_analytic_expectations_v1" +REQUIRED_BENCHMARK_CASES = { + "simple_cubic_al": "simple_cubic_al.cif", + "bcc_fe": "bcc_fe.cif", + "fcc_al": "fcc_al.cif", + "nacl": "nacl.cif", +} +FORBIDDEN_FAMILY_COUNTS = {"bcc_fe": 2, "fcc_al": 2, "nacl": 2} +WAVELENGTH_A = 1.5406 +TWO_THETA_RANGE_DEG = (5.0, 120.0) +D_SPACING_TOLERANCE_A = 1e-6 +TWO_THETA_TOLERANCE_DEG = 1e-5 +ELASTIC_ABSOLUTE_TOLERANCE_GPA = 1e-8 +ELASTIC_RELATIVE_TOLERANCE = 1e-10 +CUBIC_STIFFNESS_GPA = ( + (250.0, 150.0, 150.0, 0.0, 0.0, 0.0), + (150.0, 250.0, 150.0, 0.0, 0.0, 0.0), + (150.0, 150.0, 250.0, 0.0, 0.0, 0.0), + (0.0, 0.0, 0.0, 100.0, 0.0, 0.0), + (0.0, 0.0, 0.0, 0.0, 100.0, 0.0), + (0.0, 0.0, 0.0, 0.0, 0.0, 100.0), +) +CUBIC_DIRECTIONS = ((1, 0, 0), (1, 1, 0), (1, 1, 1)) + + +def sha256_file(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as stream: + for block in iter(lambda: stream.read(1024 * 1024), b""): + digest.update(block) + return digest.hexdigest() + + +def canonical_cubic_family(hkl: Iterable[int]) -> tuple[int, int, int]: + """Return a sign/permutation-independent cubic-family key.""" + + values = tuple(abs(int(value)) for value in hkl) + if len(values) != 3: + raise ValueError("A Miller index must have exactly three components.") + return tuple(sorted(values, reverse=True)) # type: ignore[return-value] + + +def _validated_hkl_list( + case_name: str, + field_name: str, + value: Any, + *, + expected_count: int, +) -> set[tuple[int, int, int]]: + if not isinstance(value, list) or len(value) != expected_count: + raise ValueError( + f"Benchmark case {case_name!r} must define exactly {expected_count} " + f"{field_name} Miller-index triplets." + ) + families: set[tuple[int, int, int]] = set() + for index, item in enumerate(value): + if ( + not isinstance(item, list) + or len(item) != 3 + or any(type(component) is not int for component in item) + or item == [0, 0, 0] + ): + raise ValueError( + f"Benchmark case {case_name!r} has an invalid {field_name} entry at index {index}." + ) + family = canonical_cubic_family(item) + if family in families: + raise ValueError( + f"Benchmark case {case_name!r} repeats a cubic family in {field_name}." + ) + families.add(family) + return families + + +def _validate_expectations(expectations: Any) -> Mapping[str, Mapping[str, Any]]: + """Reject missing or malformed packaged fixtures before any comparisons run.""" + + if not isinstance(expectations, Mapping) or expectations.get("schema") != EXPECTATIONS_SCHEMA: + raise ValueError("Unsupported analytic benchmark expectation schema.") + cases = expectations.get("cases") + if not isinstance(cases, Mapping): + raise ValueError("Analytic benchmark expectations must contain a cases object.") + expected_names = set(REQUIRED_BENCHMARK_CASES) + observed_names = set(cases) + if observed_names != expected_names: + missing = sorted(expected_names - observed_names) + extra = sorted(observed_names - expected_names) + raise ValueError( + "Independent engine comparison requires exactly the packaged synthetic cases " + f"{sorted(expected_names)}; missing={missing}, extra={extra}." + ) + + for case_name, expected_file in REQUIRED_BENCHMARK_CASES.items(): + specification = cases[case_name] + if not isinstance(specification, Mapping): + raise ValueError(f"Benchmark case {case_name!r} must be an object.") + required_fields = {"file", "lattice_a_A", "first_allowed", "multiplicity"} + if case_name in FORBIDDEN_FAMILY_COUNTS: + required_fields.add("forbidden") + missing_fields = sorted(required_fields - set(specification)) + if missing_fields: + raise ValueError( + f"Benchmark case {case_name!r} is missing required fields: {missing_fields}." + ) + if specification["file"] != expected_file: + raise ValueError( + f"Benchmark case {case_name!r} must use fixture {expected_file!r}." + ) + lattice_a = specification["lattice_a_A"] + if ( + isinstance(lattice_a, bool) + or not isinstance(lattice_a, (int, float)) + or not math.isfinite(float(lattice_a)) + or float(lattice_a) <= 0.0 + ): + raise ValueError(f"Benchmark case {case_name!r} has an invalid lattice_a_A.") + allowed = _validated_hkl_list( + case_name, + "first_allowed", + specification["first_allowed"], + expected_count=3, + ) + if case_name in FORBIDDEN_FAMILY_COUNTS: + forbidden = _validated_hkl_list( + case_name, + "forbidden", + specification["forbidden"], + expected_count=FORBIDDEN_FAMILY_COUNTS[case_name], + ) + if allowed & forbidden: + raise ValueError( + f"Benchmark case {case_name!r} lists a cubic family as both allowed and forbidden." + ) + elif "forbidden" in specification: + raise ValueError( + "The simple-cubic benchmark case must omit the forbidden field." + ) + + multiplicities = specification["multiplicity"] + if not isinstance(multiplicities, Mapping) or not multiplicities: + raise ValueError( + f"Benchmark case {case_name!r} must define a non-empty multiplicity object." + ) + multiplicity_families: set[tuple[int, int, int]] = set() + for key, multiplicity in multiplicities.items(): + if not isinstance(key, str): + raise ValueError(f"Benchmark case {case_name!r} has a non-string multiplicity key.") + try: + components = [int(part) for part in key.split()] + except ValueError as exc: + raise ValueError( + f"Benchmark case {case_name!r} has an invalid multiplicity key {key!r}." + ) from exc + if ( + len(components) != 3 + or key.split() != [str(component) for component in components] + or components == [0, 0, 0] + or isinstance(multiplicity, bool) + or not isinstance(multiplicity, int) + or multiplicity <= 0 + ): + raise ValueError( + f"Benchmark case {case_name!r} has an invalid multiplicity entry {key!r}." + ) + family = canonical_cubic_family(components) + if family in multiplicity_families: + raise ValueError( + f"Benchmark case {case_name!r} repeats a cubic family in multiplicity." + ) + multiplicity_families.add(family) + return cases + + +def _group_by_d_spacing( + peaks: Sequence[Mapping[str, Any]], + *, + tolerance_A: float, +) -> list[dict[str, Any]]: + """Merge coincident line families using a fixed-width d-spacing window. + + Groups are anchored to their first (largest-d) peak rather than chained + through successive neighbors, so the total width of any group cannot + exceed ``tolerance_A``. + """ + + if not math.isfinite(tolerance_A) or tolerance_A < 0: + raise ValueError("d-spacing tolerance must be a finite non-negative value.") + ordered = sorted(peaks, key=lambda peak: float(peak["d_A"]), reverse=True) + groups: list[list[Mapping[str, Any]]] = [] + for peak in ordered: + d_value = float(peak["d_A"]) + if not math.isfinite(d_value) or d_value <= 0: + raise ValueError("Peak d-spacing values must be finite and positive.") + if not groups or abs(d_value - float(groups[-1][0]["d_A"])) > tolerance_A: + groups.append([peak]) + else: + groups[-1].append(peak) + + merged: list[dict[str, Any]] = [] + for group in groups: + d_values = [float(peak["d_A"]) for peak in group] + theta_values = [float(peak["two_theta_deg"]) for peak in group] + hkls = [tuple(int(component) for component in hkl) for peak in group for hkl in peak["hkls"]] + merged.append( + { + "d_A": float(sum(d_values) / len(d_values)), + "two_theta_deg": float(sum(theta_values) / len(theta_values)), + "intensity": float(sum(float(peak["intensity"]) for peak in group)), + "hkls": [list(hkl) for hkl in sorted(set(hkls))], + "family_count": sum(int(peak.get("family_count", 1)) for peak in group), + } + ) + return merged + + +def match_d_spacing_groups( + left: Sequence[Mapping[str, Any]], + right: Sequence[Mapping[str, Any]], + *, + tolerance_A: float, +) -> tuple[list[tuple[int, int]], list[int], list[int]]: + """Match two d-sorted peak-group lists one-to-one, without using hkl labels.""" + + if not math.isfinite(tolerance_A) or tolerance_A < 0: + raise ValueError("d-spacing tolerance must be a finite non-negative value.") + left_order = sorted(range(len(left)), key=lambda index: float(left[index]["d_A"]), reverse=True) + right_order = sorted(range(len(right)), key=lambda index: float(right[index]["d_A"]), reverse=True) + matches: list[tuple[int, int]] = [] + unmatched_left: list[int] = [] + unmatched_right: list[int] = [] + li = 0 + ri = 0 + while li < len(left_order) and ri < len(right_order): + left_index = left_order[li] + right_index = right_order[ri] + delta = float(left[left_index]["d_A"]) - float(right[right_index]["d_A"]) + if abs(delta) <= tolerance_A: + matches.append((left_index, right_index)) + li += 1 + ri += 1 + elif delta > 0: + unmatched_left.append(left_index) + li += 1 + else: + unmatched_right.append(right_index) + ri += 1 + unmatched_left.extend(left_order[li:]) + unmatched_right.extend(right_order[ri:]) + return matches, unmatched_left, unmatched_right + + +def _two_theta_from_d(d_A: float, wavelength_A: float) -> float: + argument = wavelength_A / (2.0 * d_A) + if argument < 0 or argument > 1: + raise ValueError(f"d-spacing {d_A} A is inaccessible at {wavelength_A} A.") + return float(np.degrees(2.0 * np.arcsin(argument))) + + +def _selection_rule(case_name: str, hkl: tuple[int, int, int]) -> bool: + first, second, third = (abs(int(value)) for value in hkl) + if first == second == third == 0: + return False + if case_name == "bcc_fe": + return (first + second + third) % 2 == 0 + if case_name in {"fcc_al", "nacl"}: + return first % 2 == second % 2 == third % 2 + return True + + +def _peak_rows_from_diffractscout(reflections: Iterable[Any]) -> list[dict[str, Any]]: + return [ + { + "d_A": float(reflection.d_spacing_A), + "two_theta_deg": float(reflection.two_theta_deg), + "intensity": float(reflection.intensity_with_lp), + "hkls": [tuple(int(value) for value in reflection.hkl)], + "family_count": 1, + } + for reflection in reflections + ] + + +def _peak_rows_from_pymatgen(pattern: Any) -> list[dict[str, Any]]: + rows: list[dict[str, Any]] = [] + for two_theta, d_A, intensity, family_entries in zip( + pattern.x, pattern.d_hkls, pattern.y, pattern.hkls, strict=True + ): + hkls = [tuple(int(value) for value in entry["hkl"]) for entry in family_entries] + rows.append( + { + "d_A": float(d_A), + "two_theta_deg": float(two_theta), + "intensity": float(intensity), + "hkls": hkls, + "family_count": len(hkls), + } + ) + return rows + + +def _all_peak_hkls(groups: Sequence[Mapping[str, Any]]) -> set[tuple[int, int, int]]: + return { + tuple(int(component) for component in hkl) + for group in groups + for hkl in group["hkls"] + } + + +def _normalized_intensity(groups: Sequence[Mapping[str, Any]], index: int) -> float: + maximum = max(float(group["intensity"]) for group in groups) + if maximum <= 0: + return 0.0 + return 100.0 * float(groups[index]["intensity"]) / maximum + + +def _compare_case( + case_name: str, + specification: Mapping[str, Any], + cif_path: Path, + *, + xrd_calculator_type: Any, + structure_type: Any, + load_structure_fn: Any, + simulate_powder_pattern_fn: Any, + analysis_settings_type: Any, +) -> dict[str, Any]: + from diffractscout.diffraction import CU_KA_WAVELENGTH_A + + if not math.isclose(CU_KA_WAVELENGTH_A, WAVELENGTH_A, rel_tol=0.0, abs_tol=1e-12): + raise ValueError( + f"The reference wavelength ({WAVELENGTH_A} A) does not match the project's Cu Ka " + f"preset ({CU_KA_WAVELENGTH_A} A)." + ) + gemmi_structure = load_structure_fn(cif_path) + gemmi_result = simulate_powder_pattern_fn( + gemmi_structure, + analysis_settings_type( + source_preset="Cu Ka", + two_theta_min_deg=TWO_THETA_RANGE_DEG[0], + two_theta_max_deg=TWO_THETA_RANGE_DEG[1], + include_elasticity=False, + ), + ) + # Disable primitive-cell reduction so reported pymatgen HKLs refer to the + # conventional cells encoded by the packaged benchmark CIFs. + pymatgen_structure = structure_type.from_file(str(cif_path), primitive=False) + pymatgen_calculator = xrd_calculator_type(wavelength=CU_KA_WAVELENGTH_A, symprec=0) + pymatgen_pattern = pymatgen_calculator.get_pattern( + pymatgen_structure, + scaled=False, + two_theta_range=TWO_THETA_RANGE_DEG, + ) + gemmi_groups = _group_by_d_spacing( + _peak_rows_from_diffractscout(gemmi_result.reflections), + tolerance_A=D_SPACING_TOLERANCE_A, + ) + pymatgen_groups = _group_by_d_spacing( + _peak_rows_from_pymatgen(pymatgen_pattern), + tolerance_A=D_SPACING_TOLERANCE_A, + ) + matches, missing_from_pymatgen, missing_from_gemmi = match_d_spacing_groups( + gemmi_groups, + pymatgen_groups, + tolerance_A=D_SPACING_TOLERANCE_A, + ) + + peak_comparisons: list[dict[str, Any]] = [] + max_abs_delta_d = 0.0 + max_abs_delta_two_theta = 0.0 + max_abs_delta_relative_intensity = 0.0 + for gemmi_index, pymatgen_index in matches: + gemmi_group = gemmi_groups[gemmi_index] + pymatgen_group = pymatgen_groups[pymatgen_index] + delta_d = abs(float(gemmi_group["d_A"]) - float(pymatgen_group["d_A"])) + delta_two_theta = abs( + float(gemmi_group["two_theta_deg"]) - float(pymatgen_group["two_theta_deg"]) + ) + gemmi_relative = _normalized_intensity(gemmi_groups, gemmi_index) + pymatgen_relative = _normalized_intensity(pymatgen_groups, pymatgen_index) + delta_relative = abs(gemmi_relative - pymatgen_relative) + max_abs_delta_d = max(max_abs_delta_d, delta_d) + max_abs_delta_two_theta = max(max_abs_delta_two_theta, delta_two_theta) + max_abs_delta_relative_intensity = max(max_abs_delta_relative_intensity, delta_relative) + peak_comparisons.append( + { + "d_A_gemmi": float(gemmi_group["d_A"]), + "d_A_pymatgen": float(pymatgen_group["d_A"]), + "abs_delta_d_A": delta_d, + "two_theta_deg_gemmi": float(gemmi_group["two_theta_deg"]), + "two_theta_deg_pymatgen": float(pymatgen_group["two_theta_deg"]), + "abs_delta_two_theta_deg": delta_two_theta, + "gemmi_hkl_families": gemmi_group["hkls"], + "pymatgen_hkl_families": pymatgen_group["hkls"], + "gemmi_relative_intensity_max_100": gemmi_relative, + "pymatgen_relative_intensity_max_100": pymatgen_relative, + "abs_delta_relative_intensity_percentage_points": delta_relative, + } + ) + + gemmi_hkls = _all_peak_hkls(gemmi_groups) + pymatgen_hkls = _all_peak_hkls(pymatgen_groups) + expected_allowed = { + canonical_cubic_family(hkl) + for hkl in specification["first_allowed"] + } + expected_forbidden = { + canonical_cubic_family(hkl) + for hkl in specification.get("forbidden", []) + } + gemmi_families = {canonical_cubic_family(hkl) for hkl in gemmi_hkls} + pymatgen_families = {canonical_cubic_family(hkl) for hkl in pymatgen_hkls} + gemmi_rule_violations = sorted( + canonical_cubic_family(hkl) + for hkl in gemmi_hkls + if not _selection_rule(case_name, hkl) + ) + pymatgen_rule_violations = sorted( + canonical_cubic_family(hkl) + for hkl in pymatgen_hkls + if not _selection_rule(case_name, hkl) + ) + forbidden_present_gemmi = sorted(expected_forbidden & gemmi_families) + forbidden_present_pymatgen = sorted(expected_forbidden & pymatgen_families) + allowed_missing_gemmi = sorted(expected_allowed - gemmi_families) + allowed_missing_pymatgen = sorted(expected_allowed - pymatgen_families) + + position_passed = ( + not missing_from_gemmi + and not missing_from_pymatgen + and max_abs_delta_d <= D_SPACING_TOLERANCE_A + and max_abs_delta_two_theta <= TWO_THETA_TOLERANCE_DEG + ) + selection_passed = not ( + gemmi_rule_violations + or pymatgen_rule_violations + or forbidden_present_gemmi + or forbidden_present_pymatgen + or allowed_missing_gemmi + or allowed_missing_pymatgen + ) + return { + "input_cif": cif_path.relative_to(REPOSITORY_ROOT).as_posix(), + "input_sha256": sha256_file(cif_path), + "pymatgen_structure_site_count": len(pymatgen_structure), + "gemmi_reflection_family_count": len(gemmi_result.reflections), + "pymatgen_pattern_peak_count": len(pymatgen_pattern.x), + "gemmi_d_spacing_group_count": len(gemmi_groups), + "pymatgen_d_spacing_group_count": len(pymatgen_groups), + "matched_d_spacing_group_count": len(matches), + "unmatched_gemmi_group_indices": missing_from_pymatgen, + "unmatched_pymatgen_group_indices": missing_from_gemmi, + "position_max_abs_delta_d_A": max_abs_delta_d, + "position_max_abs_delta_two_theta_deg": max_abs_delta_two_theta, + "position_passed": position_passed, + "selection_rules": { + "expected_allowed_families": [list(item) for item in sorted(expected_allowed)], + "expected_forbidden_families": [list(item) for item in sorted(expected_forbidden)], + "gemmi_forbidden_present": [list(item) for item in forbidden_present_gemmi], + "pymatgen_forbidden_present": [list(item) for item in forbidden_present_pymatgen], + "gemmi_allowed_missing": [list(item) for item in allowed_missing_gemmi], + "pymatgen_allowed_missing": [list(item) for item in allowed_missing_pymatgen], + "gemmi_rule_violations": [list(item) for item in gemmi_rule_violations], + "pymatgen_rule_violations": [list(item) for item in pymatgen_rule_violations], + "passed": selection_passed, + }, + "intensity_comparison": { + "interpretation": "diagnostic_only_no_acceptance_threshold", + "max_abs_delta_relative_intensity_percentage_points": max_abs_delta_relative_intensity, + "peaks": peak_comparisons, + }, + } + + +def _cubic_elasticity_comparison( + *, + pymatgen_elastic_tensor_type: Any, + validate_elastic_tensor_fn: Any, + project_young_modulus_fn: Any, +) -> dict[str, Any]: + from diffractscout.structure import load_structure + + cif_path = BENCHMARK_ROOT / "fcc_al.cif" + structure = load_structure(cif_path) + stiffness = np.asarray(CUBIC_STIFFNESS_GPA, dtype=float) + project_tensor = validate_elastic_tensor_fn(stiffness, nature_of_data="synthetic_test_fixture") + reference_tensor = pymatgen_elastic_tensor_type.from_voigt(stiffness) + reference_compliance = reference_tensor.compliance_tensor + rows: list[dict[str, Any]] = [] + passed = True + for hkl in CUBIC_DIRECTIONS: + gemmi_modulus = project_young_modulus_fn( + project_tensor, + structure.small_structure.cell, + hkl, + ) + # Contract the 4th-rank compliance tensor to obtain Young's modulus. + # ElasticTensor.directional_elastic_mod instead contracts stiffness + # and returns a longitudinal modulus, so it is not the target quantity. + direction = np.asarray(hkl, dtype=float) + direction /= np.linalg.norm(direction) + inverse_modulus = float(reference_compliance.einsum_sequence([direction] * 4)) + pymatgen_modulus = 1.0 / inverse_modulus + if gemmi_modulus is None: + passed = False + delta = None + check = False + else: + delta = abs(float(gemmi_modulus) - pymatgen_modulus) + check = delta <= max( + ELASTIC_ABSOLUTE_TOLERANCE_GPA, + ELASTIC_RELATIVE_TOLERANCE * abs(pymatgen_modulus), + ) + passed = passed and check + rows.append( + { + "hkl_normal": list(hkl), + "gemmi_project_modulus_GPa": ( + float(gemmi_modulus) if gemmi_modulus is not None else None + ), + "pymatgen_modulus_GPa": pymatgen_modulus, + "abs_delta_GPa": delta, + "passed": check, + } + ) + return { + "input_cif": cif_path.relative_to(REPOSITORY_ROOT).as_posix(), + "input_sha256": sha256_file(cif_path), + "synthetic_stiffness_GPa": [list(row) for row in CUBIC_STIFFNESS_GPA], + "voigt_order": ["11", "22", "33", "23", "13", "12"], + "shear_convention": "engineering shear strain", + "comparison_tolerances": { + "absolute_GPa": ELASTIC_ABSOLUTE_TOLERANCE_GPA, + "relative": ELASTIC_RELATIVE_TOLERANCE, + "acceptance": "abs(delta) <= max(absolute_GPa, relative * abs(pymatgen))", + }, + "directions": rows, + "passed": passed, + } + + +def _markdown_report(report: Mapping[str, Any]) -> str: + lines = [ + "# Independent pymatgen reference comparison", + "", + f"Generated: `{report['generated_at_utc']}`", + "", + f"Overall acceptance: **{'PASS' if report['all_acceptance_checks_passed'] else 'FAIL'}**", + "", + "This receipt records independent software comparisons on explicitly synthetic benchmark fixtures. " + "It is not experimental validation, a real-material property claim, or external-user evidence.", + "", + "## Software and settings", + "", + f"- Python: `{report['software']['python']}`", + f"- Comparison script SHA-256: `{report['software']['comparison_script_sha256']}`", + f"- Gemmi: `{report['software']['gemmi']}`", + f"- pymatgen: `{report['software']['pymatgen']}`", + f"- Radiation: Cu Kα effective wavelength `{WAVELENGTH_A} Å`", + f"- 2θ range: `{TWO_THETA_RANGE_DEG[0]}–{TWO_THETA_RANGE_DEG[1]}°`", + f"- Peak grouping/matching: `|Δd| ≤ {D_SPACING_TOLERANCE_A:g} Å`; geometry acceptance also requires " + f"`|Δ2θ| ≤ {TWO_THETA_TOLERANCE_DEG:g}°`.", + "- Diffraction intensities use each engine's unscaled theoretical pattern, are grouped at coincident d, " + "and are separately normalized to a per-case maximum of 100; differences are diagnostic only.", + "", + "## Diffraction peak positions and selection rules", + "", + "| Synthetic CIF | Gemmi families | pymatgen groups | Matched d groups | max abs Δd (Å) | max abs Δ2θ (°) | position | selection | max intensity difference (points) |", + "|---|---:|---:|---:|---:|---:|:---:|:---:|---:|", + ] + for name, case in report["diffraction_cases"].items(): + lines.append( + f"| `{case['input_cif']}` ({name}) | {case['gemmi_reflection_family_count']} | " + f"{case['pymatgen_d_spacing_group_count']} | {case['matched_d_spacing_group_count']} | " + f"{case['position_max_abs_delta_d_A']:.3g} | " + f"{case['position_max_abs_delta_two_theta_deg']:.3g} | " + f"{'PASS' if case['position_passed'] else 'FAIL'} | " + f"{'PASS' if case['selection_rules']['passed'] else 'FAIL'} | " + f"{case['intensity_comparison']['max_abs_delta_relative_intensity_percentage_points']:.4g} |" + ) + lines.extend( + [ + "", + "The comparison groups lines by d-spacing rather than requiring identical representative hkl labels. " + "Pymatgen may return a single pattern peak containing several hkl families where DiffractScout " + "returns separate family rows; the comparison therefore sums coincident-family intensities before " + "per-case normalization.", + "", + "## Cubic directional Young's modulus", + "", + f"The same synthetic stiffness matrix in GPa is interpreted with Voigt order `{report['elasticity']['voigt_order']}` " + f"and {report['elasticity']['shear_convention']}. Acceptance is " + f"`|ΔE| ≤ max({ELASTIC_ABSOLUTE_TOLERANCE_GPA:g} GPa, " + f"{ELASTIC_RELATIVE_TOLERANCE:g} × |E_pymatgen|)`.", + "", + "| Plane normal | DiffractScout (GPa) | pymatgen (GPa) | abs ΔE (GPa) | Result |", + "|---|---:|---:|---:|:---:|", + ] + ) + for row in report["elasticity"]["directions"]: + lines.append( + f"| `{row['hkl_normal']}` | {row['gemmi_project_modulus_GPa']:.12g} | " + f"{row['pymatgen_modulus_GPa']:.12g} | {row['abs_delta_GPa']:.3g} | " + f"{'PASS' if row['passed'] else 'FAIL'} |" + ) + lines.extend( + [ + "", + "## Reproduction", + "", + "Install the optional Materials Project dependencies, then run from the repository root:", + "", + "```sh", + "python scripts/compare_reference_engines.py --output validation_cases/independent_engines", + "```", + "", + "The tracked JSON receipt records relative CIF paths and SHA-256 hashes, software versions, " + "thresholds, per-peak results, selection-rule outcomes, and tensor results. The script exits " + "nonzero if peak geometry/selection or elastic acceptance checks fail. Intensity differences do " + "not affect the exit status.", + "", + ] + ) + return "\n".join(lines) + + +def run_comparison(output_dir: str | Path = DEFAULT_OUTPUT) -> dict[str, Any]: + """Run all synthetic cross-engine comparisons and write JSON/Markdown receipts.""" + + if str(REPOSITORY_ROOT / "src") not in sys.path: + sys.path.insert(0, str(REPOSITORY_ROOT / "src")) + try: + from pymatgen.analysis.diffraction.xrd import XRDCalculator + from pymatgen.core import Structure + except ImportError as exc: + raise RuntimeError( + "The independent comparison needs pymatgen. Install the project's optional " + "Materials Project dependencies with `pip install -e '.[mp]'`." + ) from exc + try: + from pymatgen.core.elasticity import ElasticTensor + except ModuleNotFoundError as exc: + if exc.name != "pymatgen.core.elasticity": + raise + # pymatgen through 2025.10 exposes ElasticTensor under analysis; the + # 2026 core split moved it to pymatgen.core.elasticity. + try: + from pymatgen.analysis.elasticity.elastic import ElasticTensor + except ImportError as fallback_exc: + raise RuntimeError( + "The installed pymatgen does not expose ElasticTensor from either " + "the current core path or the legacy analysis path." + ) from fallback_exc + + from diffractscout import __version__ as diffractscout_version + from diffractscout.diffraction import simulate_powder_pattern + from diffractscout.elasticity import validate_elastic_tensor, young_modulus_hkl_normal_GPa + from diffractscout.models import AnalysisSettings + from diffractscout.structure import load_structure + + expectations_path = BENCHMARK_ROOT / "expectations.json" + expectations = json.loads(expectations_path.read_text(encoding="utf-8")) + cases = _validate_expectations(expectations) + + diffraction_cases: dict[str, Any] = {} + for case_name, specification in cases.items(): + cif_path = BENCHMARK_ROOT / str(specification["file"]) + with warnings.catch_warnings(record=True) as caught_warnings: + warnings.simplefilter("always") + case = _compare_case( + case_name, + specification, + cif_path, + xrd_calculator_type=XRDCalculator, + structure_type=Structure, + load_structure_fn=load_structure, + simulate_powder_pattern_fn=simulate_powder_pattern, + analysis_settings_type=AnalysisSettings, + ) + case["pymatgen_parse_warnings"] = list( + dict.fromkeys(str(item.message) for item in caught_warnings) + ) + diffraction_cases[case_name] = case + + elasticity = _cubic_elasticity_comparison( + pymatgen_elastic_tensor_type=ElasticTensor, + validate_elastic_tensor_fn=validate_elastic_tensor, + project_young_modulus_fn=young_modulus_hkl_normal_GPa, + ) + all_acceptance_checks_passed = ( + all(case["position_passed"] and case["selection_rules"]["passed"] for case in diffraction_cases.values()) + and elasticity["passed"] + ) + try: + gemmi_version = importlib.metadata.version("gemmi") + except importlib.metadata.PackageNotFoundError: + gemmi_version = "unknown" + try: + pymatgen_version = importlib.metadata.version("pymatgen") + except importlib.metadata.PackageNotFoundError: + pymatgen_version = "unknown" + report: dict[str, Any] = { + "schema": "diffractscout_independent_reference_engine_comparison_v1", + "generated_at_utc": datetime.now(timezone.utc).isoformat(timespec="seconds"), + "all_acceptance_checks_passed": all_acceptance_checks_passed, + "synthetic_inputs_only": True, + "software": { + "diffractscout": diffractscout_version, + "comparison_script_sha256": sha256_file(Path(__file__).resolve()), + "python": ".".join(str(value) for value in sys.version_info[:3]), + "gemmi": gemmi_version, + "pymatgen": pymatgen_version, + }, + "settings": { + "radiation": "Cu Kα effective single wavelength", + "wavelength_A": WAVELENGTH_A, + "two_theta_range_deg": list(TWO_THETA_RANGE_DEG), + "d_spacing_merge_and_match_tolerance_A": D_SPACING_TOLERANCE_A, + "two_theta_acceptance_tolerance_deg": TWO_THETA_TOLERANCE_DEG, + "intensity_comparison": "unscaled calculated peaks summed at coincident d then normalized per case to maximum 100; diagnostic only", + }, + "expectations": { + "path": expectations_path.relative_to(REPOSITORY_ROOT).as_posix(), + "sha256": sha256_file(expectations_path), + }, + "diffraction_cases": diffraction_cases, + "elasticity": elasticity, + } + + output = Path(output_dir) + if not output.is_absolute(): + output = (REPOSITORY_ROOT / output).resolve() + output.mkdir(parents=True, exist_ok=True) + json_path = output / "reference_engine_comparison.json" + markdown_path = output / "reference_engine_comparison.md" + report["output_files"] = [ + json_path.relative_to(REPOSITORY_ROOT).as_posix() + if json_path.is_relative_to(REPOSITORY_ROOT) + else json_path.name, + markdown_path.relative_to(REPOSITORY_ROOT).as_posix() + if markdown_path.is_relative_to(REPOSITORY_ROOT) + else markdown_path.name, + ] + json_path.write_text( + json.dumps(report, indent=2, ensure_ascii=False, sort_keys=True) + "\n", + encoding="utf-8", + newline="\n", + ) + markdown_path.write_text(_markdown_report(report), encoding="utf-8", newline="\n") + return report + + +def main(argv: Sequence[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--output", + type=Path, + default=DEFAULT_OUTPUT, + help="directory for the JSON and Markdown receipts (default: %(default)s)", + ) + arguments = parser.parse_args(argv) + try: + report = run_comparison(arguments.output) + except (RuntimeError, ValueError) as exc: + parser.error(str(exc)) + print( + f"Independent reference comparison: " + f"{'PASS' if report['all_acceptance_checks_passed'] else 'FAIL'}; " + f"{len(report['diffraction_cases'])} diffraction fixtures and cubic elasticity." + ) + for relative_path in report["output_files"]: + print(f" {relative_path}") + return 0 if report["all_acceptance_checks_passed"] else 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/joss_readiness.py b/scripts/joss_readiness.py index 71a3322..2db1bd6 100644 --- a/scripts/joss_readiness.py +++ b/scripts/joss_readiness.py @@ -78,6 +78,13 @@ "References", ) +PAPER_SOURCE_PATHS = ( + "paper/paper.md", + "paper/paper.bib", + "paper/fig_workflow.png", + "paper/fig_validation.png", +) + REQUIRED_REPOSITORY_FILES = ( "LICENSE", "README.md", @@ -333,10 +340,11 @@ def _release_acceptance(version: str | None) -> dict[str, Any]: } -def _load_evidence() -> dict[str, Any]: +def _load_evidence(path: Path | None = None) -> dict[str, Any]: + evidence_path = path if path is not None else EVIDENCE_PATH try: - payload = json.loads(EVIDENCE_PATH.read_text(encoding="utf-8")) - except (OSError, json.JSONDecodeError) as exc: + payload = json.loads(evidence_path.read_text(encoding="utf-8")) + except (OSError, UnicodeError, json.JSONDecodeError) as exc: return {"schema": "invalid", "error": str(exc)} return payload if isinstance(payload, dict) else {"schema": "invalid"} @@ -718,14 +726,42 @@ def _paper_source_sha256(source_paths: list[Path]) -> str: return digest.hexdigest() +def _official_build_sources_match( + build_commit: str, + current_head: str, +) -> bool: + """Require the official PDF build commit to contain the current paper source blobs.""" + + if not re.fullmatch(r"[0-9a-fA-F]{40}", build_commit) or not re.fullmatch( + r"[0-9a-fA-F]{40}", current_head + ): + return False + build_commit = build_commit.lower() + current_head = current_head.lower() + try: + _run_git("merge-base", "--is-ancestor", build_commit, current_head) + for relative_path in PAPER_SOURCE_PATHS: + build_blob = _run_git( + "rev-parse", "--verify", f"{build_commit}:{relative_path}" + ) + head_blob = _run_git( + "rev-parse", "--verify", f"{current_head}:{relative_path}" + ) + if ( + _run_git("cat-file", "-t", str(build_blob)) != "blob" + or _run_git("cat-file", "-t", str(head_blob)) != "blob" + ): + return False + if build_blob != head_blob: + return False + except (OSError, subprocess.CalledProcessError): + return False + return True + + def _paper_pdf_metadata(evidence: dict[str, Any]) -> dict[str, Any]: path = ROOT / "paper/paper.pdf" - source_paths = [ - ROOT / "paper/paper.md", - ROOT / "paper/paper.bib", - ROOT / "paper/fig_workflow.png", - ROOT / "paper/fig_validation.png", - ] + source_paths = [ROOT / relative_path for relative_path in PAPER_SOURCE_PATHS] raw_submission = evidence.get("submission_metadata") submission = raw_submission if isinstance(raw_submission, dict) else {} try: @@ -775,12 +811,16 @@ def _submission_metadata(evidence: dict[str, Any], *, current_head: str) -> dict for name in ("remote_ci_url", "official_joss_build_url") if not _is_public_https_url(values.get(name)) ] - invalid_commits = [ - name - for name in ("remote_ci_commit", "official_joss_build_commit") - if not re.fullmatch(r"[0-9a-fA-F]{40}", str(values.get(name, ""))) - or str(values.get(name, "")).lower() != current_head.lower() - ] + remote_ci_commit = str(values.get("remote_ci_commit", "")) + official_build_commit = str(values.get("official_joss_build_commit", "")) + invalid_commits: list[str] = [] + if ( + not re.fullmatch(r"[0-9a-fA-F]{40}", remote_ci_commit) + or remote_ci_commit.lower() != current_head.lower() + ): + invalid_commits.append("remote_ci_commit") + if not _official_build_sources_match(official_build_commit, current_head): + invalid_commits.append("official_joss_build_commit") return { "ok": not missing_or_false and not invalid_urls and not invalid_commits, "required_confirmations": list(SUBMISSION_CONFIRMATIONS), @@ -796,6 +836,7 @@ def _public_development_snapshot( *, public_since: date | None, as_of: date, + evidence_path: Path = EVIDENCE_PATH, ) -> dict[str, Any]: """Count only ledger-linked public repository activity, never local timestamps.""" @@ -816,11 +857,15 @@ def _public_development_snapshot( continue accepted.append(record) active_months = Counter(str(record["date"])[:7] for record in accepted) + try: + evidence_source = evidence_path.resolve().relative_to(ROOT).as_posix() + except ValueError: + evidence_source = str(evidence_path.resolve()) return { "entries": len(accepted), "active_months": dict(sorted(active_months.items())), "active_month_count": len(active_months), - "source": "docs/evidence/impact_evidence.json public_development_activity", + "source": f"{evidence_source} public_development_activity", "limitation": ( "Each ledger URL remains subject to human review; local commit dates are context only." ), @@ -928,11 +973,20 @@ def build_report( public_since: date | None, as_of: date, stage: str = "submission", + evidence_file: Path | None = None, ) -> dict[str, Any]: if stage not in STAGES: raise ValueError(f"Unknown readiness stage: {stage!r}.") - evidence = _load_evidence() + selected_evidence_path = ( + evidence_file.expanduser().resolve() if evidence_file is not None else EVIDENCE_PATH + ) + evidence = _load_evidence(selected_evidence_path) evidence_errors = validate_evidence_payload(evidence) + evidence_load_error = ( + str(evidence["error"]) + if evidence.get("schema") == "invalid" and isinstance(evidence.get("error"), str) + else None + ) citation = _citation_metadata() remote = _git_remote() version = _version_metadata() @@ -1080,6 +1134,7 @@ def build_report( evidence, public_since=public_since, as_of=as_of, + evidence_path=selected_evidence_path, ) active_month_count = int(public_development["active_month_count"]) _check( @@ -1160,7 +1215,10 @@ def build_report( "pass" if submission_source["ok"] else "block", submission_source, gate="project", - reason="Submission and publication must use a clean checkout whose HEAD matches the recorded remote CI and official paper builds.", + reason=( + "Submission and publication must use a clean checkout whose HEAD matches remote CI; " + "the official paper build must match the current paper input blobs." + ), required_from="submission", ) @@ -1242,6 +1300,8 @@ def build_report( "generated_at_utc": datetime.now(timezone.utc).isoformat(), "as_of": as_of.isoformat(), "stage": stage, + "evidence_file": str(selected_evidence_path), + "evidence_load_error": evidence_load_error, "public_since": public_since.isoformat() if public_since else None, "earliest_calendar_eligibility": eligible_on.isoformat() if eligible_on else None, "ready": selected_status["ready"], @@ -1283,6 +1343,7 @@ def report_markdown(report: dict[str, Any]) -> str: "", f"- As of: `{report['as_of']}`", f"- Selected stage: `{report['stage']}`", + f"- Evidence ledger: `{report['evidence_file']}`", f"- Public since: `{report['public_since'] or 'not recorded'}`", f"- Earliest six-month date: `{report['earliest_calendar_eligibility'] or 'not calculable'}`", f"- Status: **{'READY' if report['ready'] else 'BLOCKED'}**", @@ -1332,11 +1393,24 @@ def main() -> int: parser.add_argument("--json", action="store_true") parser.add_argument("--strict", action="store_true") parser.add_argument("--stage", choices=STAGES, default="submission") + parser.add_argument( + "--evidence-file", + metavar="PATH", + help=( + "Read a separate JSON evidence ledger (default: " + "docs/evidence/impact_evidence.json). The selected file is read-only." + ), + ) args = parser.parse_args() public_since = date.fromisoformat(args.public_since) if args.public_since else None as_of = date.fromisoformat(args.as_of) - report = build_report(public_since=public_since, as_of=as_of, stage=args.stage) + report = build_report( + public_since=public_since, + as_of=as_of, + stage=args.stage, + evidence_file=Path(args.evidence_file) if args.evidence_file else None, + ) if args.output: output = Path(args.output).expanduser().resolve() output.mkdir(parents=True, exist_ok=True) diff --git a/src/diffractscout/cli.py b/src/diffractscout/cli.py index 917a879..04a7e6d 100644 --- a/src/diffractscout/cli.py +++ b/src/diffractscout/cli.py @@ -10,8 +10,15 @@ from . import __version__ from .benchmark import run_reference_benchmarks, verify_benchmark_bundle +from .cli_presets import ( + add_analysis_options, + resolve_cli_analysis, + save_cli_preset, + show_cli_preset, +) from .demo import write_demo_inputs -from .models import AnalysisSettings, DiscoverySettings +from .inspection import format_inspection, inspect_bundle +from .models import AnalysisSettings, DiscoverySettings, PipelineResult from .pipeline import analyze_cifs, export_discovery, run_pipeline from .providers.materials_project import MaterialsProjectProvider from .utils import to_jsonable @@ -19,34 +26,7 @@ def _analysis_settings(args: argparse.Namespace) -> AnalysisSettings: - if args.energy_keV is not None: - mode = "energy" - elif args.wavelength_A is not None: - mode = "wavelength" - else: - mode = "source" - return AnalysisSettings( - input_mode=mode, # type: ignore[arg-type] - source_preset=args.source, - wavelength_A=args.wavelength_A, - energy_keV=args.energy_keV, - two_theta_min_deg=args.two_theta_min, - two_theta_max_deg=args.two_theta_max, - step_deg=args.step, - fwhm_deg=args.fwhm, - profile_eta=args.eta, - include_elasticity=not args.no_elasticity, - max_profile_points=args.max_profile_points, - max_reflection_estimate=args.max_reflection_estimate, - d_min_A=args.d_min, - d_max_A=args.d_max, - profile_model=args.profile_model, - pattern_axis=args.pattern_axis, - include_figures=bool(args.figures), - figure_preset=args.figure_preset, - export_lab_views=not args.no_lab_views, - include_patterns=not args.no_patterns, - ) + return resolve_cli_analysis(args).settings def _discovery_settings(args: argparse.Namespace) -> DiscoverySettings: @@ -68,24 +48,81 @@ def _api_key(args: argparse.Namespace) -> str: return value -def _print_result(result: object, *, as_json: bool = False) -> None: - payload = to_jsonable(result) +def _diagnostic_lines(result: PipelineResult) -> list[tuple[str, str]]: + """Render diagnostics with their recorded severity, retaining legacy warnings.""" + + lines: list[tuple[str, str]] = [] + represented_messages: set[str] = set() + seen: set[tuple[str, str, str]] = set() + for item in result.diagnostics: + level = item.level.lower() + if level not in {"warning", "error"}: + continue + name = item.item or "" + message = item.message + key = (level, name, message) + if key not in seen: + seen.add(key) + lines.append((level.upper(), f"{name}: {message}" if name else message)) + represented_messages.add(f"{name}: {message}" if name else message) + + for warning in result.warnings: + message = str(warning) + if message not in represented_messages: + key = ("warning", "", message) + if key not in seen: + seen.add(key) + lines.append(("WARNING", message)) + represented_messages.add(message) + return lines + + +def _result_artifact_paths( + result: PipelineResult, *, excel_path: str | Path | None = None +) -> dict[str, str]: + output_dir = result.output_dir + artifacts: dict[str, str] = {} + artifacts["manifest"] = str(result.manifest_path) + diagnostics_path = output_dir / "diagnostics.csv" + if diagnostics_path.is_file(): + artifacts["diagnostics"] = str(diagnostics_path) + workbook = Path(excel_path).expanduser() if excel_path is not None else output_dir / "results.xlsx" + if workbook.is_file(): + artifacts["excel"] = str(workbook.resolve()) + return artifacts + + +def _print_result( + result: PipelineResult, + *, + as_json: bool = False, + excel_path: str | Path | None = None, +) -> None: if as_json: - print(json.dumps(payload, indent=2, ensure_ascii=False)) + print(json.dumps(to_jsonable(result), indent=2, ensure_ascii=False)) return - if hasattr(result, "output_dir"): - print(f"Output: {result.output_dir}") - print(f"Manifest: {result.manifest_path}") - print(f"Analyzed phases: {len(result.analyses)}") - if result.discovery is not None: - print(f"Discovered candidates: {len(result.discovery.candidates)}") - if result.downloads: - successful = sum(1 for item in result.downloads if item.status == "ok") - print(f"Downloads: {successful}/{len(result.downloads)} successful") - for warning in result.warnings: - print(f"WARNING: {warning}", file=sys.stderr) - else: - print(payload) + artifact_paths = _result_artifact_paths(result, excel_path=excel_path) + diagnostic_lines = _diagnostic_lines(result) + warning_count = sum(1 for level, _ in diagnostic_lines if level == "WARNING") + error_count = sum(1 for level, _ in diagnostic_lines if level == "ERROR") + reflection_count = sum(len(item.reflections) for item in result.analyses) + print(f"Output: {result.output_dir}") + print(f"Manifest: {result.manifest_path}") + if "excel" in artifact_paths: + print(f"Excel: {artifact_paths['excel']}") + if "diagnostics" in artifact_paths: + print(f"Diagnostics: {artifact_paths['diagnostics']}") + print(f"Analyzed phases: {len(result.analyses)}") + print(f"Reflections: {reflection_count}") + print(f"Warnings: {warning_count}") + print(f"Errors: {error_count}") + if result.discovery is not None: + print(f"Discovered candidates: {len(result.discovery.candidates)}") + if result.downloads: + successful = sum(1 for item in result.downloads if item.status == "ok") + print(f"Downloads: {successful}/{len(result.downloads)} successful") + for level, message in diagnostic_lines: + print(f"{level}: {message}", file=sys.stderr) def _pipeline_exit_code(result: object) -> int: @@ -103,70 +140,21 @@ def _pipeline_exit_code(result: object) -> int: return 3 if any(item.level == "error" for item in diagnostics) else 0 -def _add_analysis_options(parser: argparse.ArgumentParser) -> None: - parser.add_argument("--source", default="Cu Ka", choices=("Cu Ka", "Co Ka", "Fe Ka", "Mo Ka", "Ag Ka", "Custom")) - radiation = parser.add_mutually_exclusive_group() - radiation.add_argument("--energy-keV", type=float, default=None, help="Custom X-ray energy in keV.") - radiation.add_argument("--wavelength-A", type=float, default=None, help="Custom wavelength in Angstrom.") - parser.add_argument("--two-theta-min", type=float, default=5.0) - parser.add_argument("--two-theta-max", type=float, default=120.0) - parser.add_argument("--step", type=float, default=0.02, help="Display-profile grid spacing in degrees.") - parser.add_argument("--fwhm", type=float, default=0.15, help="Display-profile FWHM in degrees.") - parser.add_argument("--eta", type=float, default=0.5, help="Pseudo-Voigt Lorentzian fraction in [0, 1].") - parser.add_argument("--no-elasticity", action="store_true", help="Do not discover, copy, or calculate paired elastic data.") - parser.add_argument("--max-profile-points", type=int, default=1_000_000, help="Safety limit for the generated display-profile grid.") - parser.add_argument("--max-reflection-estimate", type=int, default=2_000_000, help="Safety limit for reciprocal-lattice candidate generation.") - # Keep command help encodable by legacy Windows consoles (for example - # CP936/GBK). Scientific exports retain the Å symbol; terminal help uses - # the unambiguous ASCII spelling so ``--help`` cannot fail before parsing. - parser.add_argument( - "--d-min", - type=float, - default=None, - dest="d_min", - help="Minimum d-spacing filter in Angstrom.", - ) - parser.add_argument( - "--d-max", - type=float, - default=None, - dest="d_max", - help="Maximum d-spacing filter in Angstrom.", - ) - parser.add_argument( - "--profile-model", - choices=("pseudo_voigt", "gaussian", "lorentzian"), - default="pseudo_voigt", - help="Display-profile lineshape model.", - ) - parser.add_argument( - "--pattern-axis", - choices=("two_theta", "d_spacing", "q", "g"), - default="two_theta", - help=( - "Selected x coordinate in pattern_profiles.csv and Excel. " - "Figures remain on 2theta in v0.4.0." - ), +def _add_analysis_options( + parser: argparse.ArgumentParser, + *, + include_preset: bool = True, + include_output_controls: bool = True, + include_json: bool = True, + include_recursive: bool = True, +) -> None: + add_analysis_options( + parser, + include_preset=include_preset, + include_output_controls=include_output_controls, + include_json=include_json, + include_recursive=include_recursive, ) - parser.add_argument( - "--figures", - action="store_true", - help="Write 2theta figures (SVG and PNG in result bundles).", - ) - parser.add_argument("--figure-preset", default="publication", help="Named figure style preset.") - parser.add_argument( - "--no-lab-views", - action="store_true", - help="Skip laboratory convenience views in the result bundle.", - ) - parser.add_argument( - "--no-patterns", - action="store_true", - help="Skip continuous powder-pattern series in exports.", - ) - parser.add_argument("--no-excel", action="store_true", help="Skip results.xlsx; CSV and JSON remain enabled.") - parser.add_argument("--overwrite", action="store_true", help="Replace only an existing DiffractScout output bundle.") - parser.add_argument("--json", action="store_true", help="Print the final summary as JSON.") def _add_discovery_options(parser: argparse.ArgumentParser) -> None: @@ -200,7 +188,6 @@ def build_parser() -> argparse.ArgumentParser: analyze = subparsers.add_parser("analyze", help="Analyze local CIF files or directories.") analyze.add_argument("inputs", nargs="+", help="CIF files or directories.") analyze.add_argument("-o", "--output", required=True) - analyze.add_argument("--no-recursive", action="store_true") _add_analysis_options(analyze) discover = subparsers.add_parser("discover", help="Query candidate phases without downloading structures.") @@ -215,7 +202,7 @@ def build_parser() -> argparse.ArgumentParser: run.add_argument("composition", help="Alloy grade, formula, chemical system, or mp-IDs.") run.add_argument("-o", "--output", required=True) _add_discovery_options(run) - _add_analysis_options(run) + _add_analysis_options(run, include_recursive=False) run.add_argument( "--primitive", action="store_true", @@ -248,9 +235,22 @@ def build_parser() -> argparse.ArgumentParser: ) quick.add_argument("inputs", nargs="+", help="CIF files or directories.") quick.add_argument("-o", "--output", required=True, help="Bundle directory or .xlsx path.") - quick.add_argument("--no-recursive", action="store_true") _add_analysis_options(quick) + inspect = subparsers.add_parser("inspect", help="Verify a result bundle and summarize its contents.") + inspect.add_argument("bundle", help="Result bundle directory to inspect.") + inspect.add_argument("--json", action="store_true", help="Print the inspection report as JSON.") + + preset = subparsers.add_parser("preset", help="Create or inspect reusable analysis presets.") + preset_commands = preset.add_subparsers(dest="preset_command", required=True) + preset_save = preset_commands.add_parser("save", help="Save merged analysis options without running an analysis.") + preset_save.add_argument("-o", "--output", required=True, help="Preset JSON file to create.") + _add_analysis_options(preset_save, include_output_controls=False, include_json=False) + preset_save.add_argument("--overwrite", action="store_true", help="Replace an existing preset file.") + preset_show = preset_commands.add_parser("show", help="Validate and display a preset file.") + preset_show.add_argument("file", help="Preset JSON file.") + preset_show.add_argument("--json", action="store_true", help="Print the normalized preset values as JSON.") + subparsers.add_parser("gui", help="Launch the optional Tk desktop interface.") return parser @@ -260,12 +260,13 @@ def main(argv: Sequence[str] | None = None) -> int: args = parser.parse_args(argv) try: if args.command == "analyze": + analysis = resolve_cli_analysis(args) result = analyze_cifs( args.inputs, args.output, - settings=_analysis_settings(args), - recursive=not args.no_recursive, - include_excel=not args.no_excel, + settings=analysis.settings, + recursive=analysis.recursive, + include_excel=analysis.include_excel, overwrite=args.overwrite, ) _print_result(result, as_json=args.json) @@ -285,16 +286,17 @@ def main(argv: Sequence[str] | None = None) -> int: return 0 if result.discovery and result.discovery.candidates else 2 if args.command == "run": + analysis = resolve_cli_analysis(args) provider = MaterialsProjectProvider(_api_key(args)) result = run_pipeline( args.composition, provider, args.output, discovery_settings=_discovery_settings(args), - analysis_settings=_analysis_settings(args), + analysis_settings=analysis.settings, conventional_unit_cell=not args.primitive, - include_elasticity=not args.no_elasticity, - include_excel=not args.no_excel, + include_elasticity=analysis.settings.include_elasticity, + include_excel=analysis.include_excel, overwrite=args.overwrite, confirm_above=args.confirm_above, authorize_large_download=args.yes, @@ -325,6 +327,14 @@ def main(argv: Sequence[str] | None = None) -> int: print(f"ERROR: {error}", file=sys.stderr) return 0 if report["ok"] else 2 + if args.command == "inspect": + report = inspect_bundle(args.bundle) + if args.json: + print(json.dumps(report, indent=2, ensure_ascii=False)) + else: + print(format_inspection(report)) + return 0 if report.get("ok") else 2 + if args.command == "benchmark": report = run_reference_benchmarks( args.output, @@ -343,17 +353,39 @@ def main(argv: Sequence[str] | None = None) -> int: if args.command == "quick-export": from .quick_export import quick_export + excel_path = args.output if Path(args.output).suffix.lower() == ".xlsx" else None + analysis = resolve_cli_analysis(args) result = quick_export( args.inputs, args.output, - settings=_analysis_settings(args), - recursive=not args.no_recursive, - include_excel=not args.no_excel, + settings=analysis.settings, + recursive=analysis.recursive, + include_excel=analysis.include_excel, overwrite=args.overwrite, ) - _print_result(result, as_json=args.json) + _print_result(result, as_json=args.json, excel_path=excel_path) return _pipeline_exit_code(result) + if args.command == "preset": + if args.preset_command == "save": + analysis = resolve_cli_analysis(args) + target = save_cli_preset( + args.output, + analysis.preset_values, + overwrite=args.overwrite, + ) + print(f"Preset saved: {target}") + return 0 + if args.preset_command == "show": + values = show_cli_preset(args.file) + if args.json: + print(json.dumps(values, indent=2, ensure_ascii=False)) + else: + print(f"Preset: {Path(args.file).expanduser()}") + for key, value in sorted(values.items()): + print(f"{key}: {value}") + return 0 + if args.command == "gui": from .gui import main as gui_main diff --git a/src/diffractscout/cli_presets.py b/src/diffractscout/cli_presets.py new file mode 100644 index 0000000..0bbb89f --- /dev/null +++ b/src/diffractscout/cli_presets.py @@ -0,0 +1,437 @@ +from __future__ import annotations + +import argparse +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +from .gui_settings import load_analysis_preset, save_analysis_preset +from .models import AnalysisSettings + +_EXPLICIT_ATTR = "_diffractscout_explicit" + + +class _TrackedStore(argparse.Action): + """Store an option and mark its destination on this parse namespace.""" + + def __call__( + self, + parser: argparse.ArgumentParser, + namespace: argparse.Namespace, + values: Any, + option_string: str | None = None, + ) -> None: + setattr(namespace, self.dest, values) + explicit = set(getattr(namespace, _EXPLICIT_ATTR, ())) + explicit.add(self.dest) + setattr(namespace, _EXPLICIT_ATTR, explicit) + + +class _TrackedConst(argparse.Action): + """Store a fixed value and mark its destination on this parse namespace.""" + + def __init__(self, option_strings: list[str], dest: str, *, const: Any, **kwargs: Any) -> None: + super().__init__(option_strings, dest, nargs=0, **kwargs) + self.const = const + + def __call__( + self, + parser: argparse.ArgumentParser, + namespace: argparse.Namespace, + values: Any, + option_string: str | None = None, + ) -> None: + setattr(namespace, self.dest, self.const) + explicit = set(getattr(namespace, _EXPLICIT_ATTR, ())) + explicit.add(self.dest) + setattr(namespace, _EXPLICIT_ATTR, explicit) + + +@dataclass(frozen=True) +class ResolvedCliAnalysis: + settings: AnalysisSettings + include_excel: bool + recursive: bool + preset_values: dict[str, object] + + +def _tracked_value( + parser: argparse.ArgumentParser, + *option_strings: str, + dest: str | None = None, + **kwargs: Any, +) -> None: + parser.add_argument(*option_strings, dest=dest, action=_TrackedStore, **kwargs) + + +def _tracked_bool( + parser: argparse.ArgumentParser, + *, + positive: str, + negative: str, + dest: str, + default: bool, + positive_value: bool, + help: str, + negative_help: str, +) -> None: + group = parser.add_mutually_exclusive_group() + group.add_argument( + positive, + dest=dest, + action=_TrackedConst, + const=positive_value, + default=default, + help=help, + ) + group.add_argument( + negative, + dest=dest, + action=_TrackedConst, + const=not positive_value, + help=negative_help, + ) + + +def add_analysis_options( + parser: argparse.ArgumentParser, + *, + include_preset: bool = True, + include_output_controls: bool = True, + include_json: bool = True, + include_recursive: bool = True, +) -> None: + """Add shared analysis arguments with reliable per-parse explicit tracking.""" + + if include_preset: + parser.add_argument("--preset", help="Load analysis parameters from a DiffractScout preset JSON file.") + _tracked_value( + parser, + "--source", + choices=("Cu Ka", "Co Ka", "Fe Ka", "Mo Ka", "Ag Ka", "Custom"), + default="Cu Ka", + ) + radiation = parser.add_mutually_exclusive_group() + _tracked_value( + radiation, + "--energy-keV", + type=float, + default=None, + help="Custom X-ray energy in keV; selects energy mode.", + ) + _tracked_value( + radiation, + "--wavelength-A", + type=float, + default=None, + help="Custom wavelength in Angstrom; selects wavelength mode.", + ) + + float_options = ( + ("--two-theta-min", "two_theta_min", 5.0, "Minimum 2theta in degrees."), + ("--two-theta-max", "two_theta_max", 120.0, "Maximum 2theta in degrees."), + ("--step", "step", 0.02, "Display-profile grid spacing in degrees."), + ("--fwhm", "fwhm", 0.15, "Display-profile FWHM in degrees."), + ("--eta", "eta", 0.5, "Pseudo-Voigt Lorentzian fraction in [0, 1]."), + ) + for option, dest, default, help_text in float_options: + _tracked_value(parser, option, dest=dest, type=float, default=default, help=help_text) + + _tracked_bool( + parser, + positive="--elasticity", + negative="--no-elasticity", + dest="no_elasticity", + default=False, + positive_value=False, + help="Include paired elastic data (default).", + negative_help="Do not discover, copy, or calculate paired elastic data.", + ) + _tracked_value( + parser, + "--max-profile-points", + type=int, + default=1_000_000, + help="Safety limit for the generated display-profile grid.", + ) + _tracked_value( + parser, + "--max-reflection-estimate", + type=int, + default=2_000_000, + help="Safety limit for reciprocal-lattice candidate generation.", + ) + + for option, clear_option, dest, help_text in ( + ("--d-min", "--clear-d-min", "d_min", "Minimum d-spacing filter in Angstrom."), + ("--d-max", "--clear-d-max", "d_max", "Maximum d-spacing filter in Angstrom."), + ): + group = parser.add_mutually_exclusive_group() + group.add_argument(option, type=float, default=None, dest=dest, action=_TrackedStore, help=help_text) + group.add_argument( + clear_option, + dest=dest, + action=_TrackedConst, + const=None, + help=f"Clear the {dest.replace('_', '-')} filter from the loaded preset.", + ) + + _tracked_value( + parser, + "--profile-model", + choices=("pseudo_voigt", "gaussian", "lorentzian"), + default="pseudo_voigt", + help="Display-profile lineshape model.", + ) + _tracked_value( + parser, + "--pattern-axis", + choices=("two_theta", "d_spacing", "q", "g"), + default="two_theta", + help=( + "Selected x coordinate in pattern_profiles.csv and Excel. " + "Figures remain on 2theta in v0.4.0." + ), + ) + _tracked_bool( + parser, + positive="--figures", + negative="--no-figures", + dest="figures", + default=False, + positive_value=True, + help="Write 2theta figures (SVG and PNG in result bundles).", + negative_help="Skip 2theta figure files in result bundles.", + ) + _tracked_value( + parser, + "--figure-preset", + default="publication", + help="Presentation-only figure style; this setting is not stored in analysis presets.", + ) + _tracked_bool( + parser, + positive="--lab-views", + negative="--no-lab-views", + dest="no_lab_views", + default=False, + positive_value=False, + help="Include laboratory convenience views (default).", + negative_help="Skip laboratory convenience views in the result bundle.", + ) + _tracked_bool( + parser, + positive="--patterns", + negative="--no-patterns", + dest="no_patterns", + default=False, + positive_value=False, + help="Include continuous powder-pattern series (default).", + negative_help="Skip continuous powder-pattern series in exports.", + ) + _tracked_bool( + parser, + positive="--excel", + negative="--no-excel", + dest="no_excel", + default=False, + positive_value=False, + help="Write results.xlsx (default).", + negative_help="Skip results.xlsx; quick-export requires a bundle-directory output.", + ) + if include_recursive: + _tracked_bool( + parser, + positive="--recursive", + negative="--no-recursive", + dest="no_recursive", + default=False, + positive_value=False, + help="Search input directories recursively (default).", + negative_help="Do not search input directories recursively.", + ) + if include_output_controls: + parser.add_argument("--overwrite", action="store_true", help="Replace only an existing DiffractScout output bundle.") + if include_json: + parser.add_argument("--json", action="store_true", help="Print the final summary as JSON.") + + +def _default_preset_values() -> dict[str, object]: + settings = AnalysisSettings() + return { + "input_mode": settings.input_mode, + "source_preset": settings.source_preset, + "radiation_value": None, + "two_theta_min": settings.two_theta_min_deg, + "two_theta_max": settings.two_theta_max_deg, + "step": settings.step_deg, + "fwhm": settings.fwhm_deg, + "eta": settings.profile_eta, + "d_min_A": settings.d_min_A, + "d_max_A": settings.d_max_A, + "profile_model": settings.profile_model, + "pattern_axis": settings.pattern_axis, + "max_profile_points": settings.max_profile_points, + "max_reflection_estimate": settings.max_reflection_estimate, + "include_excel": True, + "include_elasticity": settings.include_elasticity, + "export_lab_views": settings.export_lab_views, + "include_patterns": settings.include_patterns, + "include_figures": settings.include_figures, + "local_recursive": True, + } + + +def _settings_from_values(values: dict[str, object], figure_preset: str) -> AnalysisSettings: + mode = str(values["input_mode"]) + source = str(values["source_preset"]) + radiation = values["radiation_value"] + wavelength = None + energy = None + if radiation is not None: + if mode == "energy": + energy = float(radiation) + elif mode == "wavelength" or (mode == "source" and source == "Custom"): + wavelength = float(radiation) + settings = AnalysisSettings( + input_mode=mode, # type: ignore[arg-type] + source_preset=source, + wavelength_A=wavelength, + energy_keV=energy, + two_theta_min_deg=float(values["two_theta_min"]), + two_theta_max_deg=float(values["two_theta_max"]), + step_deg=float(values["step"]), + fwhm_deg=float(values["fwhm"]), + profile_eta=float(values["eta"]), + include_elasticity=bool(values["include_elasticity"]), + max_profile_points=int(values["max_profile_points"]), + max_reflection_estimate=int(values["max_reflection_estimate"]), + d_min_A=values["d_min_A"], # type: ignore[arg-type] + d_max_A=values["d_max_A"], # type: ignore[arg-type] + profile_model=values["profile_model"], # type: ignore[arg-type] + pattern_axis=values["pattern_axis"], # type: ignore[arg-type] + include_figures=bool(values["include_figures"]), + figure_preset=figure_preset, + export_lab_views=bool(values["export_lab_views"]), + include_patterns=bool(values["include_patterns"]), + ) + return settings + + +def resolve_cli_analysis(args: argparse.Namespace) -> ResolvedCliAnalysis: + """Apply an optional preset, then replace only explicitly supplied options.""" + + explicit = set(getattr(args, _EXPLICIT_ATTR, ())) + values = _default_preset_values() + preset_path = getattr(args, "preset", None) + if preset_path: + loaded = load_analysis_preset(preset_path) + for field in ("radiation_value", "d_min_A", "d_max_A"): + if loaded[field] == "": + loaded[field] = None + values.update(loaded) + + if "energy_keV" in explicit: + values["input_mode"] = "energy" + values["radiation_value"] = args.energy_keV + if "source" in explicit: + values["source_preset"] = args.source + elif "wavelength_A" in explicit: + values["input_mode"] = "wavelength" + values["radiation_value"] = args.wavelength_A + if "source" in explicit: + values["source_preset"] = args.source + elif "source" in explicit: + values["source_preset"] = args.source + if args.source == "Custom": + if values["input_mode"] == "wavelength" or ( + values["input_mode"] == "source" and values["source_preset"] == "Custom" + ): + if values["radiation_value"] is None: + raise ValueError( + "Custom source requires --wavelength-A or --energy-keV. " + "For a saved energy-mode preset, pass --wavelength-A to switch units." + ) + else: + raise ValueError( + "Custom source requires a wavelength; pass --wavelength-A. " + "An energy-mode preset cannot be reinterpreted as Angstrom." + ) + values["input_mode"] = "source" + else: + values["input_mode"] = "source" + values["radiation_value"] = None + + scalar_fields = { + "two_theta_min": "two_theta_min", + "two_theta_max": "two_theta_max", + "step": "step", + "fwhm": "fwhm", + "eta": "eta", + "max_profile_points": "max_profile_points", + "max_reflection_estimate": "max_reflection_estimate", + "d_min": "d_min_A", + "d_max": "d_max_A", + "profile_model": "profile_model", + "pattern_axis": "pattern_axis", + } + for dest, field in scalar_fields.items(): + if dest in explicit: + values[field] = getattr(args, dest) + + bool_fields = { + "no_elasticity": ("include_elasticity", lambda value: not value), + "no_lab_views": ("export_lab_views", lambda value: not value), + "no_patterns": ("include_patterns", lambda value: not value), + "figures": ("include_figures", bool), + "no_excel": ("include_excel", lambda value: not value), + "no_recursive": ("local_recursive", lambda value: not value), + } + for dest, (field, convert) in bool_fields.items(): + if dest in explicit: + values[field] = convert(getattr(args, dest)) + + figure_preset = getattr(args, "figure_preset", "publication") + if "source" not in explicit and "energy_keV" not in explicit and "wavelength_A" not in explicit: + if values["input_mode"] == "source" and values["source_preset"] == "Custom" and values["radiation_value"] is None: + raise ValueError("Custom source preset is missing its wavelength value.") + settings = _settings_from_values(values, figure_preset) + return ResolvedCliAnalysis( + settings=settings, + include_excel=bool(values["include_excel"]), + recursive=bool(values["local_recursive"]), + preset_values=values, + ) + + +def save_cli_preset( + path: str | Path, + values: dict[str, object], + *, + overwrite: bool = False, +) -> Path: + """Save a CLI preset while requiring explicit permission to replace files.""" + + target = Path(path).expanduser() + if target.is_symlink(): + raise ValueError(f"Refusing to write a preset through a symbolic link: {target}") + resolved_parent = target.parent.resolve() + if any((directory / "manifest.json").is_file() for directory in (resolved_parent, *resolved_parent.parents)): + raise ValueError(f"Refusing to save an analysis preset inside a result bundle: {target}") + if target.exists(): + if not target.is_file(): + raise ValueError(f"Preset target is not a regular file: {target}") + if not overwrite: + raise FileExistsError(f"Preset already exists: {target}. Pass --overwrite to replace it.") + form_values = dict(values) + for field in ("radiation_value", "d_min_A", "d_max_A"): + if form_values[field] is None: + form_values[field] = "" + return save_analysis_preset(target, form_values, overwrite=overwrite) + + +def show_cli_preset(path: str | Path) -> dict[str, object]: + """Load and strictly validate a preset for `preset show`.""" + + values = load_analysis_preset(path) + return {key: (None if value == "" else value) for key, value in values.items()} diff --git a/src/diffractscout/excel_overview.py b/src/diffractscout/excel_overview.py new file mode 100644 index 0000000..eaf2bd7 --- /dev/null +++ b/src/diffractscout/excel_overview.py @@ -0,0 +1,492 @@ +"""Build a compact, navigable overview sheet for exported workbooks.""" + +from __future__ import annotations + +import math +from typing import Any +import unicodedata + +from openpyxl import Workbook +from openpyxl.styles import Alignment, Border, Font, PatternFill, Side +from openpyxl.utils.cell import quote_sheetname +from openpyxl.worksheet.hyperlink import Hyperlink +from openpyxl.worksheet.worksheet import Worksheet + + +_NAVY = "16324F" +_TEAL = "167D8D" +_PALE_BLUE = "F2F6FA" +_TEXT = "203040" +_WHITE = "FFFFFF" +_FONT_NAME = "Microsoft YaHei" +_TITLE_FONT = Font(name=_FONT_NAME, size=18, bold=True, color=_WHITE) +_SECTION_FONT = Font(name=_FONT_NAME, size=11, bold=True, color=_WHITE) +_HEADER_FONT = Font(name=_FONT_NAME, bold=True, color=_WHITE) +_BODY_FONT = Font(name=_FONT_NAME, size=10, color=_TEXT) +_LINK_FONT = Font(name=_FONT_NAME, color=_TEAL, underline="single") +_TITLE_ALIGNMENT = Alignment(vertical="center") +_LEFT_ALIGNMENT = Alignment(horizontal="left", vertical="center") +_CENTER_ALIGNMENT = Alignment(horizontal="center", vertical="center") +_CENTER_WRAP_ALIGNMENT = Alignment(horizontal="center", vertical="center", wrap_text=True) +_TOP_WRAP_ALIGNMENT = Alignment(vertical="top", wrap_text=True) +_DEFAULT_ALIGNMENT = Alignment() +_NAVY_FILL = PatternFill("solid", fgColor=_NAVY) +_TEAL_FILL = PatternFill("solid", fgColor=_TEAL) +_PALE_BLUE_FILL = PatternFill("solid", fgColor=_PALE_BLUE) +_WHITE_FILL = PatternFill("solid", fgColor=_WHITE) +_BOTTOM_BORDER = Border(bottom=Side(style="hair", color="D5DEE7")) + + +def _safe_text(value: Any) -> str: + """Keep untrusted workbook text literal instead of letting Excel parse formulas.""" + + text = str(value) + if text and text[0] in {"=", "+", "-", "@", "\t", "\r", "\n"}: + return "'" + text + return text + + +def _summary_values(summary: list[dict[str, Any]]) -> dict[str, Any]: + values: dict[str, Any] = {} + for row in summary: + key = row.get("key") + if isinstance(key, str): + values[key] = row.get("value") + return values + + +def _summary_text(value: Any, *, missing: str) -> str: + if value is None or value == "": + return missing + if isinstance(value, bool): + return "是" if value else "否" + if isinstance(value, (list, tuple)): + if len(value) != 2 or any(item is None for item in value): + return missing + left, right = (_summary_scalar(item) for item in value) + return f"{left}–{right}°" + return _summary_scalar(value) + + +def _summary_scalar(value: Any) -> str: + if isinstance(value, bool): + return "是" if value else "否" + if isinstance(value, float) and math.isfinite(value): + return format(value, ".5g") + return _safe_text(value) + + +def _significant_number_format(value: int | float, digits: int = 5) -> str: + """Return an Excel format with at most ``digits`` significant figures.""" + + numeric = float(value) + if not math.isfinite(numeric) or numeric == 0: + return "0" + exponent = math.floor(math.log10(abs(numeric))) + decimal_places = max(0, digits - 1 - exponent) + if exponent >= digits or decimal_places > 12: + return "0.0000E+00" + return "0" if decimal_places == 0 else "0." + ("#" * decimal_places) + + +def _wrapped_line_count(value: Any, width: float) -> int: + """Estimate wrapped lines using Excel column units and Unicode character width.""" + + text = str(value or "").replace("\r\n", "\n").replace("\r", "\n") + capacity = max(1.0, float(width) - 1.0) + lines = 1 + used = 0.0 + for character in text: + if character == "\n": + lines += 1 + used = 0.0 + continue + if unicodedata.combining(character): + advance = 0.0 + else: + advance = 2.0 if unicodedata.east_asian_width(character) in {"W", "F"} else 1.0 + if used > 0 and used + advance > capacity: + lines += 1 + used = 0.0 + used += advance + return lines + + +def _estimated_row_height(values_and_widths: tuple[tuple[Any, float], ...]) -> float: + lines = max((_wrapped_line_count(value, width) for value, width in values_and_widths), default=1) + return min(409.5, max(30.0, lines * 15.0 + 6.0)) + + +def _write_numeric_or_text(cell: Any, value: Any, *, missing: str) -> None: + if isinstance(value, (int, float)) and not isinstance(value, bool): + numeric = float(value) + if math.isfinite(numeric): + cell.value = value + cell.number_format = _significant_number_format(numeric) + cell.alignment = _TITLE_ALIGNMENT + return + cell.value = _summary_scalar(value) if value is not None else missing + + +def _display_radiation_source(value: Any, *, lab_views: bool) -> str: + if value is None or value == "": + return "无数据" if lab_views else "No data" + source = str(value) + if source.startswith("source_preset:"): + source = source.split(":", 1)[1] + sources = { + "Cu Ka": "Cu Kα", + "Co Ka": "Co Kα", + "Fe Ka": "Fe Kα", + "Mo Ka": "Mo Kα", + "Ag Ka": "Ag Kα", + } + if source in sources: + return sources[source] + if source == "wavelength_A": + return "波长输入" if lab_views else "Wavelength input" + if source == "energy_keV": + return "能量输入" if lab_views else "Energy input" + if source == "custom_source_wavelength": + return "自定义波长" if lab_views else "Custom wavelength" + return _safe_text(source) + + +def _display_profile_model(value: Any, *, lab_views: bool, missing: str) -> str: + if value is None or value == "": + return missing + model = str(value).casefold().replace("-", "_") + names = { + "pseudo_voigt": ("伪Voigt", "pseudo-Voigt"), + "gaussian": ("高斯", "Gaussian"), + "lorentzian": ("洛伦兹", "Lorentzian"), + } + if model in names: + return names[model][0 if lab_views else 1] + return _safe_text(value) + + +def _display_input_mode(value: Any, *, lab_views: bool, missing: str) -> str: + if value is None or value == "": + return missing + modes = { + "source": ("预设光源", "Source preset"), + "wavelength": ("波长", "Wavelength"), + "energy": ("能量", "Photon energy"), + } + mode = str(value).casefold() + if mode in modes: + return modes[mode][0 if lab_views else 1] + return _safe_text(value) + + +def add_overview_sheet( + workbook: Workbook, + *, + summary: list[dict[str, Any]], + phases: list[dict[str, Any]], + diagnostics: list[dict[str, Any]], + lab_views: bool, +) -> Worksheet: + """Add a concise overview using only the supplied export rows and summary. + + The caller should invoke this after creating the workbook's data sheets so + the navigation section can link only to sheets that actually exist. + """ + + title = "结果概览" if lab_views else "Overview" + if title in workbook.sheetnames: + raise ValueError(f"Workbook already contains an overview sheet named {title!r}.") + + labels = { + "title": "本次衍射分析结果概览" if lab_views else "Diffraction analysis overview", + "summary": "导出摘要" if lab_views else "Export summary", + "run": "运行条件" if lab_views else "Run conditions", + "phases": "各物相摘要" if lab_views else "Phase summary", + "navigation": "工作表导航" if lab_views else "Worksheet navigation", + "metric": "指标" if lab_views else "Metric", + "count": "数量" if lab_views else "Count", + "phase_count": "物相数" if lab_views else "Phases", + "reflection_count": "反射数" if lab_views else "Reflections", + "error_count": "错误数" if lab_views else "Errors", + "warning_count": "警告数" if lab_views else "Warnings", + "diagnostic_count": "诊断数" if lab_views else "Diagnostics", + "candidate_count": "候选数" if lab_views else "Candidates", + "parameter": "参数" if lab_views else "Parameter", + "value": "值" if lab_views else "Value", + "wavelength": "波长 (Å)" if lab_views else "Wavelength (Å)", + "energy": "能量 (keV)" if lab_views else "Energy (keV)", + "radiation": "预设光源" if lab_views else "Radiation source", + "range": "有效 2θ 范围" if lab_views else "Effective 2θ range", + "analysis_range": "计算 2θ 范围" if lab_views else "Analysis 2θ range", + "profile_range": "谱线采样范围" if lab_views else "Profile sampled range", + "profile": "线型模型" if lab_views else "Profile model", + "input_mode": "输入方式" if lab_views else "Input mode", + "phase_name": "物相名称" if lab_views else "Phase name", + "formula": "化学式" if lab_views else "Formula", + "space_group": "空间群" if lab_views else "Space group", + "peak_count": "反射数" if lab_views else "Reflections", + "elastic": "弹性数据状态" if lab_views else "Elastic data status", + "no_phases": "无分析物相" if lab_views else "No analyzed phases", + "no_data": "无数据" if lab_views else "No data", + "sheet": "工作表" if lab_views else "Worksheet", + "contents": "内容" if lab_views else "Contents", + } + + values = _summary_values(summary) + error_count = sum(str(row.get("level", "")).casefold() == "error" for row in diagnostics) + warning_count = sum( + str(row.get("level", "")).casefold() == "warning" for row in diagnostics + ) + reflection_count = sum( + count + for phase in phases + if isinstance((count := phase.get("reflection_count")), int) + and not isinstance(count, bool) + and count >= 0 + ) + if not any( + isinstance(phase.get("reflection_count"), int) + and not isinstance(phase.get("reflection_count"), bool) + for phase in phases + ): + reported_reflections = values.get("reflection_count") + if isinstance(reported_reflections, int) and not isinstance(reported_reflections, bool): + reflection_count = reported_reflections + + worksheet = workbook.create_sheet(title=title) + worksheet.sheet_view.showGridLines = False + worksheet.sheet_view.zoomScale = 90 + + worksheet.merge_cells("A1:E1") + heading = worksheet["A1"] + heading.value = _safe_text(labels["title"]) + heading.font = _TITLE_FONT + heading.fill = _NAVY_FILL + heading.alignment = _TITLE_ALIGNMENT + worksheet.row_dimensions[1].height = 36 + + row = 3 + + def section(label: str) -> int: + nonlocal row + section_row = row + worksheet.merge_cells(start_row=row, start_column=1, end_row=row, end_column=5) + cell = worksheet.cell(row=row, column=1, value=_safe_text(label)) + cell.font = _SECTION_FONT + cell.fill = _TEAL_FILL + cell.alignment = _TITLE_ALIGNMENT + worksheet.row_dimensions[row].height = 23 + row += 1 + return section_row + + section(labels["summary"]) + for column, value in enumerate( + (labels["metric"], labels["count"], labels["metric"], labels["count"]), + start=1, + ): + cell = worksheet.cell(row=row, column=column, value=_safe_text(value)) + cell.font = _HEADER_FONT + cell.fill = _NAVY_FILL + cell.alignment = _CENTER_ALIGNMENT + row += 1 + + candidate_count = values.get("candidate_count") + metric_rows = ( + (labels["phase_count"], len(phases), labels["reflection_count"], reflection_count), + (labels["error_count"], error_count, labels["warning_count"], warning_count), + ( + labels["diagnostic_count"], + len(diagnostics), + labels["candidate_count"], + candidate_count + if isinstance(candidate_count, int) and not isinstance(candidate_count, bool) + else labels["no_data"], + ), + ) + for index, metric_row in enumerate(metric_rows): + for column, value in enumerate(metric_row, start=1): + cell = worksheet.cell(row=row, column=column) + cell.value = _safe_text(value) if isinstance(value, str) else value + cell.alignment = _LEFT_ALIGNMENT if column % 2 else _CENTER_ALIGNMENT + cell.fill = _PALE_BLUE_FILL if index % 2 == 0 else _WHITE_FILL + row += 1 + + row += 1 + section(labels["run"]) + worksheet.merge_cells(start_row=row, start_column=2, end_row=row, end_column=5) + for column, value in ((1, labels["parameter"]), (2, labels["value"])): + cell = worksheet.cell(row=row, column=column, value=_safe_text(value)) + cell.font = _HEADER_FONT + cell.fill = _NAVY_FILL + cell.alignment = _CENTER_ALIGNMENT + row += 1 + missing = labels["no_data"] + wavelength = values.get("effective_wavelength_A") + energy = values.get("effective_energy_keV") + radiation = values.get("effective_radiation_source") + requested_source = values.get("source_preset_applied") + if radiation is None: + radiation = requested_source + run_rows = ( + (labels["wavelength"], wavelength, True), + (labels["energy"], energy, True), + (labels["radiation"], _display_radiation_source(radiation, lab_views=lab_views), False), + (labels["range"], _summary_text(values.get("effective_two_theta_range_deg"), missing=missing), False), + (labels["analysis_range"], _summary_text(values.get("analysis_two_theta_range_deg"), missing=missing), False), + (labels["profile_range"], _summary_text(values.get("profile_sampled_two_theta_range_deg"), missing=missing), False), + (labels["profile"], _display_profile_model(values.get("profile_model"), lab_views=lab_views, missing=missing), False), + ( + labels["input_mode"], + _display_input_mode(values.get("input_mode"), lab_views=lab_views, missing=missing), + False, + ), + ) + run_value_width = sum((24, 23, 16, 25)) + for index, (parameter, value, numeric) in enumerate(run_rows): + worksheet.merge_cells(start_row=row, start_column=2, end_row=row, end_column=5) + label_cell = worksheet.cell(row=row, column=1, value=_safe_text(parameter)) + value_cell = worksheet.cell(row=row, column=2) + if numeric: + _write_numeric_or_text(value_cell, value, missing=missing) + else: + value_cell.value = _safe_text(value) + label_cell.alignment = _TOP_WRAP_ALIGNMENT + value_cell.alignment = _TOP_WRAP_ALIGNMENT + fill = _PALE_BLUE_FILL if index % 2 == 0 else _WHITE_FILL + label_cell.fill = fill + value_cell.fill = fill + worksheet.row_dimensions[row].height = _estimated_row_height( + ((label_cell.value, 25), (value_cell.value, run_value_width)) + ) + row += 1 + + row += 1 + section(labels["phases"]) + phase_headers = ( + labels["phase_name"], + labels["formula"], + labels["space_group"], + labels["peak_count"], + labels["elastic"], + ) + for column, value in enumerate(phase_headers, start=1): + cell = worksheet.cell(row=row, column=column, value=_safe_text(value)) + cell.font = _HEADER_FONT + cell.fill = _NAVY_FILL + cell.alignment = _CENTER_WRAP_ALIGNMENT + worksheet.row_dimensions[row].height = 30 + row += 1 + if phases: + for index, phase in enumerate(phases): + symbol = phase.get("space_group_symbol") + number = phase.get("space_group_number") + if symbol not in (None, "") and number not in (None, ""): + space_group = f"{_safe_text(symbol)} ({_safe_text(number)})" + elif symbol not in (None, ""): + space_group = _safe_text(symbol) + elif number not in (None, ""): + space_group = _safe_text(number) + else: + space_group = missing + phase_values = ( + phase.get("phase_name"), + phase.get("formula"), + space_group, + phase.get("reflection_count"), + phase.get("elastic_status"), + ) + for column, value in enumerate(phase_values, start=1): + cell = worksheet.cell(row=row, column=column) + if isinstance(value, str): + cell.value = _safe_text(value) + elif value is None: + cell.value = missing + else: + cell.value = value + cell.alignment = _TOP_WRAP_ALIGNMENT + if index % 2 == 1: + cell.fill = _PALE_BLUE_FILL + worksheet.row_dimensions[row].height = _estimated_row_height( + tuple( + (cell.value, width) + for cell, width in zip( + worksheet[row][:5], (25, 24, 23, 16, 25), strict=True + ) + ) + ) + row += 1 + else: + worksheet.cell(row=row, column=1, value=_safe_text(labels["no_phases"])) + row += 1 + + row += 1 + section(labels["navigation"]) + worksheet.merge_cells(start_row=row, start_column=2, end_row=row, end_column=5) + for column, value in ((1, labels["sheet"]), (2, labels["contents"])): + cell = worksheet.cell(row=row, column=column, value=_safe_text(value)) + cell.font = _HEADER_FONT + cell.fill = _NAVY_FILL + cell.alignment = _CENTER_ALIGNMENT + row += 1 + descriptions = { + "Summary": "导出设置与来源摘要" if lab_views else "Export settings and provenance summary", + "Peaks": "完整反射数据表" if lab_views else "Full reflection table", + "Phases": "物相结构与分析参数" if lab_views else "Phase structures and analysis parameters", + "Elasticity": "弹性张量与来源信息" if lab_views else "Elastic tensors and provenance", + "Candidates": "候选材料" if lab_views else "Candidate materials", + "Downloads": "文件下载状态" if lab_views else "Download status", + "Diagnostics": "错误与警告记录" if lab_views else "Errors and warnings", + "Patterns": "衍射谱线采样数据" if lab_views else "Diffraction profile samples", + "推荐峰表": "便于筛选的中文反射表" if lab_views else "Chinese recommended-reflection view", + "使用说明": "字段与使用说明" if lab_views else "Field and usage guide", + } + for index, sheet_title in enumerate(workbook.sheetnames): + if sheet_title == title: + continue + name_cell = worksheet.cell(row=row, column=1, value=_safe_text(sheet_title)) + name_cell.hyperlink = Hyperlink( + ref=name_cell.coordinate, + location=f"{quote_sheetname(sheet_title)}!A1", + ) + name_cell.font = _LINK_FONT + name_cell.alignment = _TOP_WRAP_ALIGNMENT + description = descriptions.get( + sheet_title, + "各反射与该物相相关数据" if lab_views else "Reflection data for this phase", + ) + worksheet.merge_cells(start_row=row, start_column=2, end_row=row, end_column=5) + content_cell = worksheet.cell(row=row, column=2, value=_safe_text(description)) + content_cell.alignment = _TOP_WRAP_ALIGNMENT + worksheet.row_dimensions[row].height = _estimated_row_height( + ((name_cell.value, 25), (content_cell.value, sum((24, 23, 16, 25)))) + ) + if index % 2 == 1: + for cell in worksheet[row][:2]: + cell.fill = _PALE_BLUE_FILL + row += 1 + + for cells in worksheet.iter_rows(min_row=1, max_row=row - 1, min_col=1, max_col=5): + for cell in cells: + if cell.row > 1 and cell.value is not None: + cell.border = _BOTTOM_BORDER + if cell.alignment == _DEFAULT_ALIGNMENT: + cell.alignment = _TOP_WRAP_ALIGNMENT + if cell.font.name != _FONT_NAME: + cell.font = _BODY_FONT + + for column, width in {"A": 25, "B": 24, "C": 23, "D": 16, "E": 25}.items(): + worksheet.column_dimensions[column].width = width + worksheet.freeze_panes = "A4" + worksheet.print_area = f"A1:E{row - 1}" + worksheet.print_title_rows = "1:1" + worksheet.page_setup.orientation = worksheet.ORIENTATION_LANDSCAPE + worksheet.page_setup.paperSize = worksheet.PAPERSIZE_A4 + worksheet.page_setup.fitToWidth = 1 + worksheet.page_setup.fitToHeight = 0 + worksheet.sheet_properties.pageSetUpPr.fitToPage = True + worksheet.page_margins.left = 0.3 + worksheet.page_margins.right = 0.3 + worksheet.page_margins.top = 0.5 + worksheet.page_margins.bottom = 0.5 + return worksheet diff --git a/src/diffractscout/excel_styles.py b/src/diffractscout/excel_styles.py new file mode 100644 index 0000000..67a9eeb --- /dev/null +++ b/src/diffractscout/excel_styles.py @@ -0,0 +1,617 @@ +"""Reusable, presentation-only styles for exported Excel workbooks.""" + +from __future__ import annotations + +from math import ceil +import re +import unicodedata +from typing import Mapping, Sequence + +from openpyxl.formatting.rule import DataBarRule +from openpyxl.styles import Alignment, Border, Font, PatternFill, Side +from openpyxl.utils import get_column_letter +from openpyxl.worksheet.worksheet import Worksheet + +from .export_views import BEGINNER_PEAK_HEADERS_ZH + + +_FONT_NAME = "Microsoft YaHei" +_BODY_FONT = Font(name=_FONT_NAME, size=10, color="243447") +_HEADER_FONT = Font(name=_FONT_NAME, size=10, bold=True, color="FFFFFF") +_SECTION_FONT = Font(name=_FONT_NAME, size=11, bold=True, color="16324F") +_HEADER_ALIGNMENT = Alignment(horizontal="center", vertical="center", wrap_text=True) +_BODY_ALIGNMENT = Alignment(vertical="center") +_WRAP_ALIGNMENT = Alignment(vertical="top", wrap_text=True) +_HEADER_BOTTOM = Border(bottom=Side(style="medium", color="FFFFFF")) + +_GROUP_COLORS = { + "identity": "24476B", + "indices": "52628A", + "geometry": "28677A", + "intensity": "806017", + "elasticity": "32664F", + "quality": "714A79", + "metadata": "596779", +} +_GROUP_FILLS = { + name: PatternFill("solid", fgColor=color) for name, color in _GROUP_COLORS.items() +} +_BAND_FILL = PatternFill("solid", fgColor="F1F5F9") +_SECTION_FILL = PatternFill("solid", fgColor="E4ECF4") +_OVERVIEW_HEADER_FILL = PatternFill("solid", fgColor="24476B") +_ZH_TO_CANONICAL = {label: canonical for label, canonical in BEGINNER_PEAK_HEADERS_ZH.items()} + +_IDENTITY_HEADERS = { + "phase_name", + "cif_name", + "formula", + "space_group", + "space_group_symbol", + "material_id", + "物相名称", + "CIF文件", + "化学式", + "空间群", +} +_INDEX_HEADERS = {"h", "k", "i", "l", "hkl", "family_label", "multiplicity"} +_INDEX_LABELS = {"晶面指标", "晶面族", "多重度"} +_GEOMETRY_HEADERS = { + "d_spacing_A", + "d_A", + "theta_deg", + "two_theta_deg", + "two_theta_cu_ka_deg", + "q_invA", + "g_invA", + "a_A", + "b_A", + "c_A", + "alpha_deg", + "beta_deg", + "gamma_deg", + "cell_volume_A3", + "wavelength_A", + "energy_keV", + "d間距_Å", + "θ_deg", + "2θ_当前_deg", + "2θ_CuKa_deg", + "q_1/Å", + "g_1/Å", + "波长_Å", + "能量_keV", + "晶胞体积_Å3", +} +_INTENSITY_HEADERS = { + "normalized_intensity", + "relative_intensity", + "intensity_with_lp", + "intensity_no_lp", + "lp_factor", + "volume_normalized_intensity_with_lp", + "volume_normalized_intensity_no_lp", + "material_scattering_factor_R_hkl", + "material_scattering_factor_R_hkl_no_lp", + "inverse_R_hkl", + "inverse_R_hkl_no_lp", + "phase_relative_R_hkl_pct", + "phase_relative_R_hkl_no_lp_pct", + "rank_by_intensity", + "rank_by_R_hkl", + "rank_by_R_hkl_no_lp", + "structure_factor_sq", + "mean_structure_factor_sq_per_multiplicity", + "mean_structure_factor_abs_per_multiplicity", + "relative_intensity", + "相对强度_相内max100", + "强度排序_相内", + "强度_含LP", + "强度_无LP", + "LP因子", + "体积归一强度J_含LP_R_hkl别名", + "体积归一强度J_无LP", + "1/J_含LP", + "1/J_无LP", + "相内相对J_含LP_%", + "相内相对J_无LP_%", + "J_含LP排序", + "J_无LP排序", +} +_ELASTICITY_HEADERS = { + "young_modulus_hkl_normal_GPa", + "elastic_status", + "elastic_note", + "杨氏模量_hkl法向_GPa", + "弹性状态", + "弹性备注", +} +_QUALITY_HEADERS = { + "status", + "level", + "error", + "elasticity_status", + "elasticity_error", + "message", + "warnings", + "is_multi_family_peak", + "coincident_hkl_family_count", + "是否多族共2θ", + "共位hkl族数", +} + +# These columns remain in the workbook and retain their full values. They start +# collapsed so the main scientific columns fit on screen; each can be expanded. +_DETAIL_HEADERS = { + "cif_sha256", + "elasticity_sha256", + "data_block", + "source_url", + "methodology_url", + "raw_payload_path", + "source_metadata", + "provider_metadata", + "scientific_boundary", + "r_hkl_model_note", +} +_PEAK_DETAIL_HEADERS = { + "structure_factor_sq", + "mean_structure_factor_sq_per_multiplicity", + "mean_structure_factor_abs_per_multiplicity", + "sin_theta", + "cos_theta", + "sin_theta_over_lambda", + "sin2_theta_over_lambda2", + "elastic_note", + "formula_weight_g_mol", + "density_g_cm3", + "cell_volume_A3", + "wavelength_A", + "energy_keV", + "wavelength_source", +} +_LONG_TEXT_HEADERS = { + "cif_path", + "elasticity_path", + "source_url", + "methodology_url", + "raw_payload_path", + "source_metadata", + "provider_metadata", + "warnings", + "error", + "elasticity_error", + "elastic_note", + "scientific_boundary", + "r_hkl_model_note", + "弹性备注", + "J_R_hkl通道说明", +} +_INTEGER_HEADERS = { + "h", + "k", + "i", + "l", + "multiplicity", + "rank_by_intensity", + "rank_by_R_hkl", + "rank_by_R_hkl_no_lp", + "space_group_number", + "site_count_asymmetric", + "site_count_unit_cell", + "reflection_count", + "coincident_hkl_family_count", + "frequency", +} +_PERCENT_SCALE_HEADERS = { + "phase_relative_R_hkl_pct", + "phase_relative_R_hkl_no_lp_pct", +} +_NORMALIZED_INTENSITY_HEADERS = {"normalized_intensity", "relative_intensity"} +_RELATIVE_BAR_HEADERS = _PERCENT_SCALE_HEADERS | _NORMALIZED_INTENSITY_HEADERS +_DEGREE_HEADERS = { + "theta_deg", + "two_theta_deg", + "two_theta_cu_ka_deg", + "alpha_deg", + "beta_deg", + "gamma_deg", + "θ_deg", + "2θ_当前_deg", + "2θ_CuKa_deg", +} +_FOUR_DECIMAL_HEADERS = { + "d_spacing_A", + "d_A", + "q_invA", + "g_invA", + "wavelength_A", + "energy_keV", + "a_A", + "b_A", + "c_A", + "young_modulus_hkl_normal_GPa", + "cell_volume_A3", + "formula_weight_g_mol", + "density_g_cm3", + "q_1/Å", + "g_1/Å", + "x", +} +_SCIENTIFIC_HEADERS = { + "intensity_with_lp", + "intensity_no_lp", + "volume_normalized_intensity_with_lp", + "volume_normalized_intensity_no_lp", + "material_scattering_factor_R_hkl", + "material_scattering_factor_R_hkl_no_lp", + "inverse_R_hkl", + "inverse_R_hkl_no_lp", + "structure_factor_sq", + "mean_structure_factor_sq_per_multiplicity", + "mean_structure_factor_abs_per_multiplicity", +} + + +def cjk_display_width(value: object) -> int: + """Estimate Excel character width, counting full-width CJK characters as two.""" + + width = 0 + for character in str(value): + if unicodedata.category(character) in {"Mn", "Mc", "Me"}: + continue + width += 2 if unicodedata.east_asian_width(character) in {"F", "W"} else 1 + return width + + +def _canonical_header(header: str) -> str: + return _ZH_TO_CANONICAL.get(str(header), str(header)) + + +def number_format_for_header(header: str) -> str | None: + """Choose a display format by field meaning without changing stored values.""" + + name = _canonical_header(header) + if name in _INTEGER_HEADERS or name.endswith("_count") or name.endswith("_number"): + return "0" + if name in _PERCENT_SCALE_HEADERS: + # Values already use a 0–100 scale; Excel's percent format would multiply by 100. + return '0.00"%"' + if name in _NORMALIZED_INTENSITY_HEADERS: + return "0.00" + if name in _DEGREE_HEADERS or name.endswith("_deg"): + return "0.00000" + if name in _SCIENTIFIC_HEADERS or name in {"lp_factor", "sin_theta", "cos_theta"}: + return "0.000E+00" + if name.startswith("C") and re.fullmatch(r"C[1-6][1-6]_GPa", name): + return "0.000" + if name in _FOUR_DECIMAL_HEADERS or name.endswith("_A") or name.endswith("_invA"): + return "0.0000" + return None + + +def number_format_for_value(header: str, value: object) -> str | None: + """Use scientific notation for nonzero values that fixed decimals would erase.""" + + number_format = number_format_for_header(header) + if number_format is None or isinstance(value, bool) or not isinstance(value, (int, float)): + return number_format + if value == 0 or "E+00" in number_format or number_format == "0": + return number_format + decimal_match = re.search(r"\.(0+)", number_format) + if decimal_match is None: + return number_format + half_last_place = 0.5 * (10.0 ** -len(decimal_match.group(1))) + if abs(float(value)) >= half_last_place: + return number_format + percent_suffix = '"%"' if number_format.endswith('"%"') else "" + return f"0.000E+00{percent_suffix}" + + +def _header_group(header: str) -> str: + name = _canonical_header(header) + low = name.lower() + if name in _IDENTITY_HEADERS: + return "identity" + if name in _INDEX_HEADERS or name in _INDEX_LABELS: + return "indices" + if name in _GEOMETRY_HEADERS or name.endswith(("_deg", "_invA", "_A")): + return "geometry" + if name in _INTENSITY_HEADERS or "intensity" in low or "r_hkl" in low or "_j_" in low: + return "intensity" + if name in _ELASTICITY_HEADERS or low.startswith("cij") or re.fullmatch(r"C[1-6][1-6]_GPa", name): + return "elasticity" + if name in _QUALITY_HEADERS or low.endswith(("_status", "_error", "_warning", "_warnings")): + return "quality" + if name in _DETAIL_HEADERS or any( + token in low for token in ("sha256", "metadata", "_path", "_url", "boundary", "note") + ): + return "metadata" + return "quality" + + +def _column_width(header: str, samples: Sequence[object]) -> float: + name = _canonical_header(header) + if name in _LONG_TEXT_HEADERS or any( + token in name.lower() for token in ("_url", "_path", "metadata", "boundary", "note") + ): + return 38.0 + header_width = cjk_display_width(header) + 2 + sample_width = max((cjk_display_width(sample) for sample in samples), default=0) + 2 + width = max(header_width, sample_width) + if name == "phase_name": + return min(max(width, 20), 32) + if name == "cif_name": + return min(max(width, 20), 32) + if name in {"cif_sha256", "elasticity_sha256"}: + return 18.0 + if name in _INDEX_HEADERS: + return min(max(width, 10), 20) + if name in _GEOMETRY_HEADERS or name.endswith(("_deg", "_invA", "_A")): + return min(max(width, 11), 18) + if name in _INTENSITY_HEADERS or "intensity" in name.lower() or "r_hkl" in name.lower(): + return min(max(width, 13), 24) + return min(max(width, 11), 28) + + +def _is_wrapped_text_column(header: str, sheet_title: str) -> bool: + name = _canonical_header(header) + if sheet_title == "Patterns" and name in {"phase_name", "cif_name"}: + return False + if sheet_title == "Diagnostics" and name in {"item", "message"}: + return True + return name in {"phase_name", "cif_name"} or name in _LONG_TEXT_HEADERS or any( + token in name.lower() for token in ("_url", "_path", "metadata", "boundary", "note") + ) + + +def _identity_row_height( + values: Sequence[object], + headers: Sequence[str], + widths: Mapping[str, float], +) -> float: + line_counts = [1] + for header, value in zip(headers, values, strict=False): + if _canonical_header(str(header)) not in {"phase_name", "cif_name"}: + continue + width = max(widths.get(str(header), 1.0) - 2.0, 1.0) + line_counts.append(ceil(cjk_display_width(value) / width)) + return min(120.0, max(19.0, 4.0 + max(line_counts) * 15.0)) + + +def _wrapped_line_count(value: object, width: float) -> int: + text = str(value or "") + segments = text.splitlines() or [""] + capacity = max(width - 2.0, 1.0) + return sum(max(1, ceil(cjk_display_width(segment) / capacity)) for segment in segments) + + +def _detail_column(header: str, sheet_title: str) -> bool: + name = _canonical_header(header) + if name in _DETAIL_HEADERS: + return True + if name in {"cif_path", "elasticity_path"}: + return sheet_title != "Downloads" + is_peak_sheet = sheet_title in {"Peaks", "推荐峰表"} or sheet_title.startswith("峰_") + return is_peak_sheet and name in _PEAK_DETAIL_HEADERS + + +def _contiguous_ranges(indices: Sequence[int]) -> list[tuple[int, int]]: + if not indices: + return [] + ordered = sorted(set(indices)) + ranges: list[tuple[int, int]] = [] + start = previous = ordered[0] + for index in ordered[1:]: + if index != previous + 1: + ranges.append((start, previous)) + start = index + previous = index + ranges.append((start, previous)) + return ranges + + +def _phase_identity(header: str) -> bool: + return _canonical_header(header) == "phase_name" + + +def _set_print_layout( + sheet: Worksheet, + *, + end_column: int, + repeat_header: bool, + print_area_end_row: int | None = None, +) -> None: + sheet.page_setup.orientation = "landscape" + sheet.page_setup.paperSize = sheet.PAPERSIZE_A4 + sheet.page_setup.fitToWidth = 1 + sheet.page_setup.fitToHeight = 0 + sheet.sheet_properties.pageSetUpPr.fitToPage = True + sheet.page_margins.left = 0.25 + sheet.page_margins.right = 0.25 + sheet.page_margins.top = 0.45 + sheet.page_margins.bottom = 0.45 + sheet.print_area = f"A1:{get_column_letter(max(end_column, 1))}{print_area_end_row or sheet.max_row}" + if repeat_header: + sheet.print_title_rows = "1:1" + + +def _apply_phase_data_bars( + sheet: Worksheet, + headers: Sequence[str], + last_row: int, +) -> None: + if last_row < 2: + return + for column_index, header in enumerate(headers, start=1): + canonical = _canonical_header(header) + if canonical not in _RELATIVE_BAR_HEADERS: + continue + column = get_column_letter(column_index) + color = "64A884" if canonical in _NORMALIZED_INTENSITY_HEADERS else "5A9BD5" + rule = DataBarRule( + start_type="num", + start_value=0, + end_type="num", + end_value=100, + color=color, + showValue=True, + ) + sheet.conditional_formatting.add(f"{column}2:{column}{last_row}", rule) + + +def style_data_sheet( + sheet: Worksheet, + headers: Sequence[str], +) -> None: + """Style an existing header-first data sheet without changing its values.""" + + sheet.sheet_view.showGridLines = False + sheet.sheet_view.zoomScale = 90 if len(headers) > 12 else 100 + has_phase_identity = any(_phase_identity(str(header)) for header in headers) + sheet.freeze_panes = "C2" if has_phase_identity and len(headers) >= 3 else "A2" + sheet.sheet_properties.outlinePr.summaryRight = True + sheet.sheet_view.showOutlineSymbols = True + last_row = max(sheet.max_row, 1) + last_column = max(len(headers), 1) + sheet.auto_filter.ref = f"A1:{get_column_letter(last_column)}{last_row}" + sheet.sheet_format.defaultRowHeight = 19 + + header_lines = 1 + identity_widths: dict[str, float] = {} + for column_index, header in enumerate(headers, start=1): + cell = sheet.cell(row=1, column=column_index) + cell.font = _HEADER_FONT + cell.fill = _GROUP_FILLS[_header_group(str(header))] + cell.alignment = _HEADER_ALIGNMENT + cell.border = _HEADER_BOTTOM + samples = [ + sheet.cell(row=row_index, column=column_index).value + for row_index in range(2, min(last_row, 81) + 1) + ] + width = _column_width(str(header), samples) + canonical_header = _canonical_header(str(header)) + if sheet.title == "Diagnostics" and canonical_header == "message": + width = 80.0 + elif sheet.title == "Diagnostics" and canonical_header == "item": + width = 32.0 + letter = get_column_letter(column_index) + dimension = sheet.column_dimensions[letter] + dimension.width = width + if _canonical_header(str(header)) in {"phase_name", "cif_name"}: + identity_widths[str(header)] = width + if _detail_column(str(header), sheet.title): + dimension.hidden = True + dimension.outlineLevel = 1 + wrap_capacity = max(width - 2.0, 1.0) + header_lines = max(header_lines, ceil(cjk_display_width(header) / wrap_capacity)) + sheet.row_dimensions[1].height = max(36, min(72, 10 + header_lines * 15)) + + hidden_columns = [ + index + for index, header in enumerate(headers, start=1) + if _detail_column(str(header), sheet.title) + ] + for _, end in _contiguous_ranges(hidden_columns): + sheet.column_dimensions[get_column_letter(end)].collapsed = True + identity_columns = [ + (index, str(header)) + for index, header in enumerate(headers) + if _canonical_header(str(header)) in {"phase_name", "cif_name"} + ] + identity_height_cache: dict[tuple[object, ...], float] = {} + diagnostic_text_columns = [ + (index, str(header)) + for index, header in enumerate(headers) + if sheet.title == "Diagnostics" + and _canonical_header(str(header)) in {"item", "message"} + ] + diagnostic_widths = { + header: 80.0 if _canonical_header(header) == "message" else 32.0 + for _, header in diagnostic_text_columns + } + + for row in sheet.iter_rows(min_row=2, max_row=last_row): + for column_index, cell in enumerate(row, start=1): + header = str(headers[column_index - 1]) if column_index <= len(headers) else "" + cell.font = _BODY_FONT + wrapped_text = _is_wrapped_text_column(header, sheet.title) + cell.alignment = _WRAP_ALIGNMENT if wrapped_text else _BODY_ALIGNMENT + if cell.row % 2 == 0: + cell.fill = _BAND_FILL + if isinstance(cell.value, (int, float)) and not isinstance(cell.value, bool): + number_format = number_format_for_value(header, cell.value) + if number_format: + cell.number_format = number_format + if identity_columns and sheet.title != "Patterns": + identity_values = tuple(row[index].value for index, _ in identity_columns) + height = identity_height_cache.get(identity_values) + if height is None: + height = _identity_row_height( + identity_values, + [header for _, header in identity_columns], + identity_widths, + ) + identity_height_cache[identity_values] = height + if height > 19.0: + sheet.row_dimensions[row[0].row].height = height + if diagnostic_text_columns: + line_count = max( + _wrapped_line_count(row[index].value, diagnostic_widths[header]) + for index, header in diagnostic_text_columns + ) + height = min(409.5, max(19.0, 4.0 + line_count * 15.0)) + if height > 19.0: + sheet.row_dimensions[row[0].row].height = height + + if last_row >= 2 and last_column: + _apply_phase_data_bars(sheet, headers, last_row) + + # The overview is the preferred print view; long peak sheets print only + # through normalized intensity while retaining every later column in Excel. + normalized_column = next( + ( + index + for index, header in enumerate(headers, start=1) + if _canonical_header(str(header)) in _NORMALIZED_INTENSITY_HEADERS + ), + None, + ) + print_column = normalized_column or last_column + _set_print_layout( + sheet, + end_column=print_column, + repeat_header=True, + print_area_end_row=last_row, + ) + + +def style_guide_sheet(sheet: Worksheet, rows: Sequence[Sequence[object]]) -> None: + """Style the two-column usage guide, including its section rows.""" + + sheet.sheet_view.showGridLines = False + sheet.sheet_view.zoomScale = 95 + sheet.freeze_panes = "A2" + sheet.sheet_format.defaultRowHeight = 21 + sheet.column_dimensions["A"].width = 30 + sheet.column_dimensions["B"].width = 90 + for cell in sheet[1]: + cell.font = _HEADER_FONT + cell.fill = _OVERVIEW_HEADER_FILL + cell.alignment = _HEADER_ALIGNMENT + cell.border = _HEADER_BOTTOM + sheet.row_dimensions[1].height = 34 + for row_index, row in enumerate(sheet.iter_rows(min_row=2), start=2): + source_index = row_index - 1 + section = ( + source_index < len(rows) + and bool(rows[source_index]) + and str(rows[source_index][0]).startswith("——") + ) + for cell in row: + cell.font = _SECTION_FONT if section else _BODY_FONT + cell.alignment = _WRAP_ALIGNMENT + if section: + cell.fill = _SECTION_FILL + if section: + sheet.row_dimensions[row_index].height = 25 + _set_print_layout(sheet, end_column=2, repeat_header=True) diff --git a/src/diffractscout/export_views.py b/src/diffractscout/export_views.py index 1559b06..2bb7aae 100644 --- a/src/diffractscout/export_views.py +++ b/src/diffractscout/export_views.py @@ -99,7 +99,7 @@ def user_guide_rows() -> list[list[str]]: return [ ["项目", "说明"], ["软件", "DiffractScout — 理论粉末 XRD 参考与候选相 scout（非实验反演）"], - ["建议阅读顺序", "① 本表 使用说明 → ② 推荐峰表（中文筛选）→ ③ Peaks 完整英文列 → ④ CSV 做脚本"], + ["建议阅读顺序", "① 结果概览 → ② 本表 使用说明 → ③ 推荐峰表（中文筛选）→ ④ Peaks 完整英文列 → ⑤ CSV 做脚本"], ["推荐峰表", "分析向中文长表：冻结首行 + 自动筛选；用「物相名称」过滤多相；列顺序为 身份→hkl→几何→强度→排序→弹性"], ["Peaks", "完整规范英文峰表（与 peak_reference.csv 列名一致）；同样冻结首行 + 自动筛选"], ["Phases", "每个 CIF/相的晶胞、空间群、占位警告、弹性配对状态"], @@ -116,7 +116,7 @@ def user_guide_rows() -> list[list[str]]: ["g_1/Å / g_invA", "g = 1/d，单位 1/Å"], [], ["—— 强度列（理论） ——", ""], - ["相对强度_相内max100 / normalized_intensity", "相内最强线标为 100 的显示归一；不可跨物相直接比绝对强度"], + ["相对强度_相内max100 / normalized_intensity", "相内最强线标为 100 的显示归一；固定 0–100 数据条仅表示各物相内部相对强度，不可跨物相比较或用于定量"], ["强度_含LP / intensity_with_lp", "多重度 × |F_xray|² × Lorentz–polarization（LP）"], ["强度_无LP / intensity_no_lp", "多重度 × |F_xray|²（不含 LP）"], ["LP因子 / lp_factor", "经典粉末 LP：(1+cos²2θ)/(sin²θ cosθ)"], diff --git a/src/diffractscout/exporters.py b/src/diffractscout/exporters.py index 042c94d..6a146f0 100644 --- a/src/diffractscout/exporters.py +++ b/src/diffractscout/exporters.py @@ -13,10 +13,10 @@ import numpy as np from openpyxl import Workbook -from openpyxl.styles import Alignment, Font, PatternFill -from openpyxl.utils import get_column_letter from .diffraction import SCIENTIFIC_BOUNDARY +from .excel_overview import add_overview_sheet +from .excel_styles import style_data_sheet, style_guide_sheet from .export_views import ( ANALYSIS_PEAK_COLUMNS, BEGINNER_PEAK_HEADERS_ZH, @@ -633,26 +633,7 @@ def _add_sheet( ) else: sheet.append(["no rows", *([""] * (len(headers) - 1))]) - # Frozen header + autofilter: required for analysis-ready long tables. - sheet.freeze_panes = "A2" - last_col = get_column_letter(max(len(headers), 1)) - last_row = max(sheet.max_row, 1) - sheet.auto_filter.ref = f"A1:{last_col}{last_row}" - header_fill = PatternFill("solid", fgColor="16324F") - for cell in sheet[1]: - cell.font = Font(bold=True, color="FFFFFF") - cell.fill = header_fill - cell.alignment = Alignment(horizontal="center", vertical="center", wrap_text=True) - for column_index, header in enumerate(headers, start=1): - sampled = [str(header)] + [ - str(_cell_value(row.get(header), bundle_root)) for row in rows[:200] - ] - # Slightly wider for analysis headers with units in the name. - width = min(max(max(len(value) for value in sampled) + 2, 10), 48) - sheet.column_dimensions[get_column_letter(column_index)].width = width - for row in sheet.iter_rows(min_row=2): - for cell in row: - cell.alignment = Alignment(vertical="top", wrap_text=True) + style_data_sheet(sheet, headers) def _add_guide_sheet(workbook: Workbook, title: str, rows: list[list[str]]) -> None: @@ -662,22 +643,7 @@ def _add_guide_sheet(workbook: Workbook, title: str, rows: list[list[str]]) -> N else: for row in rows: sheet.append([_cell_value(cell) for cell in row]) - sheet.freeze_panes = "A2" - header_fill = PatternFill("solid", fgColor="16324F") - for cell in sheet[1]: - cell.font = Font(bold=True, color="FFFFFF") - cell.fill = header_fill - cell.alignment = Alignment(horizontal="center", vertical="center", wrap_text=True) - for column_index in range(1, 3): - sampled = [ - str(_cell_value(row[column_index - 1]) if column_index - 1 < len(row) else "") - for row in rows[:200] - ] or [""] - width = min(max(max(len(value) for value in sampled) + 2, 12), 72) - sheet.column_dimensions[get_column_letter(column_index)].width = width - for row in sheet.iter_rows(min_row=2): - for cell in row: - cell.alignment = Alignment(vertical="top", wrap_text=True) + style_guide_sheet(sheet, rows) def write_excel_workbook( @@ -698,7 +664,7 @@ def write_excel_workbook( ) -> Path: workbook = Workbook() workbook.remove(workbook.active) - # Lab-first when enabled: guide → Chinese long table → full Peaks, then metadata sheets. + # Lab-first when enabled: guide and Chinese analysis view accompany canonical tables. if export_lab_views: _add_guide_sheet(workbook, "使用说明", user_guide_rows()) zh_headers = list(BEGINNER_PEAK_HEADERS_ZH.keys()) @@ -723,11 +689,14 @@ def write_excel_workbook( if export_lab_views: analysis_list = analyses or [] if 0 < len(analysis_list) <= 20: + peaks_by_phase: dict[str, list[dict[str, Any]]] = {} + for row in peaks: + peaks_by_phase.setdefault(str(row.get("phase_name") or ""), []).append(row) used_titles: set[str] = set(workbook.sheetnames) for analysis in analysis_list: if not analysis.reflections: continue - phase_peaks = [row for row in peaks if row.get("phase_name") == analysis.phase_name] + phase_peaks = peaks_by_phase.get(analysis.phase_name, []) if not phase_peaks: continue title = safe_excel_sheet_title(f"峰_{analysis.phase_name}", used=used_titles) @@ -738,9 +707,18 @@ def write_excel_workbook( PEAK_HEADERS, bundle_root=bundle_root, ) - # Open on the Chinese analysis long table when present. - if "推荐峰表" in workbook.sheetnames: - workbook.active = workbook["推荐峰表"] + overview_sheet = add_overview_sheet( + workbook, + summary=summary, + phases=phases, + diagnostics=diagnostics, + lab_views=export_lab_views, + ) + workbook.move_sheet( + overview_sheet, + offset=-workbook.sheetnames.index(overview_sheet.title), + ) + workbook.active = overview_sheet path.parent.mkdir(parents=True, exist_ok=True) temporary: Path | None = None try: @@ -1074,7 +1052,7 @@ def export_result_bundle( ), "elastic_modulus": "E(n) = 1 / (q(n)^T S q(n)) under engineering-shear Voigt convention", "lab_views_schema": ( - "When effective_export_lab_views is true, the emitted results.xlsx opens on 推荐峰表 after 使用说明: " + "When effective_export_lab_views is true, the emitted results.xlsx opens on 结果概览, followed by the lab guide and analysis sheets: " "analysis-first Chinese long table (BEGINNER_PEAK_HEADERS_ZH) with freeze+autofilter, " "optional per-phase peak sheets (≤20 phases). Peaks/CSV remain the English machine schema " "with the same peak_rows values; R_hkl names are volume-normalized J aliases, not residuals." diff --git a/src/diffractscout/gui.py b/src/diffractscout/gui.py index 2db3c28..87d762d 100644 --- a/src/diffractscout/gui.py +++ b/src/diffractscout/gui.py @@ -7,6 +7,7 @@ import queue import re import shutil +import stat import subprocess import sys import tempfile @@ -24,6 +25,7 @@ from .pipeline import analyze_cifs, run_pipeline from .providers.materials_project import MaterialsProjectProvider from .selection import validate_discovery_settings +from .utils import sha256_file try: # Tk remains optional on minimal/headless Python installations. import tkinter as tk @@ -64,6 +66,28 @@ _SHORTCUT_83 = "83 keV" _SHORTCUT_CUSTOM = "Custom" _ENERGY_SHORTCUTS = (_SHORTCUT_CU, _SHORTCUT_30, _SHORTCUT_83, _SHORTCUT_CUSTOM) +_ANALYSIS_PRESET_FIELDS = ( + "input_mode", + "source_preset", + "radiation_value", + "two_theta_min", + "two_theta_max", + "step", + "fwhm", + "eta", + "d_min_A", + "d_max_A", + "profile_model", + "pattern_axis", + "max_profile_points", + "max_reflection_estimate", + "include_excel", + "include_elasticity", + "export_lab_views", + "include_patterns", + "include_figures", + "local_recursive", +) def canonical_input_identity(path: str | Path) -> str: @@ -77,6 +101,20 @@ def canonical_input_identity(path: str | Path) -> str: return str(Path(path).expanduser().resolve(strict=False)) +def _local_input_problem(path: Path) -> str | None: + """Return a concise input error without walking the contents of folders.""" + + try: + mode = path.stat().st_mode + except OSError: + return "unavailable" + if stat.S_ISDIR(mode): + return None + if stat.S_ISREG(mode): + return None if path.suffix.lower() == ".cif" else "not_cif" + return "unsupported" + + def _required_float(value: object, field: str) -> float: try: parsed = float(str(value).strip()) @@ -241,11 +279,15 @@ def __init__(self) -> None: self.events: queue.Queue[tuple[str, object]] = queue.Queue() self.running = False self._worker_thread: threading.Thread | None = None - self._preview_dirs: list[Path] = [] + # Each preview records its directory, workbook path, and SHA-256 + # at creation. Changed previews are retained when the GUI closes. + self._preview_dirs: list[tuple[Path, Path, str]] = [] self.last_output: Path | None = None self.local_inputs: list[Path] = [] self.elastic_overrides: dict[str, ElasticTensor] = {} self._run_buttons: list[ttk.Button] = [] + self._result_buttons: list[ttk.Button] = [] + self._preset_buttons: list[ttk.Button] = [] self._radiation_source_widgets: list[ttk.Combobox] = [] self._radiation_value_widgets: list[ttk.Entry] = [] self._cij_widgets: list[Any] = [] @@ -960,6 +1002,28 @@ def _analysis_controls(self, parent: ttk.Frame) -> None: # remains vertically scrollable and the run action stays pinned. box.columnconfigure(1, weight=1) + preset_actions = ttk.Frame(box, style="Card.TFrame") + preset_actions.grid(row=0, column=0, columnspan=2, sticky="ew", pady=(0, 3)) + preset_actions.columnconfigure((0, 1), weight=1) + load_button = ttk.Button( + preset_actions, + text=self._t("load_parameters"), + style="Secondary.TButton", + command=self._load_analysis_preset, + ) + load_button.grid(row=0, column=0, sticky="ew", padx=(0, 3)) + self._register_text(load_button, "load_parameters") + save_button = ttk.Button( + preset_actions, + text=self._t("save_parameters"), + style="Secondary.TButton", + command=self._save_analysis_preset, + ) + save_button.grid(row=0, column=1, sticky="ew", padx=(3, 0)) + self._register_text(save_button, "save_parameters") + self._preset_buttons.extend((load_button, save_button)) + row_offset = 1 + def add_label_field(row: int, key: str, variable: Any, *, values: tuple[str, ...] | None = None) -> Any: label = ttk.Label(box, text=self._t(key), style="Card.TLabel") label.grid(row=row, column=0, sticky="w", pady=4) @@ -979,15 +1043,15 @@ def add_label_field(row: int, key: str, variable: Any, *, values: tuple[str, ... self._add_hover_help(field, "help_pattern_axis") return field - add_label_field(0, "energy_shortcut", self.energy_shortcut, values=_ENERGY_SHORTCUTS) - add_label_field(1, "input_mode", self.input_mode, values=("source", "energy", "wavelength")) + add_label_field(row_offset + 0, "energy_shortcut", self.energy_shortcut, values=_ENERGY_SHORTCUTS) + add_label_field(row_offset + 1, "input_mode", self.input_mode, values=("source", "energy", "wavelength")) source = add_label_field( - 2, + row_offset + 2, "source_preset", self.source_preset, values=("Cu Ka", "Co Ka", "Fe Ka", "Mo Ka", "Ag Ka", "Custom"), ) - value = add_label_field(3, "radiation_value", self.radiation_value) + value = add_label_field(row_offset + 3, "radiation_value", self.radiation_value) self._radiation_source_widgets.append(source) self._radiation_value_widgets.append(value) @@ -1000,10 +1064,10 @@ def add_label_field(row: int, key: str, variable: Any, *, values: tuple[str, ... ("d_min", self.d_min_A), ("d_max", self.d_max_A), ) - for row, (key, variable) in enumerate(labels, start=4): + for row, (key, variable) in enumerate(labels, start=row_offset + 4): add_label_field(row, key, variable) - add_label_field(4 + len(labels), "profile_model", self.profile_model, values=_PROFILE_MODELS) - add_label_field(5 + len(labels), "pattern_axis", self.pattern_axis, values=_PATTERN_AXES) + add_label_field(row_offset + 4 + len(labels), "profile_model", self.profile_model, values=_PROFILE_MODELS) + add_label_field(row_offset + 5 + len(labels), "pattern_axis", self.pattern_axis, values=_PATTERN_AXES) limits = self._labeled_frame(parent, "resource_guards", padding=10) limits.pack(fill="x", pady=(8, 0)) @@ -1021,6 +1085,97 @@ def add_label_field(row: int, key: str, variable: Any, *, values: tuple[str, ... ) limits.columnconfigure(1, weight=1) + def _analysis_preset_values(self) -> dict[str, object]: + values = { + field: getattr(self, field).get() + for field in _ANALYSIS_PRESET_FIELDS + } + # When Excel is switched off, the checkbox is disabled and forced + # false. Save the user's independent lab-view preference instead. + if self._lab_views_forced_off: + values["export_lab_views"] = self._lab_views_preference + return values + + def _save_analysis_preset(self) -> None: + if getattr(self, "running", False): + return + selected = filedialog.asksaveasfilename( + title=self._t("dialog_save_parameters"), + initialfile="diffractscout-analysis.json", + defaultextension=".json", + filetypes=((self._t("filetype_preset"), "*.json"), (self._t("filetype_all"), "*.*")), + confirmoverwrite=False, + ) + if not selected: + return + target = Path(selected).expanduser() + if target.exists() and not messagebox.askyesno( + self._t("dialog_overwrite_preset_title"), + self._t("dialog_overwrite_preset_message", path=target), + ): + return + try: + from .gui_settings import save_analysis_preset + + save_analysis_preset(target, self._analysis_preset_values()) + self._set_status("status_parameters_saved") + self._log(self._t("log_parameters_saved", path=target), "success") + except Exception as exc: + messagebox.showerror(self._t("err_save_parameters"), str(exc)) + + def _load_analysis_preset(self) -> None: + if getattr(self, "running", False): + return + selected = filedialog.askopenfilename( + title=self._t("dialog_load_parameters"), + filetypes=((self._t("filetype_preset"), "*.json"), (self._t("filetype_all"), "*.*")), + ) + if not selected: + return + try: + from .gui_settings import load_analysis_preset + + values = load_analysis_preset(Path(selected).expanduser()) + expected = set(_ANALYSIS_PRESET_FIELDS) + if set(values) != expected: + missing = sorted(expected - set(values)) + extra = sorted(set(values) - expected) + raise ValueError( + self._t( + "err_preset_fields", + missing=", ".join(missing) or "—", + extra=", ".join(extra) or "—", + ) + ) + # Validate the complete loaded form before changing any Tk + # variable, so a rejected preset cannot be partly applied. + analysis_settings_from_form(values) + + was_syncing_radiation = self._syncing_radiation + was_syncing_outputs = self._syncing_output_dependencies + self._syncing_radiation = True + self._syncing_output_dependencies = True + try: + for field in _ANALYSIS_PRESET_FIELDS: + getattr(self, field).set(values[field]) + self._previous_radiation_mode = self.input_mode.get() + self._previous_source_preset = self.source_preset.get() + self._previous_radiation_value = self.radiation_value.get() + self._radiation_initialized = True + self._lab_views_preference = bool(values["export_lab_views"]) + self._lab_views_forced_off = not bool(values["include_excel"]) + finally: + self._syncing_radiation = was_syncing_radiation + self._syncing_output_dependencies = was_syncing_outputs + self._sync_radiation_controls() + self._sync_output_dependencies() + self._set_status("status_parameters_loaded") + self._log( + self._t("log_parameters_loaded", path=selected), "success" + ) + except Exception as exc: + messagebox.showerror(self._t("err_load_parameters"), str(exc)) + def _build_cij_panel(self, parent: ttk.Frame) -> None: box = self._labeled_frame(parent, "cij_panel", padding=8) box.pack(fill="x", pady=(8, 0)) @@ -1170,23 +1325,58 @@ def _build_status_bar(self) -> None: # Keep the button's requested height inside the bar at the # minimum window size; otherwise vertical pack padding clips its # native glyph area before the user can resize the window. - bar = tk.Frame(self, bg="#E5EDF3", height=42) + bar = tk.Frame(self, bg="#E5EDF3", height=48) bar.pack(fill="x", side="bottom") bar.pack_propagate(False) - ttk.Label(bar, textvariable=self.status_text, background="#E5EDF3", foreground=NAVY).pack( - side="left", padx=14 - ) - self.progress = ttk.Progressbar(bar, mode="indeterminate", length=140) - self.progress.pack(side="right", padx=(8, 14), pady=7) - self.open_button = ttk.Button( + self.status_label = ttk.Label( bar, - text=self._t("open_result"), + textvariable=self.status_text, + background="#E5EDF3", + foreground=NAVY, + width=30, + wraplength=230, + anchor="w", + justify="left", + ) + self.progress = ttk.Progressbar(bar, mode="indeterminate", length=100) + self.progress.pack(side="right", padx=(6, 12), pady=8) + result_actions = ttk.Frame(bar) + result_actions.pack(side="right", padx=(4, 0), pady=4) + self.preview_button = ttk.Button( + result_actions, + text=self._t("preview_excel"), + style="Secondary.TButton", + command=self._preview_last_workbook, + state="disabled", + ) + self.preview_button.grid(row=0, column=0, padx=(0, 3)) + self._register_text(self.preview_button, "preview_excel") + self._add_hover_help(self.preview_button, "help_preview_excel") + self.save_excel_button = ttk.Button( + result_actions, + text=self._t("save_excel_as"), + style="Secondary.TButton", + command=self._save_excel_copy, + state="disabled", + ) + self.save_excel_button.grid(row=0, column=1, padx=3) + self._register_text(self.save_excel_button, "save_excel_as") + self.open_folder_button = ttk.Button( + result_actions, + text=self._t("open_result_folder"), style="Secondary.TButton", - command=self._open_last_output, + command=self._open_last_output_folder, state="disabled", ) - self.open_button.pack(side="right", pady=4) - self._register_text(self.open_button, "open_result") + self.open_folder_button.grid(row=0, column=2, padx=(3, 0)) + self._register_text(self.open_folder_button, "open_result_folder") + self._result_buttons.extend( + (self.preview_button, self.save_excel_button, self.open_folder_button) + ) + self.status_label.pack(side="left", fill="both", expand=True, padx=(14, 4)) + # Keep this alias for code and downstream tests that referred to + # the former single result action. + self.open_button = self.preview_button def _enable_dnd(self, widget: Any) -> None: if not _HAS_DND or DND_FILES is None: @@ -1437,11 +1627,18 @@ def _sync_radiation_controls(self) -> None: label.configure(text=self._t(unit_key)) if not self._syncing_shortcut: # Keep shortcut label coherent when mode is edited manually. + energy_value = self._valid_radiation_value( + self.radiation_value.get() + ) if mode == "source" and source == "Cu Ka": expected = _SHORTCUT_CU - elif mode == "energy" and self.radiation_value.get().strip() == "30": + elif mode == "energy" and energy_value is not None and math.isclose( + energy_value, 30.0, rel_tol=0.0, abs_tol=1e-9 + ): expected = _SHORTCUT_30 - elif mode == "energy" and self.radiation_value.get().strip() == "83": + elif mode == "energy" and energy_value is not None and math.isclose( + energy_value, 83.0, rel_tol=0.0, abs_tol=1e-9 + ): expected = _SHORTCUT_83 else: expected = _SHORTCUT_CUSTOM @@ -1602,12 +1799,41 @@ def _add_cif_folder(self) -> None: def _add_input_paths(self, paths: Any) -> None: existing = {path.resolve() for path in self.local_inputs} + invalid: list[tuple[Path, str]] = [] for path in paths: resolved = Path(path).expanduser().resolve() + problem = _local_input_problem(resolved) + if problem is not None: + invalid.append((resolved, problem)) + continue if resolved not in existing: self.local_inputs.append(resolved) existing.add(resolved) self._refresh_inputs() + if invalid: + details = self._format_input_problems(invalid) + title = self._t("err_invalid_local_inputs_title") + message = self._t("err_invalid_local_inputs", details=details) + if self.local_inputs: + messagebox.showwarning(title, message) + else: + messagebox.showerror(title, message) + + def _format_input_problems(self, problems: list[tuple[Path, str]]) -> str: + shown = problems[:5] + details = "\n".join( + self._t( + "invalid_input_item", + path=path, + reason=self._t(f"input_problem_{reason}"), + ) + for path, reason in shown + ) + if len(problems) > len(shown): + details += "\n" + self._t( + "invalid_input_more", n=len(problems) - len(shown) + ) + return details def _remove_inputs(self) -> None: selected = set(self.input_list.curselection()) @@ -1711,6 +1937,10 @@ def field_label(raw: str) -> str: if "2theta range must satisfy" in lowered or "2θ range must satisfy" in lowered: return self._t("validation_range") + if "no cif files were found in the supplied inputs" in lowered: + return self._t("err_no_cif_found") + if lowered.startswith("input path does not exist:"): + return self._t("err_input_disappeared", path=detail.split(":", 1)[-1].strip()) if "diffraction settings must be finite numbers" in lowered: return self._t("validation_finite") if "step_deg and fwhm_deg must be positive" in lowered: @@ -1785,7 +2015,30 @@ def _run_local(self) -> None: except ValueError as exc: messagebox.showerror(self._t("err_title_settings"), self._validation_message(exc)) return - inputs = [str(path) for path in self.local_inputs] + valid_inputs: list[Path] = [] + invalid_inputs: list[tuple[Path, str]] = [] + for path in self.local_inputs: + problem = _local_input_problem(path) + if problem is None: + valid_inputs.append(path) + else: + invalid_inputs.append((path, problem)) + if invalid_inputs: + details = self._format_input_problems(invalid_inputs) + title = self._t("err_invalid_local_inputs_title") + message = self._t("err_invalid_local_inputs", details=details) + messagebox.showerror(title, message) + return + if not valid_inputs: + messagebox.showerror( + self._t("err_title_missing"), self._t("err_no_cif_found") + ) + return + + # Folder contents are deliberately not scanned here. The pipeline + # walks selected folders in its worker thread, so empty folders + # are reported without freezing Tk's event loop. + inputs = [str(path) for path in valid_inputs] overrides = dict(self.elastic_overrides) if self.elastic_overrides else None recursive = bool(self.local_recursive.get()) include_excel = bool(self.include_excel.get()) @@ -1864,6 +2117,8 @@ def _start_task(self, label: str, function: Callable[[], PipelineResult]) -> Non for button in self._run_buttons: button.configure(state="disabled") self._log(self._t("log_started", label=label), "info") + for button in self._result_buttons + self._preset_buttons: + button.configure(state="disabled") self._worker_thread = threading.Thread(target=self._worker, args=(function,), daemon=False) self._worker_thread.start() @@ -1879,20 +2134,27 @@ def _finish_task(self) -> None: for button in self._run_buttons: button.configure(state="normal") DiffractScoutApp._update_open_button_state(self) + for button in self._preset_buttons: + button.configure(state="normal") def _update_open_button_state(self) -> None: - if getattr(self, "running", False): - self.open_button.configure(state="disabled") - return + busy = bool(getattr(self, "running", False)) target = getattr(self, "last_output", None) - usable = False + folder_exists = False + workbook_exists = False if target is not None: path = Path(target) - usable = path.is_dir() and ( - (path / "manifest.json").is_file() - or (path / "results.xlsx").is_file() - ) - self.open_button.configure(state="normal" if usable else "disabled") + folder_exists = path.is_dir() + workbook_exists = folder_exists and (path / "results.xlsx").is_file() + self.preview_button.configure( + state="normal" if workbook_exists and not busy else "disabled" + ) + self.save_excel_button.configure( + state="normal" if workbook_exists and not busy else "disabled" + ) + self.open_folder_button.configure( + state="normal" if folder_exists and not busy else "disabled" + ) def _log(self, text: str, level: str = "info") -> None: timestamp = datetime.now().strftime("%H:%M:%S") @@ -1914,6 +2176,12 @@ def _poll(self) -> None: assert isinstance(result, PipelineResult) self.last_output = result.output_dir error_count = sum(item.level == "error" for item in result.diagnostics) + warning_count = sum( + item.level == "warning" for item in result.diagnostics + ) + peak_count = sum( + len(analysis.reflections) for analysis in result.analyses + ) if not result.analyses: completion_key = "status_completed_empty" log_level = "warning" @@ -1952,10 +2220,21 @@ def _poll(self) -> None: if diagnostic.level == "warning" else "info", ) - dialog = messagebox.showwarning if (not result.analyses or error_count) else messagebox.showinfo + dialog = ( + messagebox.showwarning + if (not result.analyses or error_count or warning_count) + else messagebox.showinfo + ) dialog( self._t("dialog_result_title"), - self._t("dialog_result_message", path=result.output_dir), + self._t( + "dialog_result_message", + path=result.output_dir, + phases=len(result.analyses), + peaks=peak_count, + warnings=warning_count, + errors=error_count, + ), ) else: exc, details = payload @@ -1964,7 +2243,7 @@ def _poll(self) -> None: self._log(str(details), "error") messagebox.showerror( self._t("dialog_failed_title"), - self._t("dialog_failed_message", error=str(exc)), + self._t("dialog_failed_message", error=self._validation_message(exc)), ) try: self._poll_after_id = self.after(120, self._poll) @@ -1982,51 +2261,136 @@ def _clear_log(self) -> None: self.log.configure(state="disabled") def _preview_workbook(self, workbook: Path) -> Path: - """Copy a result workbook outside its bundle before opening it. - - Excel creates ``~$results.xlsx`` beside an opened workbook. The - bundle is immutable evidence, so opening a preview copy keeps - verifier inputs unchanged while retaining the user's workbook. - """ + """Open a temporary copy and retain it if Excel changed its contents.""" preview_dir = Path(tempfile.mkdtemp(prefix="diffractscout-preview-")) preview_path = preview_dir / workbook.name try: shutil.copy2(workbook, preview_path) + original_hash = sha256_file(preview_path) except Exception: shutil.rmtree(preview_dir, ignore_errors=True) raise - self._preview_dirs.append(preview_dir) + self._preview_dirs.append((preview_dir, preview_path, original_hash)) return preview_path - def _cleanup_preview_dirs(self) -> None: - remaining: list[Path] = [] - for preview_dir in list(self._preview_dirs): + def _cleanup_preview_dirs(self) -> list[Path]: + """Delete unchanged previews; report changed or still-locked copies.""" + + remaining: list[tuple[Path, Path, str]] = [] + retained: list[Path] = [] + for preview_dir, preview_path, original_hash in list(self._preview_dirs): try: - if preview_dir.is_dir(): - shutil.rmtree(preview_dir) + if not preview_dir.is_dir(): + continue + if preview_path.is_file() and sha256_file(preview_path) != original_hash: + remaining.append((preview_dir, preview_path, original_hash)) + retained.append(preview_path) + continue + children = list(preview_dir.iterdir()) + if not children: + preview_dir.rmdir() + continue + if ( + len(children) != 1 + or children[0] != preview_path + or not children[0].is_file() + or children[0].is_symlink() + ): + remaining.append((preview_dir, preview_path, original_hash)) + retained.append(preview_dir) + continue + # Remove only the unchanged workbook that the app created. + # Any lock or extra file means a user may still be working + # in this directory, so preserve the whole directory. + children[0].unlink() + preview_dir.rmdir() except OSError: - # Excel may still hold a preview open; leaving a temp - # directory is safer than touching any user bundle. - remaining.append(preview_dir) - continue + # Excel may still hold the file open. Keep it and show the + # path to the user instead of losing saved workbook edits. + remaining.append((preview_dir, preview_path, original_hash)) + retained.append( + preview_path if preview_path.exists() else preview_dir + ) self._preview_dirs = remaining + return retained + + def _preview_last_workbook(self) -> None: + if getattr(self, "running", False) or self.last_output is None: + return + bundle = Path(self.last_output) + workbook = bundle / "results.xlsx" + if not workbook.is_file(): + self._update_open_button_state() + return + try: + preview = self._preview_workbook(workbook) + open_path(preview) + self._log(self._t("log_excel_preview", path=preview), "info") + except Exception as exc: + messagebox.showerror(self._t("err_open_result"), str(exc)) - def _open_last_output(self) -> None: - if self.last_output is None: + def _open_last_output_folder(self) -> None: + if getattr(self, "running", False) or self.last_output is None: return - target: Path = Path(self.last_output) - if not target.is_dir(): + folder = Path(self.last_output) + if not folder.is_dir(): self._update_open_button_state() return - xlsx = target / "results.xlsx" try: - if xlsx.is_file(): - target = self._preview_workbook(xlsx) - open_path(target) + open_path(folder) except Exception as exc: messagebox.showerror(self._t("err_open_result"), str(exc)) + def _save_excel_copy(self) -> None: + if getattr(self, "running", False) or self.last_output is None: + return + bundle = Path(self.last_output).resolve() + workbook = bundle / "results.xlsx" + if not workbook.is_file(): + self._update_open_button_state() + return + try: + selected = filedialog.asksaveasfilename( + title=self._t("dialog_save_excel"), + initialdir=str(bundle.parent), + initialfile=f"{bundle.name}_results.xlsx", + defaultextension=".xlsx", + filetypes=((self._t("filetype_excel"), "*.xlsx"), (self._t("filetype_all"), "*.*")), + # Use one explicit, localized confirmation below on all + # platforms instead of relying on native dialog defaults. + confirmoverwrite=False, + ) + if not selected: + return + requested = Path(selected).expanduser() + if requested.is_symlink(): + raise ValueError(self._t("err_excel_copy_symlink")) + target = requested.parent.resolve(strict=False) / requested.name + try: + target.relative_to(bundle) + except ValueError: + pass + else: + raise ValueError(self._t("err_excel_copy_in_bundle")) + if target.suffix.lower() != ".xlsx": + raise ValueError(self._t("err_excel_copy_extension")) + exists = target.exists() + if exists and not target.is_file(): + raise ValueError(self._t("err_excel_copy_not_file")) + if exists and not messagebox.askyesno( + self._t("dialog_overwrite_excel_title"), + self._t("dialog_overwrite_excel_message", path=target), + ): + return + from .quick_export import _copy_excel_atomic + + _copy_excel_atomic(workbook, target, overwrite=exists) + self._set_status("status_excel_saved") + self._log(self._t("log_excel_saved", path=target), "success") + except Exception as exc: + messagebox.showerror(self._t("err_save_excel"), str(exc)) + def _on_close(self) -> None: if self.running or (self._worker_thread is not None and self._worker_thread.is_alive()): # Do not destroy a live controller or rely on daemon-thread @@ -2037,7 +2401,13 @@ def _on_close(self) -> None: self._t("msg_close_running"), ) return - self._cleanup_preview_dirs() + retained = self._cleanup_preview_dirs() + if retained: + paths = "\n".join(str(path) for path in retained) + messagebox.showinfo( + self._t("msg_preview_retained_title"), + self._t("msg_preview_retained", paths=paths), + ) self.destroy() def destroy(self) -> None: diff --git a/src/diffractscout/gui_i18n.py b/src/diffractscout/gui_i18n.py index 090ad67..0265c83 100644 --- a/src/diffractscout/gui_i18n.py +++ b/src/diffractscout/gui_i18n.py @@ -90,6 +90,24 @@ "copy", "clear_log", "open_result", + "preview_excel", + "save_excel_as", + "open_result_folder", + "help_preview_excel", + "status_excel_saved", + "log_excel_preview", + "log_excel_saved", + "dialog_save_excel", + "filetype_excel", + "dialog_overwrite_excel_title", + "dialog_overwrite_excel_message", + "err_save_excel", + "err_excel_copy_symlink", + "err_excel_copy_in_bundle", + "err_excel_copy_not_file", + "err_excel_copy_extension", + "msg_preview_retained_title", + "msg_preview_retained", "status_ready", "status_busy", "status_completed", @@ -100,6 +118,29 @@ "status_log_copied", "inputs_none", "inputs_count", + "err_invalid_local_inputs_title", + "err_invalid_local_inputs", + "invalid_input_item", + "invalid_input_more", + "input_problem_unavailable", + "input_problem_not_cif", + "input_problem_unsupported", + "err_no_cif_found", + "err_input_disappeared", + "load_parameters", + "save_parameters", + "dialog_save_parameters", + "dialog_load_parameters", + "filetype_preset", + "dialog_overwrite_preset_title", + "dialog_overwrite_preset_message", + "err_save_parameters", + "err_load_parameters", + "err_preset_fields", + "status_parameters_saved", + "status_parameters_loaded", + "log_parameters_saved", + "log_parameters_loaded", "err_missing_local", "err_missing_local_inputs", "err_missing_output", @@ -239,6 +280,24 @@ "copy": "复制", "clear_log": "清空", "open_result": "打开结果", + "preview_excel": "预览 Excel", + "save_excel_as": "另存 Excel", + "open_result_folder": "结果文件夹", + "help_preview_excel": "打开结果工作簿的临时副本，避免 Excel 锁文件进入结果包。关闭程序时，未修改的副本会清理；已修改或含有其他文件的副本目录会保留并提示路径。", + "status_excel_saved": "Excel 副本已保存", + "log_excel_preview": "临时 Excel 预览：{path}", + "log_excel_saved": "Excel 副本已保存到：{path}", + "dialog_save_excel": "另存结果 Excel 副本", + "filetype_excel": "Excel 工作簿", + "dialog_overwrite_excel_title": "覆盖 Excel 文件", + "dialog_overwrite_excel_message": "文件已存在，是否覆盖？\n{path}", + "err_save_excel": "保存 Excel 副本失败", + "err_excel_copy_symlink": "不能覆盖符号链接，请选择普通文件路径。", + "err_excel_copy_in_bundle": "Excel 副本必须保存在结果包目录之外，以保护原始结果与清单。", + "err_excel_copy_not_file": "所选目标不是普通文件。", + "err_excel_copy_extension": "请使用 .xlsx 扩展名保存 Excel 工作簿。", + "msg_preview_retained_title": "Excel 预览副本已保留", + "msg_preview_retained": "以下预览副本含有修改或其他文件，未自动删除：\n{paths}", "status_ready": "就绪", "status_busy": "运行中：{label}", "status_completed": "已完成", @@ -249,6 +308,29 @@ "status_log_copied": "活动日志已复制", "inputs_none": "未选择 CIF 输入", "inputs_count": "{n} 个输入路径已选择", + "err_invalid_local_inputs_title": "本地输入无效", + "err_invalid_local_inputs": "以下输入未添加或已不可用。请修正或移除后再运行；其他有效选择已保留：\n{details}", + "invalid_input_item": "{path} — {reason}", + "invalid_input_more": "另有 {n} 项未显示", + "input_problem_unavailable": "路径不存在或无法访问", + "input_problem_not_cif": "普通文件不是 CIF 文件", + "input_problem_unsupported": "路径不是普通文件或文件夹", + "err_no_cif_found": "所选文件夹中没有找到 CIF 文件。请检查递归扫描设置和文件夹内容。", + "err_input_disappeared": "运行期间输入路径已不存在：{path}", + "load_parameters": "加载参数", + "save_parameters": "保存参数", + "dialog_save_parameters": "保存 DiffractScout 参数", + "dialog_load_parameters": "加载 DiffractScout 参数", + "filetype_preset": "DiffractScout 参数", + "dialog_overwrite_preset_title": "覆盖参数文件", + "dialog_overwrite_preset_message": "文件已存在，是否覆盖？\n{path}", + "err_save_parameters": "保存参数失败", + "err_load_parameters": "加载参数失败", + "err_preset_fields": "参数文件字段不匹配。缺少：{missing}；多出：{extra}", + "status_parameters_saved": "参数已保存", + "status_parameters_loaded": "参数已加载", + "log_parameters_saved": "参数已保存到：{path}", + "log_parameters_loaded": "已加载参数：{path}", "err_missing_local": "请至少添加一个 CIF 文件或文件夹，并选择结果目录。", "err_missing_local_inputs": "请至少添加一个 CIF 文件或文件夹。", "err_missing_output": "请选择结果目录。", @@ -265,7 +347,7 @@ "msg_close_running": "流程仍在运行。为保护事务结果，请等待流程完成后再关闭窗口。", "msg_close_title": "关闭 DiffractScout", "dialog_result_title": "DiffractScout", - "dialog_result_message": "完成：输出已保存到 {path}。", + "dialog_result_message": "完成。物相：{phases}；衍射峰：{peaks}；警告：{warnings}；错误：{errors}。\n结果目录：{path}", "dialog_failed_title": "DiffractScout 流程失败", "dialog_failed_message": "{error}", "dialog_file_select": "选择 CIF 文件", @@ -385,6 +467,24 @@ "copy": "Copy", "clear_log": "Clear", "open_result": "Open result", + "preview_excel": "Preview Excel", + "save_excel_as": "Save Excel As", + "open_result_folder": "Result Folder", + "help_preview_excel": "Opens a temporary workbook copy so Excel lock files stay out of the result bundle. Unchanged copies are removed when the app closes; edited copies or folders with extra files are retained and their paths are shown.", + "status_excel_saved": "Excel copy saved", + "log_excel_preview": "Temporary Excel preview: {path}", + "log_excel_saved": "Excel copy saved to: {path}", + "dialog_save_excel": "Save a copy of the result workbook", + "filetype_excel": "Excel workbooks", + "dialog_overwrite_excel_title": "Replace Excel file", + "dialog_overwrite_excel_message": "This file already exists. Replace it?\n{path}", + "err_save_excel": "Could not save Excel copy", + "err_excel_copy_symlink": "Cannot replace a symbolic link. Choose a regular file path.", + "err_excel_copy_in_bundle": "Save the Excel copy outside the result bundle to protect its original results and manifest.", + "err_excel_copy_not_file": "The selected target is not a regular file.", + "err_excel_copy_extension": "Save the workbook with the .xlsx extension.", + "msg_preview_retained_title": "Excel preview copy retained", + "msg_preview_retained": "These preview copies contain edits or extra files and were not removed:\n{paths}", "status_ready": "Ready", "status_busy": "Running: {label}", "status_completed": "Completed", @@ -395,6 +495,29 @@ "status_log_copied": "Activity log copied", "inputs_none": "No CIF inputs selected", "inputs_count": "{n} input path(s) selected", + "err_invalid_local_inputs_title": "Invalid local inputs", + "err_invalid_local_inputs": "These inputs were not added or are no longer available. Fix or remove them before running; other valid selections are preserved:\n{details}", + "invalid_input_item": "{path} — {reason}", + "invalid_input_more": "{n} additional item(s) omitted", + "input_problem_unavailable": "Path does not exist or cannot be accessed", + "input_problem_not_cif": "Regular file is not a CIF file", + "input_problem_unsupported": "Path is not a regular file or folder", + "err_no_cif_found": "No CIF files were found in the selected folders. Check the recursive scan option and folder contents.", + "err_input_disappeared": "Input path no longer exists: {path}", + "load_parameters": "Load parameters", + "save_parameters": "Save parameters", + "dialog_save_parameters": "Save DiffractScout parameters", + "dialog_load_parameters": "Load DiffractScout parameters", + "filetype_preset": "DiffractScout parameters", + "dialog_overwrite_preset_title": "Replace parameter file", + "dialog_overwrite_preset_message": "This file already exists. Replace it?\n{path}", + "err_save_parameters": "Could not save parameters", + "err_load_parameters": "Could not load parameters", + "err_preset_fields": "Parameter file fields do not match. Missing: {missing}; extra: {extra}", + "status_parameters_saved": "Parameters saved", + "status_parameters_loaded": "Parameters loaded", + "log_parameters_saved": "Parameters saved to: {path}", + "log_parameters_loaded": "Loaded parameters: {path}", "err_missing_local": "Add at least one CIF file or folder and choose a result directory.", "err_missing_local_inputs": "Add at least one CIF file or folder.", "err_missing_output": "Choose a result directory.", @@ -411,7 +534,7 @@ "msg_close_running": "A workflow is still running. Please wait for its safe completion before closing this window.", "msg_close_title": "Close DiffractScout", "dialog_result_title": "DiffractScout", - "dialog_result_message": "Completed: output saved to {path}.", + "dialog_result_message": "Completed. Phases: {phases}; indexed peaks: {peaks}; warnings: {warnings}; errors: {errors}.\nResult folder: {path}", "dialog_failed_title": "DiffractScout workflow failed", "dialog_failed_message": "{error}", "dialog_file_select": "Select CIF files", diff --git a/src/diffractscout/gui_settings.py b/src/diffractscout/gui_settings.py new file mode 100644 index 0000000..740dc4e --- /dev/null +++ b/src/diffractscout/gui_settings.py @@ -0,0 +1,371 @@ +"""Strict persistence for reusable local GUI analysis presets. + +Presets intentionally contain only analysis and local-export controls. They +must never capture credentials, input/output paths, overwrite choices, or +per-structure elasticity overrides. +""" + +from __future__ import annotations + +import json +import math +import os +import tempfile +from collections.abc import Mapping +from pathlib import Path +from typing import Any + +from .diffraction import X_RAY_SOURCES_A, validate_analysis_settings +from .models import AnalysisSettings + +_PRESET_SCHEMA = "diffractscout_analysis_preset" +_PRESET_VERSION = 1 +_MAX_PRESET_BYTES = 64 * 1024 + +# These names correspond to analysis/local controls in DiffractScoutApp's +# `_create_variables`; keep this allowlist explicit so sensitive GUI state can +# never be serialized by accident. +_PRESET_FIELDS = frozenset( + { + "input_mode", + "source_preset", + "radiation_value", + "two_theta_min", + "two_theta_max", + "step", + "fwhm", + "eta", + "d_min_A", + "d_max_A", + "profile_model", + "pattern_axis", + "max_profile_points", + "max_reflection_estimate", + "include_excel", + "include_elasticity", + "export_lab_views", + "include_patterns", + "include_figures", + "local_recursive", + } +) +_BOOL_FIELDS = frozenset( + { + "include_excel", + "include_elasticity", + "export_lab_views", + "include_patterns", + "include_figures", + "local_recursive", + } +) +_FLOAT_FIELDS = { + "two_theta_min": "2θ minimum", + "two_theta_max": "2θ maximum", + "step": "Profile step", + "fwhm": "FWHM", + "eta": "Pseudo-Voigt η", +} +_INTEGER_FIELDS = { + "max_profile_points": "Maximum profile points", + "max_reflection_estimate": "Maximum reciprocal candidates", +} + + +def _error(message: str) -> ValueError: + return ValueError(f"Invalid analysis preset: {message}") + + +def _finite_number(value: object, field: str, *, allow_text: bool) -> float: + if isinstance(value, bool) or (not allow_text and not isinstance(value, (int, float))): + raise _error(f"{field} must be a finite JSON number.") + if allow_text and not isinstance(value, (str, int, float)): + raise _error(f"{field} must be a number.") + try: + parsed = float(value.strip() if isinstance(value, str) else value) + except (TypeError, ValueError, OverflowError) as exc: + raise _error(f"{field} must be a number.") from exc + if not math.isfinite(parsed): + raise _error(f"{field} must be finite.") + return parsed + + +def _optional_number(value: object, field: str, *, allow_text: bool) -> float | None: + if allow_text and isinstance(value, str) and not value.strip(): + return None + if not allow_text and value is None: + return None + if value is None: + raise _error(f"{field} must be a finite number when set.") + return _finite_number(value, field, allow_text=allow_text) + + +def _positive_integer(value: object, field: str, *, allow_text: bool) -> int: + if isinstance(value, bool): + raise _error(f"{field} must be a positive integer.") + if allow_text: + if not isinstance(value, (str, int)): + raise _error(f"{field} must be an integer.") + try: + parsed = int(value.strip() if isinstance(value, str) else value) + except (ValueError, TypeError, OverflowError) as exc: + raise _error(f"{field} must be an integer.") from exc + else: + if type(value) is not int: + raise _error(f"{field} must be a positive JSON integer.") + parsed = value + if parsed < 1: + raise _error(f"{field} must be a positive integer.") + return parsed + + +def _require_string(values: Mapping[str, object], field: str) -> str: + value = values[field] + if not isinstance(value, str): + raise _error(f"{field} must be a string.") + return value + + +def _normalize_values(values: object, *, from_form: bool) -> dict[str, object]: + if not isinstance(values, Mapping): + raise _error("values must be a JSON object.") + supplied = set(values) + missing = sorted(_PRESET_FIELDS - supplied) + extra = sorted(supplied - _PRESET_FIELDS) + if missing: + raise _error(f"missing required field(s): {', '.join(missing)}.") + if extra: + raise _error(f"unknown or unsupported field(s): {', '.join(extra)}.") + + input_mode = _require_string(values, "input_mode") + if input_mode not in {"source", "wavelength", "energy"}: + raise _error("input_mode must be source, wavelength, or energy.") + source_preset = _require_string(values, "source_preset") + if source_preset not in X_RAY_SOURCES_A: + raise _error( + "source_preset must be one of: " + ", ".join(sorted(X_RAY_SOURCES_A)) + "." + ) + + raw_radiation = values["radiation_value"] + if isinstance(raw_radiation, bool): + raise _error("radiation_value must be a number or an empty string.") + if from_form and not isinstance(raw_radiation, (str, int, float)): + raise _error("radiation_value must be a number or an empty string.") + if not from_form and raw_radiation is not None and not isinstance(raw_radiation, (int, float)): + raise _error("radiation_value must be a JSON number or null.") + consumes_radiation = input_mode != "source" or source_preset == "Custom" + if consumes_radiation: + radiation = _optional_number(raw_radiation, "radiation_value", allow_text=from_form) + else: + # A built-in source does not consume this disabled entry. Discard stale + # GUI text, matching analysis_settings_from_form's behavior. + radiation = None + + normalized: dict[str, object] = { + "input_mode": input_mode, + "source_preset": source_preset, + "radiation_value": radiation, + "d_min_A": _optional_number(values["d_min_A"], "d_min_A", allow_text=from_form), + "d_max_A": _optional_number(values["d_max_A"], "d_max_A", allow_text=from_form), + "profile_model": _require_string(values, "profile_model"), + "pattern_axis": _require_string(values, "pattern_axis"), + } + normalized.update( + { + field: _finite_number(values[field], label, allow_text=from_form) + for field, label in _FLOAT_FIELDS.items() + } + ) + normalized.update( + { + field: _positive_integer(values[field], label, allow_text=from_form) + for field, label in _INTEGER_FIELDS.items() + } + ) + for field in _BOOL_FIELDS: + value = values[field] + if type(value) is not bool: + raise _error(f"{field} must be a boolean.") + normalized[field] = value + + analysis = AnalysisSettings( + input_mode=input_mode, # type: ignore[arg-type] + source_preset=source_preset, + wavelength_A=( + float(radiation) + if radiation is not None + and (input_mode == "wavelength" or (input_mode == "source" and source_preset == "Custom")) + else None + ), + energy_keV=(float(radiation) if radiation is not None and input_mode == "energy" else None), + two_theta_min_deg=float(normalized["two_theta_min"]), + two_theta_max_deg=float(normalized["two_theta_max"]), + step_deg=float(normalized["step"]), + fwhm_deg=float(normalized["fwhm"]), + profile_eta=float(normalized["eta"]), + include_elasticity=bool(normalized["include_elasticity"]), + max_profile_points=int(normalized["max_profile_points"]), + max_reflection_estimate=int(normalized["max_reflection_estimate"]), + d_min_A=normalized["d_min_A"], # type: ignore[arg-type] + d_max_A=normalized["d_max_A"], # type: ignore[arg-type] + profile_model=normalized["profile_model"], # type: ignore[arg-type] + pattern_axis=normalized["pattern_axis"], # type: ignore[arg-type] + include_figures=bool(normalized["include_figures"]), + export_lab_views=bool(normalized["export_lab_views"]), + include_patterns=bool(normalized["include_patterns"]), + ) + try: + validate_analysis_settings(analysis) + except (TypeError, ValueError, OverflowError) as exc: + raise _error(str(exc)) from exc + + # Return empty strings for unset optional GUI entries. Tk StringVars accept + # these normalized numbers directly via `.set()`. + normalized["radiation_value"] = "" if radiation is None else radiation + normalized["d_min_A"] = "" if normalized["d_min_A"] is None else normalized["d_min_A"] + normalized["d_max_A"] = "" if normalized["d_max_A"] is None else normalized["d_max_A"] + return normalized + + +def _serialized_values(values: Mapping[str, object]) -> dict[str, object]: + """Encode blank GUI entries as JSON null for all optional numeric fields.""" + + serialized = dict(values) + for field in ("radiation_value", "d_min_A", "d_max_A"): + if serialized[field] == "": + serialized[field] = None + return serialized + + +def _reject_duplicate_pairs(pairs: list[tuple[str, Any]]) -> dict[str, Any]: + result: dict[str, Any] = {} + for key, value in pairs: + if key in result: + raise _error(f"duplicate JSON key {key!r}.") + result[key] = value + return result + + +def _reject_nonstandard_constant(value: str) -> None: + raise _error(f"non-finite JSON number {value} is not allowed.") + + +def _target_path(path: str | Path) -> Path: + try: + raw = os.fspath(path) + except TypeError as exc: + raise TypeError("preset path must be a string or path-like value") from exc + if not isinstance(raw, str) or not raw.strip(): + raise ValueError("preset path must be a non-empty string") + return Path(raw).expanduser() + + +def save_analysis_preset( + path: str | Path, + values: Mapping[str, object], + *, + overwrite: bool = True, +) -> Path: + """Validate and atomically save a complete local analysis preset. + + With ``overwrite=False``, publish via an atomic same-directory hard link, + which fails if another process has already created the target. + """ + + target = _target_path(path) + normalized = _normalize_values(values, from_form=True) + document = { + "schema": _PRESET_SCHEMA, + "version": _PRESET_VERSION, + "values": _serialized_values(normalized), + } + encoded = (json.dumps(document, ensure_ascii=False, indent=2, allow_nan=False) + "\n").encode( + "utf-8" + ) + if len(encoded) > _MAX_PRESET_BYTES: + raise _error(f"preset exceeds the {_MAX_PRESET_BYTES}-byte size limit.") + + target.parent.mkdir(parents=True, exist_ok=True) + temporary_path: Path | None = None + try: + with tempfile.NamedTemporaryFile( + mode="wb", + prefix=".diffractscout-preset-", + suffix=".tmp", + dir=target.parent, + delete=False, + ) as stream: + temporary_path = Path(stream.name) + stream.write(encoded) + stream.flush() + os.fsync(stream.fileno()) + if overwrite: + os.replace(temporary_path, target) + temporary_path = None + else: + try: + os.link(temporary_path, target) + except FileExistsError as exc: + raise FileExistsError( + f"Preset already exists: {target}. Pass --overwrite to replace it." + ) from exc + except OSError as exc: + raise OSError( + "Could not atomically publish the preset without overwriting " + f"the target; the target was left unchanged: {exc}" + ) from exc + try: + temporary_path.unlink() + except OSError: + # Publication succeeded; the final cleanup below retries. + pass + else: + temporary_path = None + finally: + if temporary_path is not None: + try: + temporary_path.unlink() + except OSError: + # Keep a write/replace failure as the primary exception. + pass + return target + + +def load_analysis_preset(path: str | Path) -> dict[str, object]: + """Read, fully validate, and return one flat GUI-ready analysis preset. + + The returned mapping is constructed only after every schema, type, and + scientific-settings check succeeds, so callers can apply it atomically. + """ + + source = _target_path(path) + with source.open("rb") as stream: + raw = stream.read(_MAX_PRESET_BYTES + 1) + if len(raw) > _MAX_PRESET_BYTES: + raise _error(f"preset exceeds the {_MAX_PRESET_BYTES}-byte size limit.") + try: + text = raw.decode("utf-8") + document = json.loads( + text, + object_pairs_hook=_reject_duplicate_pairs, + parse_constant=_reject_nonstandard_constant, + ) + except (UnicodeDecodeError, json.JSONDecodeError) as exc: + raise _error(f"could not parse UTF-8 JSON ({exc}).") from exc + if not isinstance(document, dict): + raise _error("document root must be a JSON object.") + if set(document) != {"schema", "version", "values"}: + missing = sorted({"schema", "version", "values"} - set(document)) + extra = sorted(set(document) - {"schema", "version", "values"}) + details = [] + if missing: + details.append("missing " + ", ".join(missing)) + if extra: + details.append("unknown " + ", ".join(extra)) + raise _error("document fields are invalid: " + "; ".join(details) + ".") + if document["schema"] != _PRESET_SCHEMA: + raise _error(f"unsupported schema {document['schema']!r}.") + version = document["version"] + if type(version) is not int or version != _PRESET_VERSION: + raise _error(f"unsupported version {version!r}; expected {_PRESET_VERSION}.") + return _normalize_values(document["values"], from_form=False) diff --git a/src/diffractscout/inspection.py b/src/diffractscout/inspection.py new file mode 100644 index 0000000..4ea1274 --- /dev/null +++ b/src/diffractscout/inspection.py @@ -0,0 +1,520 @@ +"""Read-only inspection of verified DiffractScout result bundles.""" + +from __future__ import annotations + +import csv +from contextlib import contextmanager +import json +import math +import os +from pathlib import Path +from typing import Any, Iterator + +from .validation import verify_bundle + +INSPECTION_SCHEMA = "diffractscout_inspection_v1" +_MAX_METADATA_JSON_BYTES = 4 * 1024 * 1024 +_MAX_PROVENANCE_JSON_BYTES = 64 * 1024 * 1024 +_MAX_DIAGNOSTIC_PREVIEW = 30 +_MAX_DIAGNOSTIC_TEXT_CHARS = 500 +_MAX_ERROR_PREVIEW = 100 +_REQUIRED_FILES = ( + "phase_summary.csv", + "peak_reference.csv", + "diagnostics.csv", + "provenance.json", + "candidate_index.csv", + "download_index.csv", +) +_OUTPUT_FILES = { + "phase_summary_csv": "phase_summary.csv", + "peak_reference_csv": "peak_reference.csv", + "diagnostics_csv": "diagnostics.csv", + "results_xlsx": "results.xlsx", + "provenance_json": "provenance.json", + "pattern_profiles_csv": "pattern_profiles.csv", + "elasticity_csv": "elasticity.csv", + "candidate_index_csv": "candidate_index.csv", + "download_index_csv": "download_index.csv", + "readme": "README.md", +} + + +def _bounded_text(value: object, limit: int = _MAX_DIAGNOSTIC_TEXT_CHARS) -> str: + text = str(value) + if len(text) <= limit: + return text + return text[: max(0, limit - 1)] + "…" + + +def _verification_summary(report: dict[str, Any]) -> dict[str, Any]: + errors = report.get("errors") + if not isinstance(errors, list): + errors = ["Bundle verification returned an invalid error list."] + return { + "ok": bool(report.get("ok")), + "manifest": str(report.get("manifest") or ""), + "files_checked": len(report.get("files", [])) + if isinstance(report.get("files"), list) + else 0, + "error_count": len(errors), + "errors": [_bounded_text(error, 2_000) for error in errors[:_MAX_ERROR_PREVIEW]], + "errors_truncated": len(errors) > _MAX_ERROR_PREVIEW, + } + + +def _failed_report( + bundle: Path, + verification: dict[str, Any], + inspection_errors: list[str] | None = None, +) -> dict[str, Any]: + report: dict[str, Any] = { + "schema": INSPECTION_SCHEMA, + "bundle": str(bundle), + "ok": False, + "verification": verification, + } + if inspection_errors is not None: + report["inspection"] = { + "ok": False, + "errors": [_bounded_text(error, 2_000) for error in inspection_errors], + } + return report + + +def _reject_duplicate_json_keys(pairs: list[tuple[str, Any]]) -> dict[str, Any]: + result: dict[str, Any] = {} + for key, value in pairs: + if key in result: + raise ValueError(f"duplicate JSON key: {key}") + result[key] = value + return result + + +def _reject_json_constant(value: str) -> None: + raise ValueError(f"non-standard JSON value: {value}") + + +def _read_limited_json(path: Path, *, label: str, max_bytes: int) -> dict[str, Any]: + size = path.stat().st_size + if size > max_bytes: + raise ValueError( + f"{label} exceeds the {max_bytes}-byte inspection limit." + ) + with path.open("rb") as handle: + data = handle.read(max_bytes + 1) + if len(data) > max_bytes: + raise ValueError(f"{label} changed while it was being read.") + payload = json.loads( + data.decode("utf-8"), + object_pairs_hook=_reject_duplicate_json_keys, + parse_constant=_reject_json_constant, + ) + if not isinstance(payload, dict): + raise ValueError(f"{label} root must be a JSON object.") + return payload + + +def _read_small_json(path: Path, *, label: str) -> dict[str, Any]: + return _read_limited_json( + path, + label=label, + max_bytes=_MAX_METADATA_JSON_BYTES, + ) + + +def _read_provenance_fields(path: Path) -> dict[str, Any]: + payload = _read_limited_json( + path, + label="provenance.json", + max_bytes=_MAX_PROVENANCE_JSON_BYTES, + ) + return payload + + +def _safe_manifest_entries(manifest: dict[str, Any]) -> dict[str, dict[str, Any]]: + raw_entries = manifest.get("files") + if not isinstance(raw_entries, list): + raise ValueError("Manifest 'files' must be a list.") + entries: dict[str, dict[str, Any]] = {} + for index, entry in enumerate(raw_entries): + if not isinstance(entry, dict): + raise ValueError(f"Manifest file entry {index} must be an object.") + raw_path = entry.get("path") + if not isinstance(raw_path, str) or not raw_path.strip(): + raise ValueError(f"Manifest file entry {index} has no path.") + relative = Path(raw_path) + normalized = relative.as_posix() + if ( + relative.is_absolute() + or normalized in {".", "manifest.json"} + or ".." in relative.parts + or any(part in {"", "."} for part in relative.parts) + ): + raise ValueError(f"Unsafe manifest path: {raw_path!r}.") + if normalized in entries: + raise ValueError(f"Duplicate manifest path: {normalized}.") + entries[normalized] = entry + return entries + + +@contextmanager +def _csv_rows(path: Path) -> Iterator[tuple[list[str], Iterator[list[str]]]]: + """Open a CSV as a stream and reject ambiguous or malformed row shapes.""" + + with path.open("r", encoding="utf-8-sig", newline="") as handle: + reader = csv.reader(handle, strict=True) + headers = next(reader, None) + if not headers or any(not header for header in headers): + raise ValueError(f"{path.name} has no complete header row.") + if len(set(headers)) != len(headers): + raise ValueError(f"{path.name} contains duplicate column names.") + + def rows() -> Iterator[list[str]]: + for row in reader: + if not row: + continue + if len(row) != len(headers): + raise ValueError( + f"{path.name} row {reader.line_num} has {len(row)} fields; " + f"expected {len(headers)}." + ) + yield row + + yield headers, rows() + + +def _count_phases(path: Path) -> tuple[int, int]: + with _csv_rows(path) as (headers, rows): + required = {"phase_name", "reflection_count"} + missing = required - set(headers) + if missing: + raise ValueError( + f"{path.name} is missing required columns: {', '.join(sorted(missing))}." + ) + reflection_index = headers.index("reflection_count") + count = 0 + reflections = 0 + for row in rows: + count += 1 + try: + phase_reflections = int(row[reflection_index]) + except ValueError as exc: + raise ValueError( + f"{path.name} row {count + 1} has an invalid reflection_count." + ) from exc + if phase_reflections < 0: + raise ValueError( + f"{path.name} row {count + 1} has a negative reflection_count." + ) + reflections += phase_reflections + return count, reflections + + +def _count_reflections(path: Path) -> int: + with _csv_rows(path) as (headers, rows): + if "phase_name" not in headers: + raise ValueError(f"{path.name} is missing the phase_name column.") + return sum(1 for _ in rows) + + +def _count_records(path: Path, *, required_column: str) -> int: + with _csv_rows(path) as (headers, rows): + if required_column not in headers: + raise ValueError( + f"{path.name} is missing the {required_column} column." + ) + return sum(1 for _ in rows) + + +def _inspect_diagnostics(path: Path) -> tuple[dict[str, int], list[dict[str, str]]]: + levels = {"info": 0, "warning": 0, "error": 0, "other": 0} + preview: list[dict[str, str]] = [] + with _csv_rows(path) as (headers, rows): + required = {"stage", "item", "level", "message"} + missing = required - set(headers) + if missing: + raise ValueError( + f"{path.name} is missing required columns: {', '.join(sorted(missing))}." + ) + indexes = {key: headers.index(key) for key in required} + for row in rows: + level = row[indexes["level"]].strip().lower() + levels[level if level in {"info", "warning", "error"} else "other"] += 1 + if len(preview) < _MAX_DIAGNOSTIC_PREVIEW: + preview.append( + { + "stage": _bounded_text(row[indexes["stage"]]), + "item": _bounded_text(row[indexes["item"]]), + "level": _bounded_text(row[indexes["level"]], 100), + "message": _bounded_text(row[indexes["message"]]), + } + ) + return levels, preview + + +def _validate_settings(value: object) -> dict[str, object] | None: + if value is None: + return None + if not isinstance(value, dict): + raise ValueError("provenance.json is missing the analysis_settings object.") + result: dict[str, object] = {} + for key, item in value.items(): + if not isinstance(key, str): + raise ValueError("analysis_settings keys must be strings.") + if item is None or isinstance(item, (str, bool, int)): + result[key] = _bounded_text(item) if isinstance(item, str) else item + elif isinstance(item, float) and math.isfinite(item): + result[key] = item + else: + raise ValueError(f"analysis setting {key!r} is not a finite scalar value.") + return result + + +def inspect_bundle(bundle: str | Path) -> dict[str, Any]: + """Verify and summarize a result bundle without modifying or executing it. + + A bundle that fails integrity verification is returned without result counts, + settings, or file conclusions. CSVs are read row by row, and diagnostic text + is bounded in the report while the severity counts include every row. + """ + + supplied = Path(bundle).expanduser() + supplied = Path(os.path.abspath(os.fspath(supplied))) + manifest_path = supplied if supplied.name == "manifest.json" else supplied / "manifest.json" + root = manifest_path.parent + + try: + manifest_size = manifest_path.stat().st_size + except OSError: + # Missing, inaccessible, and unsafe paths are diagnosed by the bundle + # verifier; this preflight exists only to bound its JSON input size. + pass + else: + if manifest_size > _MAX_METADATA_JSON_BYTES: + verification = _verification_summary( + { + "ok": False, + "manifest": str(manifest_path), + "files": [], + "errors": [ + "manifest.json exceeds the " + f"{_MAX_METADATA_JSON_BYTES}-byte inspection limit." + ], + } + ) + return _failed_report(root, verification) + + try: + verification_raw = verify_bundle(supplied) + except (OSError, RecursionError, RuntimeError, UnicodeError, ValueError) as exc: + verification_raw = { + "ok": False, + "manifest": str(manifest_path), + "files": [], + "errors": [f"Bundle verification failed: {type(exc).__name__}: {exc}"], + } + verification = _verification_summary(verification_raw) + if not verification["ok"]: + return _failed_report(root, verification) + + try: + manifest = _read_small_json(manifest_path, label="manifest.json") + if manifest.get("schema") != "diffractscout_bundle_manifest_v1": + raise ValueError("manifest.json has an unsupported bundle schema.") + entries = _safe_manifest_entries(manifest) + missing = [name for name in _REQUIRED_FILES if name not in entries] + if missing: + raise ValueError( + "Manifest does not list required bundle files: " + ", ".join(missing) + "." + ) + + # Only fixed, known filenames are opened. Manifest entries are used to + # confirm presence and describe the inventory; they never select a path + # that this module will read. + phase_path = root / "phase_summary.csv" + peak_path = root / "peak_reference.csv" + diagnostic_path = root / "diagnostics.csv" + candidate_path = root / "candidate_index.csv" + download_path = root / "download_index.csv" + provenance_path = root / "provenance.json" + phase_count, phase_reflection_count = _count_phases(phase_path) + reflection_count = _count_reflections(peak_path) + candidate_count = _count_records(candidate_path, required_column="material_id") + download_count = _count_records(download_path, required_column="material_id") + diagnostic_levels, diagnostic_preview = _inspect_diagnostics(diagnostic_path) + diagnostic_count = sum(diagnostic_levels.values()) + provenance = _read_provenance_fields(provenance_path) + if provenance.get("schema") != "diffractscout_provenance_v1": + raise ValueError("provenance.json has an unsupported schema.") + software_versions = provenance.get("software_versions") + if not isinstance(software_versions, dict): + raise ValueError("provenance.json is missing the software_versions object.") + version = software_versions.get("diffractscout") + if not isinstance(version, str) or not version.strip(): + raise ValueError("provenance.json is missing the DiffractScout version.") + if "analysis_settings" not in provenance: + raise ValueError("provenance.json is missing the analysis_settings field.") + settings = _validate_settings(provenance["analysis_settings"]) + + inspection_errors: list[str] = [] + if phase_reflection_count != reflection_count: + inspection_errors.append( + "phase_summary.csv reflection counts do not match peak_reference.csv rows." + ) + manifest_summary = manifest.get("summary") + if manifest_summary is not None and not isinstance(manifest_summary, dict): + inspection_errors.append("Manifest 'summary' must be an object.") + if isinstance(manifest_summary, dict): + actual_counts = { + "phase_count": phase_count, + "reflection_count": reflection_count, + "candidate_count": candidate_count, + "download_count": download_count, + "diagnostic_count": diagnostic_count, + "error_count": diagnostic_levels["error"], + } + for key, actual in actual_counts.items(): + declared = manifest_summary.get(key) + if declared is not None and ( + isinstance(declared, bool) + or not isinstance(declared, int) + or declared != actual + ): + inspection_errors.append( + f"Manifest summary {key} does not match the verified CSV contents." + ) + + if inspection_errors: + return _failed_report(root, verification, inspection_errors) + + outputs = { + output_key: ( + str(root / filename) if filename in entries else None + ) + for output_key, filename in _OUTPUT_FILES.items() + } + members = [] + verification_paths = { + item.get("path") + for item in verification_raw.get("files", []) + if isinstance(item, dict) and item.get("ok") is True + } + for relative, entry in entries.items(): + members.append( + { + "path": relative, + "role": entry.get("role") if isinstance(entry.get("role"), str) else None, + "size_bytes": entry.get("size_bytes"), + "verified": relative in verification_paths, + } + ) + + report: dict[str, Any] = { + "schema": INSPECTION_SCHEMA, + "bundle": str(root), + "ok": not inspection_errors, + "verification": verification, + "inspection": {"ok": not inspection_errors, "errors": inspection_errors}, + "software_version": version.strip(), + "analysis_settings": settings, + "counts": { + "phases": phase_count, + "reflections": reflection_count, + "candidates": candidate_count, + "downloads": download_count, + "diagnostics": diagnostic_count, + "diagnostics_by_level": diagnostic_levels, + }, + "files": { + "manifest": str(manifest_path), + "count": len(members), + "outputs": outputs, + "members": members, + }, + "diagnostics_preview": diagnostic_preview, + "diagnostics_preview_truncated": diagnostic_count > len(diagnostic_preview), + } + return report + except (OSError, csv.Error, RecursionError, RuntimeError, UnicodeError, ValueError) as exc: + return _failed_report( + root, + verification, + [f"Could not inspect verified bundle: {type(exc).__name__}: {exc}"], + ) + + +def format_inspection(report: dict[str, Any]) -> str: + """Return a concise human-readable report for :func:`inspect_bundle`.""" + + lines = [f"Bundle: {report.get('bundle', '(unknown)')}"] + verification = report.get("verification") + if isinstance(verification, dict): + if verification.get("ok"): + lines.append( + "Verification: passed " + f"({verification.get('files_checked', 0)} files checked)" + ) + else: + lines.append("Verification: failed") + for error in verification.get("errors", []): + lines.append(f" - {error}") + if verification.get("errors_truncated"): + lines.append(" - Additional verification errors omitted.") + + inspection = report.get("inspection") + if isinstance(inspection, dict) and not inspection.get("ok"): + lines.append("Inspection: failed") + for error in inspection.get("errors", []): + lines.append(f" - {error}") + return "\n".join(lines) + if not report.get("ok"): + return "\n".join(lines) + + lines.append(f"DiffractScout version: {report.get('software_version', 'unknown')}") + counts = report.get("counts") + if isinstance(counts, dict): + levels = counts.get("diagnostics_by_level", {}) + lines.append( + "Results: " + f"{counts.get('phases', 0)} phases, " + f"{counts.get('reflections', 0)} reflections, " + f"{counts.get('candidates', 0)} candidates, " + f"{counts.get('downloads', 0)} downloads, " + f"{counts.get('diagnostics', 0)} diagnostics " + f"(info {levels.get('info', 0)}, warning {levels.get('warning', 0)}, " + f"error {levels.get('error', 0)}, other {levels.get('other', 0)})" + ) + + settings = report.get("analysis_settings") + if isinstance(settings, dict): + lines.append("Analysis settings:") + for key, value in settings.items(): + lines.append(f" {key}: {value}") + elif settings is None and report.get("ok"): + lines.append("Analysis settings: not recorded") + + files = report.get("files") + if isinstance(files, dict): + lines.append(f"Files: {files.get('count', 0)} verified bundle members") + outputs = files.get("outputs") + if isinstance(outputs, dict): + lines.append("Key outputs:") + for name, path in outputs.items(): + lines.append(f" {name}: {path if path is not None else 'not included'}") + + previews = report.get("diagnostics_preview") + if isinstance(previews, list) and previews: + lines.append("Diagnostics:") + for item in previews: + if isinstance(item, dict): + lines.append( + f" [{item.get('level', 'unknown')}] " + f"{item.get('stage', '')} / {item.get('item', '')}: " + f"{item.get('message', '')}" + ) + if report.get("diagnostics_preview_truncated"): + lines.append(" Additional diagnostics omitted from display.") + elif isinstance(counts, dict) and counts.get("diagnostics") == 0: + lines.append("Diagnostics: none") + return "\n".join(lines) diff --git a/src/diffractscout/quick_export.py b/src/diffractscout/quick_export.py index 9d7af7c..ccb5706 100644 --- a/src/diffractscout/quick_export.py +++ b/src/diffractscout/quick_export.py @@ -16,9 +16,10 @@ from pathlib import Path from typing import Mapping, Sequence -from .models import AnalysisSettings, PipelineResult, XrayInputMode +from .cli_presets import add_analysis_options, resolve_cli_analysis +from .models import AnalysisSettings, PipelineResult from .pipeline import _acquire_transaction_lock, _release_transaction_lock, analyze_cifs -from .utils import sha256_file, to_jsonable +from .utils import sha256_file # Keyword names accepted as AnalysisSettings fields when building defaults. _SETTINGS_KEYS = frozenset(AnalysisSettings.__dataclass_fields__) @@ -192,6 +193,28 @@ def _copy_excel_atomic(source: Path, target: Path, *, overwrite: bool) -> None: pass +def _add_bundle_recovery_context( + error: OSError | RuntimeError, *, target: Path, bundle_dir: Path, workbook: Path +) -> None: + """Explain bundle recovery without changing exception type or target metadata.""" + + context = f"Excel publication failed for target {target}. A bundle remains at {bundle_dir}. " + if workbook.is_file(): + context += ( + f"Run `diffractscout verify {bundle_dir}` before reusing it. If verification " + f"passes, recover the workbook from {workbook}." + ) + else: + context += ( + f"The expected workbook {workbook} is missing. Run `diffractscout verify " + f"{bundle_dir}` and rerun the export if needed." + ) + if isinstance(error, OSError) and error.strerror: + error.strerror = f"{error.strerror}. {context}" + else: + error.args = (f"{error}. {context}", *error.args[1:]) + + def _default_settings(**overrides: object) -> AnalysisSettings: """Cu Kα, 5–120°, lab views on; other fields match AnalysisSettings defaults.""" @@ -365,9 +388,13 @@ def quick_export( excel_target: Path | None = None if output_path.suffix.lower() == ".xlsx": excel_target = output_path if output_path.is_absolute() else output_path.resolve() + if not include_excel: + raise ValueError( + "An .xlsx output path requires Excel output; include_excel=False " + "(--no-excel) is incompatible. Choose a bundle directory to skip Excel." + ) bundle_dir = excel_target.with_name(f"{excel_target.stem}_bundle") # Excel shortcut always materializes the workbook in the bundle first. - include_excel = True else: bundle_dir = output_path if output_path.is_absolute() else output_path.resolve() @@ -388,11 +415,20 @@ def quick_export( elastic_overrides=elastic_overrides, # type: ignore[arg-type] ) source_xlsx = result.output_dir / "results.xlsx" - if not source_xlsx.is_file(): - raise RuntimeError( - f"Expected results.xlsx in bundle {result.output_dir}, but it is missing." + try: + if not source_xlsx.is_file(): + raise RuntimeError( + f"Expected results.xlsx in bundle {result.output_dir}, but it is missing." + ) + _copy_excel_atomic(source_xlsx, excel_target, overwrite=overwrite) + except (OSError, RuntimeError) as exc: + _add_bundle_recovery_context( + exc, + target=excel_target, + bundle_dir=result.output_dir, + workbook=source_xlsx, ) - _copy_excel_atomic(source_xlsx, excel_target, overwrite=overwrite) + raise return result finally: _release_transaction_lock(lock_path, warning_sink=warning_sink) @@ -423,62 +459,7 @@ def build_parser() -> argparse.ArgumentParser: required=True, help="Bundle directory, or an .xlsx path (bundle becomes _bundle/).", ) - parser.add_argument("--no-recursive", action="store_true") - parser.add_argument("--no-excel", action="store_true", help="Skip Excel (bundle dir mode only).") - parser.add_argument("--overwrite", action="store_true") - parser.add_argument("--json", action="store_true") - parser.add_argument("--source", default="Cu Ka") - radiation = parser.add_mutually_exclusive_group() - radiation.add_argument( - "--wavelength-A", - type=float, - default=None, - help="Explicit X-ray wavelength in Angstrom (selects wavelength mode).", - ) - radiation.add_argument( - "--energy-keV", - type=float, - default=None, - help="Explicit photon energy in keV (selects energy mode).", - ) - parser.add_argument("--two-theta-min", type=float, default=5.0) - parser.add_argument("--two-theta-max", type=float, default=120.0) - parser.add_argument("--step", type=float, default=0.02) - parser.add_argument("--fwhm", type=float, default=0.15) - parser.add_argument("--eta", type=float, default=0.5) - parser.add_argument("--no-elasticity", action="store_true") - parser.add_argument( - "--max-profile-points", - type=int, - default=1_000_000, - help="Safety limit for the generated display-profile grid.", - ) - parser.add_argument( - "--max-reflection-estimate", - type=int, - default=2_000_000, - help="Safety limit for reciprocal-lattice candidate generation.", - ) - parser.add_argument("--d-min", type=float, default=None, dest="d_min") - parser.add_argument("--d-max", type=float, default=None, dest="d_max") - parser.add_argument( - "--profile-model", - choices=("pseudo_voigt", "gaussian", "lorentzian"), - default="pseudo_voigt", - ) - parser.add_argument( - "--pattern-axis", - choices=("two_theta", "d_spacing", "q", "g"), - default="two_theta", - help=( - "Selected x coordinate in pattern_profiles.csv and Excel. " - "Figures remain on 2theta in v0.4.0." - ), - ) - parser.add_argument("--figures", action="store_true") - parser.add_argument("--figure-preset", default="publication") - parser.add_argument("--no-lab-views", action="store_true") - parser.add_argument("--no-patterns", action="store_true") + add_analysis_options(parser) return parser @@ -486,51 +467,13 @@ def main(argv: Sequence[str] | None = None) -> int: parser = build_parser() args = parser.parse_args(argv) try: - input_mode: XrayInputMode - if args.wavelength_A is not None: - input_mode = "wavelength" - wavelength_A = args.wavelength_A - energy_keV = None - elif args.energy_keV is not None: - input_mode = "energy" - wavelength_A = None - energy_keV = args.energy_keV - else: - input_mode = "source" - wavelength_A = None - energy_keV = None - if args.source == "Custom": - raise ValueError( - "Custom source requires --wavelength-A or --energy-keV." - ) - settings = AnalysisSettings( - input_mode=input_mode, - source_preset=args.source, - wavelength_A=wavelength_A, - energy_keV=energy_keV, - two_theta_min_deg=args.two_theta_min, - two_theta_max_deg=args.two_theta_max, - step_deg=args.step, - fwhm_deg=args.fwhm, - profile_eta=args.eta, - include_elasticity=not args.no_elasticity, - max_profile_points=args.max_profile_points, - max_reflection_estimate=args.max_reflection_estimate, - d_min_A=args.d_min, - d_max_A=args.d_max, - profile_model=args.profile_model, - pattern_axis=args.pattern_axis, - include_figures=bool(args.figures), - figure_preset=args.figure_preset, - export_lab_views=not args.no_lab_views, - include_patterns=not args.no_patterns, - ) + analysis = resolve_cli_analysis(args) result = quick_export( args.inputs, args.output, - settings=settings, - recursive=not args.no_recursive, - include_excel=not args.no_excel, + settings=analysis.settings, + recursive=analysis.recursive, + include_excel=analysis.include_excel, overwrite=args.overwrite, ) except ( @@ -545,24 +488,11 @@ def main(argv: Sequence[str] | None = None) -> int: print(f"ERROR: {exc}", file=sys.stderr) return 2 - if args.json: - import json + from .cli import _pipeline_exit_code, _print_result - print(json.dumps(to_jsonable(result), indent=2, ensure_ascii=False)) - else: - print(f"Output: {result.output_dir}") - print(f"Manifest: {result.manifest_path}") - print(f"Analyzed phases: {len(result.analyses)}") - if str(args.output).lower().endswith(".xlsx"): - print(f"Excel: {Path(args.output).expanduser().resolve()}") - for warning in result.warnings: - print(f"WARNING: {warning}", file=sys.stderr) - - if not result.analyses: - return 2 - if any(item.level == "error" for item in result.diagnostics): - return 3 - return 0 + excel_path = args.output if Path(args.output).suffix.lower() == ".xlsx" else None + _print_result(result, as_json=args.json, excel_path=excel_path) + return _pipeline_exit_code(result) if __name__ == "__main__": diff --git a/tests/test_cli.py b/tests/test_cli.py index 05ba88b..2f9328e 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -1,9 +1,20 @@ +import json from pathlib import Path +from types import SimpleNamespace import pytest -from diffractscout.cli import _analysis_settings, _pipeline_exit_code, build_parser, main -from diffractscout.models import DiagnosticRecord +from diffractscout.cli import ( + _analysis_settings, + _diagnostic_lines, + _pipeline_exit_code, + _print_result, + build_parser, + main, +) +from diffractscout.models import DiagnosticRecord, PipelineResult +import diffractscout.cli_presets as cli_presets +from diffractscout.cli_presets import resolve_cli_analysis, save_cli_preset from diffractscout.validation import verify_bundle @@ -13,6 +24,90 @@ def test_demo_cli(tmp_path: Path) -> None: assert verify_bundle(output)["ok"] +def test_cli_partial_batch_reports_severity_counts_and_artifact_paths( + demo_inputs: Path, tmp_path: Path, capsys +) -> None: + (demo_inputs / "empty.cif").write_text("", encoding="utf-8") + output = tmp_path / "partial" + + code = main( + ["analyze", str(demo_inputs), "-o", str(output), "--no-excel"] + ) + + captured = capsys.readouterr() + assert code == 3 + assert "ERROR: empty.cif:" in captured.err + assert "WARNING: empty.cif:" not in captured.err + assert "Analyzed phases: 1" in captured.out + reflection_line = next( + line for line in captured.out.splitlines() if line.startswith("Reflections: ") + ) + assert int(reflection_line.split(": ", 1)[1]) > 0 + assert "Warnings: 0" in captured.out + assert "Errors: 1" in captured.out + assert f"Diagnostics: {output / 'diagnostics.csv'}" in captured.out + assert "Excel:" not in captured.out + + +def test_cli_json_keeps_pipeline_result_serialization_contract( + tmp_path: Path, capsys +) -> None: + output = tmp_path / "json-demo" + code = main(["demo", "-o", str(output), "--no-excel", "--json"]) + + payload = json.loads(capsys.readouterr().out) + assert code == 0 + assert set(payload) == { + "output_dir", + "discovery", + "downloads", + "analyses", + "manifest_path", + "warnings", + "diagnostics", + } + assert "summary" not in payload + assert "artifact_paths" not in payload + + +def test_cli_summary_deduplicates_diagnostics_and_legacy_warnings( + tmp_path: Path, capsys +) -> None: + bundle = tmp_path / "summary" + bundle.mkdir() + (bundle / "diagnostics.csv").write_text("diagnostics", encoding="utf-8") + (bundle / "results.xlsx").write_bytes(b"workbook") + message = "failed to parse" + result = PipelineResult( + output_dir=bundle, + discovery=None, + downloads=[], + analyses=[SimpleNamespace(reflections=[object(), object()])], # type: ignore[list-item] + manifest_path=bundle / "manifest.json", + warnings=[f"broken.cif: {message}", "transaction cleanup", "transaction cleanup"], + diagnostics=[ + DiagnosticRecord("analysis", "broken.cif", "error", message), + DiagnosticRecord("analysis", "broken.cif", "error", message), + ], + ) + + lines = _diagnostic_lines(result) + _print_result(result) + captured = capsys.readouterr() + + assert lines == [ + ("ERROR", f"broken.cif: {message}"), + ("WARNING", "transaction cleanup"), + ] + assert f"Excel: {bundle / 'results.xlsx'}" in captured.out + assert f"Diagnostics: {bundle / 'diagnostics.csv'}" in captured.out + assert "Reflections: 2" in captured.out + assert "Warnings: 1" in captured.out + assert "Errors: 1" in captured.out + assert captured.err.count("ERROR: broken.cif:") == 1 + assert captured.err.count("WARNING: transaction cleanup") == 1 + + def test_analysis_cli_flags_parse() -> None: parser = build_parser() args = parser.parse_args( @@ -76,6 +171,283 @@ def test_quick_export_help_is_legacy_windows_console_safe(capsys) -> None: help_text.encode("cp936") +def test_analysis_parser_explicit_tracking_does_not_leak_between_parses() -> None: + parser = build_parser() + explicit = parser.parse_args( + ["analyze", "sample.cif", "-o", "first", "--no-excel", "--two-theta-max", "80"] + ) + defaults = parser.parse_args(["analyze", "sample.cif", "-o", "second"]) + + assert explicit._diffractscout_explicit == {"no_excel", "two_theta_max"} + assert not getattr(defaults, "_diffractscout_explicit", set()) + + +def test_preset_save_show_and_cli_values_override_preset(tmp_path: Path, capsys) -> None: + preset_path = tmp_path / "analysis.json" + assert main( + [ + "preset", + "save", + "-o", + str(preset_path), + "--energy-keV", + "30", + "--two-theta-max", + "75", + "--d-min", + "0.8", + "--d-max", + "3.2", + "--no-excel", + "--no-elasticity", + "--no-lab-views", + "--no-patterns", + "--no-figures", + "--no-recursive", + ] + ) == 0 + assert preset_path.is_file() + assert not (tmp_path / "analysis").is_dir() + + parser = build_parser() + args = parser.parse_args( + [ + "analyze", + "sample.cif", + "-o", + str(tmp_path / "bundle"), + "--preset", + str(preset_path), + "--excel", + "--elasticity", + "--lab-views", + "--patterns", + "--figures", + "--recursive", + "--two-theta-max", + "80", + "--clear-d-min", + ] + ) + resolved = resolve_cli_analysis(args) + assert resolved.settings.input_mode == "energy" + assert resolved.settings.energy_keV == pytest.approx(30) + assert resolved.settings.two_theta_max_deg == pytest.approx(80) + assert resolved.settings.d_min_A is None + assert resolved.settings.d_max_A == pytest.approx(3.2) + assert resolved.settings.include_elasticity is True + assert resolved.settings.export_lab_views is True + assert resolved.settings.include_patterns is True + assert resolved.settings.include_figures is True + assert resolved.include_excel is True + assert resolved.recursive is True + assert main(["preset", "show", str(preset_path), "--json"]) == 0 + shown = json.loads(capsys.readouterr().out.split("Preset saved:", 1)[-1].split("\n", 1)[-1]) + assert shown["input_mode"] == "energy" + assert shown["include_excel"] is False + assert shown["local_recursive"] is False + assert len(shown) == 20 + + enabled_preset = tmp_path / "enabled.json" + assert main( + [ + "preset", + "save", + "-o", + str(enabled_preset), + "--excel", + "--elasticity", + "--lab-views", + "--patterns", + "--figures", + "--recursive", + ] + ) == 0 + disabled = resolve_cli_analysis( + build_parser().parse_args( + [ + "analyze", + "sample.cif", + "-o", + "out", + "--preset", + str(enabled_preset), + "--no-excel", + "--no-elasticity", + "--no-lab-views", + "--no-patterns", + "--no-figures", + "--no-recursive", + ] + ) + ) + assert disabled.include_excel is False + assert disabled.recursive is False + assert disabled.settings.include_elasticity is False + assert disabled.settings.export_lab_views is False + assert disabled.settings.include_patterns is False + assert disabled.settings.include_figures is False + + +def test_preset_radiation_switching_uses_units_and_source_mode(tmp_path: Path) -> None: + energy_preset = tmp_path / "energy.json" + assert main( + ["preset", "save", "-o", str(energy_preset), "--energy-keV", "30"] + ) == 0 + parser = build_parser() + builtin = resolve_cli_analysis( + parser.parse_args( + ["analyze", "sample.cif", "-o", "out", "--preset", str(energy_preset), "--source", "Co Ka"] + ) + ) + assert builtin.settings.input_mode == "source" + assert builtin.settings.source_preset == "Co Ka" + assert builtin.settings.energy_keV is None + assert builtin.settings.wavelength_A is None + + with pytest.raises(ValueError, match="cannot be reinterpreted as Angstrom"): + resolve_cli_analysis( + parser.parse_args( + ["analyze", "sample.cif", "-o", "out", "--preset", str(energy_preset), "--source", "Custom"] + ) + ) + + custom_preset = tmp_path / "custom.json" + assert main( + [ + "preset", + "save", + "-o", + str(custom_preset), + "--source", + "Custom", + "--wavelength-A", + "1.2", + ] + ) == 0 + custom = resolve_cli_analysis( + parser.parse_args(["analyze", "sample.cif", "-o", "out", "--preset", str(custom_preset)]) + ) + assert custom.settings.input_mode == "wavelength" + assert custom.settings.source_preset == "Custom" + assert custom.settings.wavelength_A == pytest.approx(1.2) + + +def test_explicit_command_default_overrides_preset_value(tmp_path: Path) -> None: + preset_path = tmp_path / "non-default.json" + assert main( + ["preset", "save", "-o", str(preset_path), "--two-theta-max", "75"] + ) == 0 + args = build_parser().parse_args( + [ + "analyze", + "sample.cif", + "-o", + "out", + "--preset", + str(preset_path), + "--two-theta-max", + "120", + ] + ) + assert resolve_cli_analysis(args).settings.two_theta_max_deg == pytest.approx(120) + + +def test_preset_save_refuses_existing_files_and_result_bundles( + tmp_path: Path, monkeypatch, capsys +) -> None: + preset_path = tmp_path / "existing.json" + preset_path.write_text("preserve", encoding="utf-8") + assert main(["preset", "save", "-o", str(preset_path)]) == 2 + assert preset_path.read_text(encoding="utf-8") == "preserve" + assert main(["preset", "save", "-o", str(preset_path), "--overwrite"]) == 0 + + bundle = tmp_path / "bundle" + assert main(["demo", "-o", str(bundle), "--no-excel"]) == 0 + manifest_path = bundle / "manifest.json" + manifest_before = manifest_path.read_bytes() + preset_inside_bundle = bundle / "parameters.json" + assert main(["preset", "save", "-o", str(preset_inside_bundle), "--overwrite"]) == 2 + assert not preset_inside_bundle.exists() + assert main(["preset", "save", "-o", str(manifest_path), "--overwrite"]) == 2 + assert manifest_path.read_bytes() == manifest_before + assert verify_bundle(bundle)["ok"] + + target = tmp_path / "symlink.json" + original = Path.is_symlink + monkeypatch.setattr(Path, "is_symlink", lambda path: path == target or original(path)) + with pytest.raises(ValueError, match="symbolic link"): + save_cli_preset(target, resolve_cli_analysis(build_parser().parse_args( + ["analyze", "sample.cif", "-o", "unused"] + )).preset_values) + assert "Pass --overwrite" in capsys.readouterr().err + + +def test_preset_save_no_clobber_is_atomic_against_competing_creator( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + target = tmp_path / "racing-preset.json" + analysis = resolve_cli_analysis( + build_parser().parse_args(["analyze", "sample.cif", "-o", "unused"]) + ) + save_impl = cli_presets.save_analysis_preset + + def create_competing_target( + path: str | Path, + values: dict[str, object], + *, + overwrite: bool = True, + ) -> Path: + # This executes after save_cli_preset's existence check and before its + # atomic publication attempt. + target.write_text("created by competing process", encoding="utf-8") + return save_impl(path, values, overwrite=overwrite) + + monkeypatch.setattr(cli_presets, "save_analysis_preset", create_competing_target) + + with pytest.raises(FileExistsError, match="Pass --overwrite"): + save_cli_preset(target, analysis.preset_values) + + assert target.read_text(encoding="utf-8") == "created by competing process" + assert list(tmp_path.glob(".diffractscout-preset-*.tmp")) == [] + + +def test_invalid_preset_fails_before_analysis_or_output_creation( + tmp_path: Path, monkeypatch, capsys +) -> None: + import diffractscout.cli as cli + + preset_path = tmp_path / "invalid.json" + preset_path.write_text("{}", encoding="utf-8") + output = tmp_path / "should-not-exist" + called = False + + def fail_if_called(*_args: object, **_kwargs: object) -> object: + nonlocal called + called = True + raise AssertionError("analysis should not be called") + + monkeypatch.setattr(cli, "analyze_cifs", fail_if_called) + assert main( + ["analyze", "sample.cif", "-o", str(output), "--preset", str(preset_path)] + ) == 2 + assert not called + assert not output.exists() + assert "Invalid analysis preset" in capsys.readouterr().err + + +def test_inspect_cli_returns_json_inspection_report(tmp_path: Path, capsys) -> None: + bundle = tmp_path / "inspect-demo" + assert main(["demo", "-o", str(bundle), "--no-excel"]) == 0 + capsys.readouterr() + assert main(["inspect", str(bundle), "--json"]) == 0 + report = json.loads(capsys.readouterr().out) + assert report["schema"] == "diffractscout_inspection_v1" + assert report["ok"] is True + assert report["verification"]["ok"] is True + assert report["counts"]["phases"] >= 1 + + def test_cli_empty_input_directory_returns_error_exit_2( tmp_path: Path, capsys ) -> None: diff --git a/tests/test_excel_overview.py b/tests/test_excel_overview.py new file mode 100644 index 0000000..f318bc5 --- /dev/null +++ b/tests/test_excel_overview.py @@ -0,0 +1,215 @@ +"""Focused checks for the workbook overview and its internal navigation.""" + +from __future__ import annotations + +from openpyxl import Workbook, load_workbook + +from diffractscout.excel_overview import add_overview_sheet + + +def _workbook_with_data_sheets() -> Workbook: + workbook = Workbook() + workbook.active.title = "Summary" + workbook.create_sheet("User's Results") + workbook.create_sheet("Diagnostics") + return workbook + + +def test_overview_summarizes_supplied_rows_and_all_phases(tmp_path) -> None: + workbook = _workbook_with_data_sheets() + phases = [ + { + "phase_name": f"phase-{index}", + "formula": "Al2O3", + "space_group_symbol": "R -3 c", + "space_group_number": 167, + "reflection_count": index, + "elastic_status": "not_requested", + } + for index in range(1, 27) + ] + summary = [ + {"key": "effective_wavelength_A", "value": 1.540612345678}, + {"key": "effective_energy_keV", "value": 8.047000000123}, + {"key": "effective_radiation_source", "value": "source_preset:Cu Ka"}, + {"key": "effective_two_theta_range_deg", "value": [10.0, 90.0]}, + {"key": "analysis_two_theta_range_deg", "value": [5.0, 120.0]}, + {"key": "profile_sampled_two_theta_range_deg", "value": [5.0, 120.0]}, + {"key": "profile_model", "value": "pseudo_voigt"}, + {"key": "input_mode", "value": "source"}, + {"key": "candidate_count", "value": 4}, + ] + diagnostics = [ + {"level": "error", "message": "failed"}, + {"level": "warning", "message": "check"}, + {"level": "info", "message": "ready"}, + ] + + overview = add_overview_sheet( + workbook, + summary=summary, + phases=phases, + diagnostics=diagnostics, + lab_views=True, + ) + + assert overview.title == "结果概览" + assert overview["B5"].value == 26 + assert overview["D5"].value == sum(range(1, 27)) + assert overview["B6"].value == 1 + assert overview["D6"].value == 1 + assert overview["B11"].value == 1.540612345678 + assert overview["B11"].number_format == "0.####" + assert overview["B12"].value == 8.047000000123 + assert overview["B12"].number_format == "0.####" + assert overview["B13"].value == "Cu Kα" + assert overview["B14"].value == "10–90°" + assert overview["B17"].value == "伪Voigt" + assert overview["B18"].value == "预设光源" + phase_header = next( + row for row in range(1, overview.max_row + 1) if overview.cell(row, 1).value == "物相名称" + ) + assert overview.cell(phase_header + 1, 1).value == "phase-1" + assert overview.cell(phase_header + 26, 1).value == "phase-26" + assert overview.page_setup.orientation == "landscape" + assert str(overview.page_setup.paperSize) == str(overview.PAPERSIZE_A4) + assert overview.page_setup.fitToWidth == 1 + assert overview.page_setup.fitToHeight == 0 + assert {cell.font.name for cells in overview.iter_rows() for cell in cells} == { + "Microsoft YaHei" + } + + path = tmp_path / "overview.xlsx" + workbook.save(path) + reopened = load_workbook(path, data_only=False) + saved = reopened["结果概览"] + navigation_header = next( + row for row in range(1, saved.max_row + 1) if saved.cell(row, 1).value == "工作表" + ) + user_sheet_row = next( + row + for row in range(navigation_header + 1, saved.max_row + 1) + if saved.cell(row, 1).value == "User's Results" + ) + user_sheet_link = saved.cell(user_sheet_row, 1).hyperlink + assert user_sheet_link is not None + assert user_sheet_link.target is None + assert user_sheet_link.location == "'User''s Results'!A1" + + +def test_overview_escapes_untrusted_text_and_leaves_no_formulas() -> None: + workbook = _workbook_with_data_sheets() + summary = [ + {"key": "effective_radiation_source", "value": "=HYPERLINK(\"https://bad.invalid\")"}, + {"key": "profile_model", "value": "+CMD|' /C calc'!A0"}, + {"key": "input_mode", "value": "energy"}, + ] + phases = [ + { + "phase_name": "=1+1", + "formula": "@SUM(A1:A2)", + "space_group_symbol": "P1", + "space_group_number": 1, + "reflection_count": 2, + "elastic_status": "-not checked", + } + ] + + overview = add_overview_sheet( + workbook, + summary=summary, + phases=phases, + diagnostics=[], + lab_views=False, + ) + + assert overview.title == "Overview" + string_values = [ + cell.value + for row in overview.iter_rows() + for cell in row + if isinstance(cell.value, str) + ] + assert any(value.startswith("'=HYPERLINK") for value in string_values) + assert any(value.startswith("'+CMD") for value in string_values) + assert overview["B18"].value == "Photon energy" + phase_header = next( + row for row in range(1, overview.max_row + 1) if overview.cell(row, 1).value == "Phase name" + ) + assert overview.cell(phase_header + 1, 1).value == "'=1+1" + assert overview.cell(phase_header + 1, 2).value == "'@SUM(A1:A2)" + assert overview.cell(phase_header + 1, 5).value == "'-not checked" + assert all( + cell.data_type != "f" + for row in overview.iter_rows() + for cell in row + ) + + +def test_empty_overview_reports_no_results_and_links_only_existing_sheets() -> None: + workbook = Workbook() + workbook.active.title = "Peaks" + + overview = add_overview_sheet( + workbook, + summary=[], + phases=[], + diagnostics=[], + lab_views=False, + ) + + assert overview["B5"].value == 0 + assert overview["D5"].value == 0 + assert any( + cell.value == "No analyzed phases" + for row in overview.iter_rows() + for cell in row + ) + links = [ + cell.hyperlink + for row in overview.iter_rows() + for cell in row + if cell.hyperlink is not None + ] + assert len(links) == 1 + assert links[0].target is None + assert links[0].location == "'Peaks'!A1" + + +def test_long_phase_text_gets_wrapped_row_height_and_scientific_number_format() -> None: + workbook = _workbook_with_data_sheets() + phase = { + "phase_name": "长物相名称" * 12, + "formula": "Al0.12345678901234567890Mg0.98765432109876543210SiO4", + "space_group_symbol": "P 63/mmc", + "space_group_number": 194, + "reflection_count": 18, + "elastic_status": "elasticity not requested; no source record was supplied; status remains unavailable", + } + + overview = add_overview_sheet( + workbook, + summary=[ + {"key": "effective_wavelength_A", "value": 1e-15}, + {"key": "effective_energy_keV", "value": 1e14}, + {"key": "effective_radiation_source", "value": "custom_source_wavelength"}, + ], + phases=[phase], + diagnostics=[], + lab_views=True, + ) + + assert overview["B11"].value == 1e-15 + assert overview["B11"].number_format == "0.0000E+00" + assert overview["B12"].value == 1e14 + assert overview["B12"].number_format == "0.0000E+00" + assert overview["B13"].value == "自定义波长" + phase_header = next( + row + for row in range(1, overview.max_row + 1) + if overview.cell(row, 1).value == "物相名称" + ) + phase_row = phase_header + 1 + assert overview.cell(phase_row, 1).value == phase["phase_name"] + assert overview.row_dimensions[phase_row].height > 30 + assert overview.row_dimensions[phase_row].height <= 409.5 diff --git a/tests/test_export_views.py b/tests/test_export_views.py index 78627c4..3915488 100644 --- a/tests/test_export_views.py +++ b/tests/test_export_views.py @@ -6,6 +6,7 @@ from pathlib import Path from openpyxl import load_workbook +from openpyxl.utils import get_column_letter from diffractscout.demo import write_demo_inputs from diffractscout.export_views import ( @@ -33,8 +34,12 @@ def test_demo_analyze_writes_lab_view_sheets(tmp_path: Path) -> None: assert "推荐峰表" in workbook.sheetnames assert "使用说明" in workbook.sheetnames assert "Peaks" in workbook.sheetnames - # Lab-first workbook opens on the Chinese analysis long table. - assert workbook.active.title == "推荐峰表" + assert workbook.sheetnames[0] == "结果概览" + assert workbook.active.title == "结果概览" + overview = workbook["结果概览"] + assert overview["A1"].value + assert "物相数" in [overview.cell(row, 1).value for row in range(1, overview.max_row + 1)] + assert any(cell.hyperlink for row in overview.iter_rows() for cell in row) recommend = workbook["推荐峰表"] headers = [cell.value for cell in next(recommend.iter_rows(min_row=1, max_row=1))] assert headers[0] == "物相名称" @@ -63,12 +68,14 @@ def test_peak_and_recommend_sheets_have_freeze_and_autofilter(tmp_path: Path) -> workbook = load_workbook(result.output_dir / "results.xlsx", data_only=False) for sheet_name in ("Peaks", "推荐峰表"): sheet = workbook[sheet_name] - assert sheet.freeze_panes == "A2" + assert sheet.freeze_panes == "C2" assert sheet.auto_filter.ref assert sheet.auto_filter.ref.startswith("A1:") headers = [cell.value for cell in next(sheet.iter_rows(min_row=1, max_row=1))] assert headers[0] in {"phase_name", "物相名称"} assert sheet.max_row >= 2 + assert sheet.row_dimensions[1].height >= 36 + assert sheet.page_setup.orientation == "landscape" def test_analysis_peak_columns_present_in_peaks_and_csv(tmp_path: Path) -> None: @@ -78,10 +85,12 @@ def test_analysis_peak_columns_present_in_peaks_and_csv(tmp_path: Path) -> None: assert col in PEAK_HEADERS workbook = load_workbook(result.output_dir / "results.xlsx", data_only=False) peak_headers = [cell.value for cell in next(workbook["Peaks"].iter_rows(min_row=1, max_row=1))] + assert peak_headers == PEAK_HEADERS for col in ANALYSIS_PEAK_COLUMNS: assert col in peak_headers with (result.output_dir / "peak_reference.csv").open(encoding="utf-8-sig", newline="") as handle: fieldnames = list(csv.DictReader(handle).fieldnames or []) + assert fieldnames == PEAK_HEADERS for col in ANALYSIS_PEAK_COLUMNS: assert col in fieldnames # Analysis-first ordering: identity and geometry before deep SF extras. @@ -138,7 +147,9 @@ def test_include_patterns_false_skips_profile_export(tmp_path: Path) -> None: include_excel=True, ) assert not (result.output_dir / "pattern_profiles.csv").exists() - workbook = load_workbook(result.output_dir / "results.xlsx", read_only=True) + workbook = load_workbook(result.output_dir / "results.xlsx", read_only=False) + assert workbook.sheetnames[0] == "结果概览" + assert workbook.active.title == "结果概览" assert "Patterns" not in workbook.sheetnames assert "Peaks" in workbook.sheetnames @@ -152,6 +163,8 @@ def test_export_lab_views_false_omits_chinese_sheets(tmp_path: Path) -> None: include_excel=True, ) workbook = load_workbook(result.output_dir / "results.xlsx", read_only=True) + assert workbook.sheetnames[0] == "Overview" + assert workbook.active.title == "Overview" assert "推荐峰表" not in workbook.sheetnames assert "使用说明" not in workbook.sheetnames assert "Peaks" in workbook.sheetnames @@ -247,5 +260,99 @@ def test_write_excel_workbook_freeze_filter_on_empty_peaks(tmp_path: Path) -> No ) workbook = load_workbook(path) peaks = workbook["Peaks"] - assert peaks.freeze_panes == "A2" + assert peaks.freeze_panes == "C2" assert peaks.auto_filter.ref + + +def test_workbook_styles_preserve_values_and_fold_only_details(tmp_path: Path) -> None: + inputs = write_demo_inputs(tmp_path / "inputs") + result = analyze_cifs( + [inputs], + tmp_path / "bundle", + settings=AnalysisSettings(export_lab_views=True), + include_excel=True, + ) + workbook = load_workbook(result.output_dir / "results.xlsx", data_only=False) + peaks = workbook["Peaks"] + headers = [cell.value for cell in peaks[1]] + normalized_column = headers.index("normalized_intensity") + 1 + spacing_column = headers.index("d_spacing_A") + 1 + sf_column = headers.index("structure_factor_sq") + 1 + hash_column = headers.index("cif_sha256") + 1 + + assert peaks.cell(2, normalized_column).value == result.analyses[0].reflections[0].normalized_intensity + assert peaks.cell(2, normalized_column).number_format == "0.00" + assert peaks.cell(2, spacing_column).number_format == "0.0000" + assert peaks.column_dimensions[get_column_letter(sf_column)].hidden + assert peaks.column_dimensions[get_column_letter(sf_column)].outlineLevel == 1 + assert not peaks.column_dimensions[get_column_letter(normalized_column)].hidden + assert peaks.column_dimensions[get_column_letter(hash_column)].hidden + assert peaks.cell(2, hash_column).value + assert peaks.print_area == f"'Peaks'!$A$1:${get_column_letter(normalized_column)}${peaks.max_row}" + + chinese = workbook["推荐峰表"] + chinese_headers = [cell.value for cell in chinese[1]] + chinese_d_column = chinese_headers.index("d间距_Å") + 1 + chinese_intensity_column = chinese_headers.index("相对强度_相内max100") + 1 + assert chinese.cell(2, chinese_d_column).number_format == "0.0000" + assert chinese.cell(2, chinese_intensity_column).number_format == "0.00" + assert chinese.print_area == f"'推荐峰表'!$A$1:${get_column_letter(chinese_intensity_column)}${chinese.max_row}" + + diagnostics = workbook["Diagnostics"] + diagnostic_headers = [cell.value for cell in diagnostics[1]] + message_column = diagnostic_headers.index("message") + 1 + assert not diagnostics.column_dimensions[get_column_letter(message_column)].hidden + + # Data bars use a fixed 0–100 range rather than cross-sheet autoscaling. + bars = [ + rule.dataBar + for area, rules in peaks.conditional_formatting._cf_rules.items() + for rule in rules + if rule.type == "dataBar" + and str(area.sqref) + == f"{get_column_letter(normalized_column)}2:{get_column_letter(normalized_column)}{peaks.max_row}" + ] + assert bars + assert [(bound.type, bound.val) for bound in bars[0].cfvo] == [("num", 0), ("num", 100)] + + +def test_number_formats_keep_small_nonzero_values_visible() -> None: + from diffractscout.excel_styles import cjk_display_width, number_format_for_value + + assert cjk_display_width("晶胞") == 4 + assert number_format_for_value("d_spacing_A", 0.000001) == "0.000E+00" + assert number_format_for_value("d_spacing_A", 0.0) == "0.0000" + assert number_format_for_value("phase_relative_R_hkl_pct", 0.000001) == '0.000E+00"%"' + assert number_format_for_value("相内相对J_含LP_%", 0.0) == '0.00"%"' + + +def test_diagnostics_message_and_item_wrap_without_exceeding_excel_height_limit( + tmp_path: Path, +) -> None: + path = tmp_path / "diagnostics.xlsx" + write_excel_workbook( + path, + summary=[], + phases=[], + peaks=[], + elasticity=[], + candidates=[], + downloads=[], + diagnostics=[ + { + "stage": "parse", + "item": "phase-name-with-context-" * 8, + "level": "warning", + "message": "This diagnostic message explains a concrete input issue. " * 20, + } + ], + patterns=[], + export_lab_views=False, + include_patterns=False, + ) + diagnostics = load_workbook(path)["Diagnostics"] + assert diagnostics.column_dimensions["B"].width == 32 + assert diagnostics.column_dimensions["D"].width == 80 + assert diagnostics["B2"].alignment.wrap_text + assert diagnostics["D2"].alignment.wrap_text + assert 19 < diagnostics.row_dimensions[2].height <= 409.5 diff --git a/tests/test_gui.py b/tests/test_gui.py index ec25982..984981d 100644 --- a/tests/test_gui.py +++ b/tests/test_gui.py @@ -14,7 +14,7 @@ discovery_settings_from_form, ) from diffractscout.gui_i18n import REQUIRED_KEYS, STRINGS, assert_language_parity, t -from diffractscout.models import AnalysisSettings +from diffractscout.models import AnalysisSettings, DiagnosticRecord, PipelineResult from diffractscout.diffraction import ENERGY_WAVELENGTH_KEV_A, resolve_wavelength from diffractscout.validation import verify_bundle @@ -199,19 +199,38 @@ def stop(self): controller = type("Controller", (), {})() controller.last_output = good controller.running = True - controller.open_button = Widget() + controller.preview_button = Widget() + controller.save_excel_button = Widget() + controller.open_folder_button = Widget() controller.progress = Progress() controller._run_buttons = [] + controller._preset_buttons = [] + controller._finish_task = gui_module.DiffractScoutApp._finish_task.__get__(controller) gui_module.DiffractScoutApp._update_open_button_state(controller) - assert controller.open_button.states[-1]["state"] == "disabled" + assert controller.open_folder_button.states[-1]["state"] == "disabled" + assert controller.preview_button.states[-1]["state"] == "disabled" + assert controller.save_excel_button.states[-1]["state"] == "disabled" controller.running = False gui_module.DiffractScoutApp._finish_task(controller) - assert controller.open_button.states[-1]["state"] == "normal" + assert controller.open_folder_button.states[-1]["state"] == "normal" + assert controller.preview_button.states[-1]["state"] == "disabled" + assert controller.save_excel_button.states[-1]["state"] == "disabled" + + workbook = good / "results.xlsx" + workbook.write_bytes(b"workbook") + gui_module.DiffractScoutApp._update_open_button_state(controller) + assert controller.preview_button.states[-1]["state"] == "normal" + assert controller.save_excel_button.states[-1]["state"] == "normal" (good / "manifest.json").unlink() - gui_module.DiffractScoutApp._finish_task(controller) - assert controller.open_button.states[-1]["state"] == "disabled" + workbook.unlink() + payload.unlink() + good.rmdir() + gui_module.DiffractScoutApp._update_open_button_state(controller) + assert controller.open_folder_button.states[-1]["state"] == "disabled" + assert controller.preview_button.states[-1]["state"] == "disabled" + assert controller.save_excel_button.states[-1]["state"] == "disabled" def test_analysis_form_rejects_unknown_profile_model() -> None: @@ -418,6 +437,119 @@ def test_cif_file_dialog_accepts_mixed_case_suffixes() -> None: assert en_types[0] == t("en", "filetype_cif") +def test_local_input_add_rejects_bad_paths_without_scanning_folders(tmp_path, monkeypatch) -> None: + valid = tmp_path / "valid.cif" + valid.write_text("data_valid\n", encoding="utf-8") + folder = tmp_path / "empty-folder" + folder.mkdir() + non_cif = tmp_path / "notes.txt" + non_cif.write_text("not a CIF\n", encoding="utf-8") + missing = tmp_path / "missing.cif" + + class InputList: + def __init__(self): + self.items = [] + + def delete(self, *_args): + self.items.clear() + + def insert(self, _index, item): + self.items.append(item) + + class Variable: + def __init__(self): + self.value = "" + + def set(self, value): + self.value = value + + controller = type("Controller", (), {})() + controller.local_inputs = [] + controller.input_list = InputList() + controller.input_count_text = Variable() + controller.lang = "en" + controller._t = lambda key, **fmt: t(controller.lang, key, **fmt) + controller._refresh_inputs = gui_module.DiffractScoutApp._refresh_inputs.__get__(controller) + controller._format_input_problems = ( + gui_module.DiffractScoutApp._format_input_problems.__get__(controller) + ) + warnings = [] + errors = [] + monkeypatch.setattr(gui_module.messagebox, "showwarning", lambda *args: warnings.append(args)) + monkeypatch.setattr(gui_module.messagebox, "showerror", lambda *args: errors.append(args)) + + def unexpected_walk(_path, *_args, **_kwargs): + raise AssertionError("the Tk thread must not walk folder contents") + + monkeypatch.setattr(Path, "iterdir", unexpected_walk) + monkeypatch.setattr(Path, "rglob", unexpected_walk) + gui_module.DiffractScoutApp._add_input_paths( + controller, [valid, folder, non_cif, missing] + ) + + assert controller.local_inputs == [valid.resolve(), folder.resolve()] + assert len(warnings) == 1 + assert "not a CIF file" in warnings[0][1] + assert str(missing) in warnings[0][1] + assert errors == [] + assert len(controller.input_list.items) == 2 + + +def test_local_run_blocks_stale_input_and_does_not_walk_folder_on_tk_thread( + tmp_path, monkeypatch +) -> None: + valid = tmp_path / "valid.cif" + valid.write_text("data_valid\n", encoding="utf-8") + folder = tmp_path / "empty-folder" + folder.mkdir() + missing = tmp_path / "missing.cif" + + class Variable: + def __init__(self, value): + self.value = value + + def get(self): + return self.value + + def set(self, value): + self.value = value + + controller = type("Controller", (), {})() + controller.running = False + controller.local_inputs = [valid, missing] + controller.local_output = Variable(str(tmp_path / "bundle")) + controller.elastic_overrides = {} + controller.local_recursive = Variable(True) + controller.include_excel = Variable(False) + controller.overwrite = Variable(False) + controller.lang = "en" + controller._t = lambda key, **fmt: t(controller.lang, key, **fmt) + controller._form_analysis_settings = lambda: AnalysisSettings() + scheduled = [] + errors = [] + warnings = [] + controller._start_task = lambda label, function: scheduled.append((label, function)) + controller._format_input_problems = ( + gui_module.DiffractScoutApp._format_input_problems.__get__(controller) + ) + monkeypatch.setattr(gui_module.messagebox, "showerror", lambda *args: errors.append(args)) + monkeypatch.setattr(gui_module.messagebox, "showwarning", lambda *args: warnings.append(args)) + gui_module.DiffractScoutApp._run_local(controller) + assert scheduled == [] + assert errors and "does not exist" in errors[0][1] + + controller.local_inputs = [valid, folder] + + def unexpected_walk(_path, *_args, **_kwargs): + raise AssertionError("the Tk thread must not walk folder contents") + + monkeypatch.setattr(Path, "iterdir", unexpected_walk) + monkeypatch.setattr(Path, "rglob", unexpected_walk) + gui_module.DiffractScoutApp._run_local(controller) + assert len(scheduled) == 1 + assert warnings == [] + + def test_status_text_rerenders_in_the_selected_language() -> None: """Language switches must retranslate busy/completed/failed status, not only Ready.""" @@ -468,6 +600,64 @@ def set(self, value: str) -> None: assert controller.status_text.get() == t("zh", "status_failed") +def test_completion_dialog_reports_phase_peak_warning_and_error_counts( + tmp_path, monkeypatch +) -> None: + class Widget: + def configure(self, **_kwargs): + return None + + class Progress: + def stop(self): + return None + + result = PipelineResult( + output_dir=tmp_path / "bundle", + discovery=None, + downloads=[], + analyses=[ + type("Analysis", (), {"reflections": [1, 2]})(), + type("Analysis", (), {"reflections": [1]})(), + ], + manifest_path=tmp_path / "bundle" / "manifest.json", + diagnostics=[ + DiagnosticRecord("analysis", "phase-a", "warning", "warning text"), + DiagnosticRecord("analysis", "phase-b", "error", "error text"), + ], + ) + controller = type("Controller", (), {})() + controller.running = True + controller.events = gui_module.queue.Queue() + controller.events.put(("done", result)) + controller.progress = Progress() + controller._run_buttons = [] + controller._preset_buttons = [] + controller._finish_task = gui_module.DiffractScoutApp._finish_task.__get__(controller) + controller._poll = gui_module.DiffractScoutApp._poll.__get__(controller) + controller.preview_button = Widget() + controller.save_excel_button = Widget() + controller.open_folder_button = Widget() + controller.lang = "en" + controller._t = lambda key, **fmt: t(controller.lang, key, **fmt) + controller._set_status = lambda *_args, **_kwargs: None + controller._log = lambda *_args, **_kwargs: None + controller.after = lambda *_args: "poll-id" + dialogs = [] + monkeypatch.setattr( + gui_module.messagebox, + "showwarning", + lambda title, message: dialogs.append((title, message)), + ) + + gui_module.DiffractScoutApp._poll(controller) + + assert len(dialogs) == 1 + assert "Phases: 2" in dialogs[0][1] + assert "indexed peaks: 3" in dialogs[0][1] + assert "warnings: 1" in dialogs[0][1] + assert "errors: 1" in dialogs[0][1] + + def test_create_app_language_switch_rerenders_completion_status() -> None: app = _create_test_app() try: @@ -492,6 +682,87 @@ def test_create_app_language_switch_rerenders_completion_status() -> None: app.destroy() +@pytest.mark.parametrize( + ("energy", "shortcut"), + [(30.0, "30 keV"), (83.0, "83 keV")], +) +def test_analysis_preset_roundtrip_preserves_shortcuts_and_lab_preference( + tmp_path, monkeypatch, energy, shortcut +) -> None: + app = _create_test_app() + try: + preset = tmp_path / "analysis.json" + app.input_mode.set("energy") + app.radiation_value.set(str(int(energy))) + app.energy_shortcut.set(shortcut) + app.two_theta_min.set("7") + app.local_recursive.set(False) + app.include_excel.set(False) + app.update_idletasks() + assert app._lab_views_preference is True + + monkeypatch.setattr( + gui_module.filedialog, + "asksaveasfilename", + lambda **_kwargs: str(preset), + ) + app._save_analysis_preset() + serialized = json.loads(preset.read_text(encoding="utf-8")) + assert "api_key" not in serialized["values"] + assert "mp_key" not in serialized["values"] + assert "local_output" not in serialized["values"] + assert serialized["values"]["export_lab_views"] is True + + app.input_mode.set("source") + app.radiation_value.set("1.5406") + app.two_theta_min.set("5") + app.local_recursive.set(True) + monkeypatch.setattr( + gui_module.filedialog, + "askopenfilename", + lambda **_kwargs: str(preset), + ) + app._load_analysis_preset() + app.update_idletasks() + + assert app.input_mode.get() == "energy" + assert float(app.radiation_value.get()) == energy + assert app.energy_shortcut.get() == shortcut + assert app.two_theta_min.get() == "7.0" + assert app.local_recursive.get() is False + assert app.include_excel.get() is False + assert app.export_lab_views.get() is False + assert app._lab_views_preference is True + app.include_excel.set(True) + app.update_idletasks() + assert app.export_lab_views.get() is True + finally: + app.destroy() + + +def test_invalid_preset_is_rejected_before_changing_form(tmp_path, monkeypatch) -> None: + app = _create_test_app() + try: + app.input_mode.set("energy") + app.radiation_value.set("40") + before = (app.input_mode.get(), app.radiation_value.get(), app.two_theta_min.get()) + errors = [] + monkeypatch.setattr( + gui_module.filedialog, + "askopenfilename", + lambda **_kwargs: str(tmp_path / "invalid.json"), + ) + monkeypatch.setattr( + gui_module.messagebox, "showerror", lambda *args: errors.append(args) + ) + app._load_analysis_preset() + after = (app.input_mode.get(), app.radiation_value.get(), app.two_theta_min.get()) + assert after == before + assert errors + finally: + app.destroy() + + def test_analysis_form_rejects_non_numeric_value() -> None: with pytest.raises(ValueError, match="Profile step"): analysis_settings_from_form( @@ -573,33 +844,148 @@ def test_preview_workbook_is_copied_outside_bundle(tmp_path, monkeypatch) -> Non assert preview.parent != workbook.parent assert preview.read_bytes() == workbook.read_bytes() assert not list(bundle.glob("~$*")) - gui_module.DiffractScoutApp._cleanup_preview_dirs(controller) + assert gui_module.DiffractScoutApp._cleanup_preview_dirs(controller) == [] assert not preview.parent.exists() +def test_preview_cleanup_preserves_saved_edits_and_extra_files(tmp_path) -> None: + source = tmp_path / "results.xlsx" + source.write_bytes(b"original workbook") + controller = type("Controller", (), {"_preview_dirs": []})() + preview = gui_module.DiffractScoutApp._preview_workbook(controller, source) + preview.write_bytes(b"edited workbook") + + assert gui_module.DiffractScoutApp._cleanup_preview_dirs(controller) == [preview] + assert preview.read_bytes() == b"edited workbook" + assert preview.parent.is_dir() + + unchanged = gui_module.DiffractScoutApp._preview_workbook(controller, source) + extra_file = unchanged.parent / "saved-copy.xlsx" + extra_file.write_bytes(b"user saved copy") + assert gui_module.DiffractScoutApp._cleanup_preview_dirs(controller) == [ + preview, + unchanged.parent, + ] + assert unchanged.read_bytes() == b"original workbook" + assert extra_file.read_bytes() == b"user saved copy" + + def test_preview_cleanup_retains_busy_directory_for_retry(tmp_path, monkeypatch) -> None: preview_dir = tmp_path / "preview" preview_dir.mkdir() - controller = type("Controller", (), {"_preview_dirs": [preview_dir]})() - real_rmtree = gui_module.shutil.rmtree + preview = preview_dir / "results.xlsx" + preview.write_bytes(b"unchanged workbook") + from diffractscout.utils import sha256_file + + original_hash = sha256_file(preview) + controller = type( + "Controller", (), {"_preview_dirs": [(preview_dir, preview, original_hash)]} + )() + real_unlink = Path.unlink calls: list[Path] = [] def fail_once(path, *args, **kwargs): calls.append(Path(path)) if len(calls) == 1: raise OSError("preview is still open") - return real_rmtree(path, *args, **kwargs) + return real_unlink(path, *args, **kwargs) - monkeypatch.setattr(gui_module.shutil, "rmtree", fail_once) - gui_module.DiffractScoutApp._cleanup_preview_dirs(controller) - assert controller._preview_dirs == [preview_dir] + monkeypatch.setattr(Path, "unlink", fail_once) + assert gui_module.DiffractScoutApp._cleanup_preview_dirs(controller) == [preview] + assert controller._preview_dirs == [(preview_dir, preview, original_hash)] assert preview_dir.is_dir() - gui_module.DiffractScoutApp._cleanup_preview_dirs(controller) + assert gui_module.DiffractScoutApp._cleanup_preview_dirs(controller) == [] assert controller._preview_dirs == [] assert not preview_dir.exists() +def test_preview_and_folder_actions_open_distinct_targets(tmp_path, monkeypatch) -> None: + bundle = tmp_path / "bundle" + bundle.mkdir() + workbook = bundle / "results.xlsx" + workbook.write_bytes(b"workbook bytes") + controller = type("Controller", (), {})() + controller.running = False + controller.last_output = bundle + controller._preview_dirs = [] + controller._t = lambda key, **fmt: t("en", key, **fmt) + controller._log = lambda *_args: None + controller._update_open_button_state = lambda: None + controller._preview_workbook = gui_module.DiffractScoutApp._preview_workbook.__get__(controller) + opened = [] + monkeypatch.setattr(gui_module, "open_path", lambda path: opened.append(Path(path))) + + gui_module.DiffractScoutApp._preview_last_workbook(controller) + assert opened[0] != workbook + assert opened[0].parent != bundle + assert opened[0].read_bytes() == workbook.read_bytes() + + gui_module.DiffractScoutApp._open_last_output_folder(controller) + assert opened[1] == bundle + gui_module.DiffractScoutApp._cleanup_preview_dirs(controller) + + +def test_save_excel_copy_uses_explicit_overwrite_and_rejects_bundle_targets( + tmp_path, monkeypatch +) -> None: + bundle = tmp_path / "bundle" + bundle.mkdir() + workbook = bundle / "results.xlsx" + workbook.write_bytes(b"source workbook") + destination = tmp_path / "saved.xlsx" + selected = [str(destination)] + dialog_options = {} + monkeypatch.setattr( + gui_module.filedialog, + "asksaveasfilename", + lambda **kwargs: (dialog_options.update(kwargs), selected[-1])[1], + ) + confirmations = [] + answers = [False, True] + + def confirm(*args): + confirmations.append(args) + return answers.pop(0) + + monkeypatch.setattr(gui_module.messagebox, "askyesno", confirm) + errors = [] + monkeypatch.setattr(gui_module.messagebox, "showerror", lambda *args: errors.append(args)) + controller = type("Controller", (), {})() + controller.running = False + controller.last_output = bundle + controller.lang = "en" + controller._t = lambda key, **fmt: t("en", key, **fmt) + controller._set_status = lambda *_args, **_kwargs: None + controller._log = lambda *_args: None + controller._update_open_button_state = lambda: None + + gui_module.DiffractScoutApp._save_excel_copy(controller) + assert destination.read_bytes() == workbook.read_bytes() + assert dialog_options["confirmoverwrite"] is False + assert dialog_options["defaultextension"] == ".xlsx" + assert confirmations == [] + + destination.write_bytes(b"prior file") + gui_module.DiffractScoutApp._save_excel_copy(controller) + assert destination.read_bytes() == b"prior file" + assert len(confirmations) == 1 + + gui_module.DiffractScoutApp._save_excel_copy(controller) + assert destination.read_bytes() == workbook.read_bytes() + assert len(confirmations) == 2 + + selected.append(str(bundle / "inside.xlsx")) + gui_module.DiffractScoutApp._save_excel_copy(controller) + assert errors[-1][1] == t("en", "err_excel_copy_in_bundle") + assert not (bundle / "inside.xlsx").exists() + + selected.append(str(tmp_path / "wrong.csv")) + gui_module.DiffractScoutApp._save_excel_copy(controller) + assert errors[-1][1] == t("en", "err_excel_copy_extension") + assert not (tmp_path / "wrong.csv").exists() + + def test_i18n_required_keys_zh_en_parity() -> None: assert_language_parity() for key in REQUIRED_KEYS: @@ -640,6 +1026,19 @@ def test_validation_message_localizes_known_range_in_zh_and_en() -> None: assert "minimum" in en and "maximum" in en +def test_local_input_pipeline_errors_are_localized() -> None: + for lang in ("zh", "en"): + controller = _validation_controller(lang) + no_cif = gui_module.DiffractScoutApp._validation_message( + controller, FileNotFoundError("No CIF files were found in the supplied inputs.") + ) + assert no_cif == t(lang, "err_no_cif_found") + disappeared = gui_module.DiffractScoutApp._validation_message( + controller, FileNotFoundError("Input path does not exist: C:/data/a.cif") + ) + assert "C:/data/a.cif" in disappeared + + def test_validation_message_keeps_unknown_detail_with_localized_fallback() -> None: detail = "new backend invariant 42" for lang in ("zh", "en"): @@ -839,8 +1238,15 @@ def test_action_button_geometry_has_no_clipped_glyph_area(geometry: str, lang: s app._set_language(lang) app.update_idletasks() app.update() - buttons = [app.open_button, app.btn_apply_cubic] + buttons = [ + app.preview_button, + app.save_excel_button, + app.open_folder_button, + app.btn_apply_cubic, + ] buttons.extend(button for button in app._run_buttons if button.winfo_ismapped()) + buttons.extend(button for button in app._preset_buttons if button.winfo_ismapped()) + assert app.status_label.winfo_width() > 0 for button in buttons: assert button.winfo_width() >= button.winfo_reqwidth(), str(button) assert button.winfo_height() >= button.winfo_reqheight(), str(button) @@ -927,7 +1333,9 @@ def fake_analyze(inputs, output, **kwargs): monkeypatch.setattr(gui_module, "analyze_cifs", fake_analyze) controller.running = False - controller.local_inputs = [tmp_path / "input.cif"] + input_cif = tmp_path / "input.cif" + input_cif.write_text("data_input\n", encoding="utf-8") + controller.local_inputs = [input_cif] controller.local_output = Variable(str(tmp_path / "bundle")) controller.elastic_overrides = {} controller.local_recursive = Variable(True) diff --git a/tests/test_gui_settings.py b/tests/test_gui_settings.py new file mode 100644 index 0000000..cf6ef91 --- /dev/null +++ b/tests/test_gui_settings.py @@ -0,0 +1,170 @@ +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from diffractscout.gui_settings import load_analysis_preset, save_analysis_preset + + +def _form_values() -> dict[str, object]: + return { + "input_mode": "source", + "source_preset": "Cu Ka", + "radiation_value": "stale disabled text", + "two_theta_min": "5", + "two_theta_max": "120", + "step": "0.02", + "fwhm": "0.15", + "eta": "0.5", + "d_min_A": "0.8", + "d_max_A": "4.0", + "profile_model": "pseudo_voigt", + "pattern_axis": "q", + "max_profile_points": "1000000", + "max_reflection_estimate": "2000000", + "include_excel": True, + "include_elasticity": False, + "export_lab_views": True, + "include_patterns": False, + "include_figures": True, + "local_recursive": False, + } + + +def _read_document(path: Path) -> dict[str, object]: + return json.loads(path.read_text(encoding="utf-8")) + + +def test_analysis_preset_round_trip_is_versioned_and_gui_ready(tmp_path: Path) -> None: + target = tmp_path / "presets" / "local-analysis.json" + + saved_path = save_analysis_preset(target, _form_values()) + document = _read_document(target) + loaded = load_analysis_preset(target) + + assert saved_path == target + assert document["schema"] == "diffractscout_analysis_preset" + assert document["version"] == 1 + saved_values = document["values"] + assert isinstance(saved_values, dict) + assert saved_values["radiation_value"] is None # Inactive built-in source field is discarded. + assert saved_values["d_min_A"] == pytest.approx(0.8) + assert saved_values["include_elasticity"] is False + assert loaded["radiation_value"] == "" + assert loaded["d_min_A"] == pytest.approx(0.8) + assert loaded["d_max_A"] == pytest.approx(4.0) + assert loaded["max_profile_points"] == 1_000_000 + assert loaded["max_reflection_estimate"] == 2_000_000 + assert loaded["pattern_axis"] == "q" + assert loaded["include_excel"] is True + assert loaded["export_lab_views"] is True + assert loaded["local_recursive"] is False + + +def test_analysis_preset_accepts_custom_radiation_and_unset_d_bounds(tmp_path: Path) -> None: + values = _form_values() + values.update( + input_mode="wavelength", + radiation_value="1.78897", + d_min_A="", + d_max_A="", + ) + target = save_analysis_preset(tmp_path / "wavelength.json", values) + + loaded = load_analysis_preset(target) + + assert loaded["radiation_value"] == pytest.approx(1.78897) + assert loaded["d_min_A"] == "" + assert loaded["d_max_A"] == "" + + +def test_save_rejects_unsupported_sensitive_fields_without_replacing_file( + tmp_path: Path, +) -> None: + target = tmp_path / "existing.json" + target.write_text("keep this file", encoding="utf-8") + values = _form_values() + values["mp_api_key"] = "secret" + + with pytest.raises(ValueError, match="unknown or unsupported field.*mp_api_key"): + save_analysis_preset(target, values) + + assert target.read_text(encoding="utf-8") == "keep this file" + assert list(tmp_path.glob(".diffractscout-preset-*.tmp")) == [] + + +@pytest.mark.parametrize( + ("field", "value", "message"), + [ + ("two_theta_min", 120, "2theta range"), + ("eta", float("nan"), "finite"), + ("d_min_A", -1.0, "finite positive"), + ("max_profile_points", True, "positive integer"), + ("include_excel", "false", "must be a boolean"), + ("profile_model", "unknown", "Unknown profile_model"), + ], +) +def test_load_rejects_invalid_types_and_scientific_settings( + tmp_path: Path, field: str, value: object, message: str +) -> None: + target = save_analysis_preset(tmp_path / "valid.json", _form_values()) + document = _read_document(target) + values = document["values"] + assert isinstance(values, dict) + values[field] = value + target.write_text(json.dumps(document, allow_nan=True), encoding="utf-8") + + with pytest.raises(ValueError, match=message): + load_analysis_preset(target) + + +@pytest.mark.parametrize( + ("changes", "message"), + [ + ({"schema": "other"}, "unsupported schema"), + ({"version": 2}, "unsupported version"), + ({"extra": "value"}, "unknown extra"), + ], +) +def test_load_rejects_unknown_schema_version_and_document_fields( + tmp_path: Path, changes: dict[str, object], message: str +) -> None: + target = save_analysis_preset(tmp_path / "valid.json", _form_values()) + document = _read_document(target) + document.update(changes) + target.write_text(json.dumps(document), encoding="utf-8") + + with pytest.raises(ValueError, match=message): + load_analysis_preset(target) + + +def test_load_rejects_unknown_preset_fields_and_duplicate_json_keys(tmp_path: Path) -> None: + target = save_analysis_preset(tmp_path / "valid.json", _form_values()) + document = _read_document(target) + values = document["values"] + assert isinstance(values, dict) + values["output_path"] = "C:/private/output" + target.write_text(json.dumps(document), encoding="utf-8") + with pytest.raises(ValueError, match="unknown or unsupported field.*output_path"): + load_analysis_preset(target) + + target.write_text( + '{"schema":"diffractscout_analysis_preset",' + '"schema":"diffractscout_analysis_preset", "version":1,"values":{}}', + encoding="utf-8", + ) + with pytest.raises(ValueError, match="duplicate JSON key 'schema'"): + load_analysis_preset(target) + + +def test_load_rejects_oversized_preset_and_invalid_json(tmp_path: Path) -> None: + target = tmp_path / "bad.json" + target.write_text("{" + " " * (64 * 1024), encoding="utf-8") + with pytest.raises(ValueError, match="size limit"): + load_analysis_preset(target) + + target.write_text("not JSON", encoding="utf-8") + with pytest.raises(ValueError, match="could not parse UTF-8 JSON"): + load_analysis_preset(target) diff --git a/tests/test_inspection.py b/tests/test_inspection.py new file mode 100644 index 0000000..5f56d0f --- /dev/null +++ b/tests/test_inspection.py @@ -0,0 +1,339 @@ +from __future__ import annotations + +import csv +import json +from pathlib import Path +import pickle + +import diffractscout.inspection as inspection_module +from diffractscout.inspection import format_inspection, inspect_bundle +from diffractscout.models import AnalysisSettings, CandidateRecord, DiscoverySettings +from diffractscout.pipeline import analyze_cifs, export_discovery +from diffractscout.utils import sha256_file + + +def _make_bundle( + output: Path, + inputs: Path, + *, + settings: AnalysisSettings | None = None, + include_excel: bool = True, +) -> Path: + analyze_cifs( + [inputs], + output, + settings=settings, + include_excel=include_excel, + ) + return output + + +def _rewrite_manifest_member(bundle: Path, relative: str) -> None: + path = bundle / relative + manifest_path = bundle / "manifest.json" + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + manifest["files"] = [ + entry for entry in manifest["files"] if entry["path"] != relative + ] + manifest["files"].append( + { + "path": relative, + "role": "documentation", + "sha256": sha256_file(path), + "size_bytes": path.stat().st_size, + } + ) + manifest_path.write_text( + json.dumps(manifest, indent=2, ensure_ascii=False) + "\n", + encoding="utf-8", + ) + + +def _write_sentinel(path: str) -> None: + Path(path).write_text("unpickled", encoding="utf-8") + + +class _PickleCanary: + def __init__(self, path: Path) -> None: + self.path = str(path) + + def __reduce__(self) -> tuple[object, tuple[str]]: + return (_write_sentinel, (self.path,)) + + +class _DiscoveryProvider: + name = "offline-inspection-fixture" + + def search_subsystem(self, chemsys: str, **_kwargs: object) -> list[CandidateRecord]: + return [ + CandidateRecord( + material_id="fixture-1", + formula="Al", + queried_chemsys=chemsys, + source_provider=self.name, + ) + ] + + def metadata(self) -> dict[str, object]: + return {"provider": self.name, "offline": True} + + +def test_inspect_verified_bundle_reports_settings_counts_and_files( + demo_inputs: Path, tmp_path: Path +) -> None: + bundle = _make_bundle(tmp_path / "result", demo_inputs) + + report = inspect_bundle(bundle) + + assert report["ok"] is True + assert report["verification"]["ok"] is True + assert report["software_version"] + assert report["analysis_settings"]["input_mode"] == "source" + assert report["counts"]["phases"] == 1 + assert report["counts"]["reflections"] > 0 + assert report["counts"]["diagnostics"] == 0 + assert report["files"]["outputs"]["results_xlsx"] == str(bundle / "results.xlsx") + assert all(item["verified"] for item in report["files"]["members"]) + assert "Verification: passed" in format_inspection(report) + + +def test_inspect_multiphase_bundle_counts_warnings( + tmp_path: Path, +) -> None: + benchmark_inputs = Path(__file__).parents[1] / "src" / "diffractscout" / "benchmark_data" + bundle = _make_bundle( + tmp_path / "multiphase", + benchmark_inputs, + settings=AnalysisSettings(step_deg=0.1, include_elasticity=False), + include_excel=False, + ) + + report = inspect_bundle(bundle) + + assert report["ok"] is True + assert report["counts"]["phases"] == 4 + assert report["counts"]["reflections"] == 43 + assert report["counts"]["diagnostics_by_level"]["warning"] == 4 + assert report["files"]["outputs"]["results_xlsx"] is None + assert len(report["diagnostics_preview"]) == 4 + + +def test_partial_error_bundle_is_still_inspectable( + demo_inputs: Path, tmp_path: Path +) -> None: + (demo_inputs / "invalid.cif").write_text("not a CIF\n", encoding="utf-8") + bundle = _make_bundle(tmp_path / "partial", demo_inputs, include_excel=False) + + report = inspect_bundle(bundle) + + assert report["ok"] is True + assert report["counts"]["phases"] == 1 + assert report["counts"]["diagnostics_by_level"]["error"] == 1 + assert report["diagnostics_preview"][0]["level"] == "error" + assert "[error]" in format_inspection(report) + + +def test_inspect_discovery_bundle_with_zero_phases_and_missing_settings( + tmp_path: Path, +) -> None: + bundle = tmp_path / "discovery" + export_discovery( + "Al", + _DiscoveryProvider(), # type: ignore[arg-type] + bundle, + discovery_settings=DiscoverySettings(mode="single_chemsys", max_total=1), + include_excel=False, + ) + provenance_path = bundle / "provenance.json" + provenance = json.loads(provenance_path.read_text(encoding="utf-8")) + provenance["analysis_settings"] = None + provenance["discovery"]["opaque_provider_payload"] = "x" * (5 * 1024 * 1024) + provenance_path.write_text(json.dumps(provenance), encoding="utf-8") + _rewrite_manifest_member(bundle, "provenance.json") + + report = inspect_bundle(bundle) + + assert report["ok"] is True + assert report["counts"]["phases"] == 0 + assert report["counts"]["reflections"] == 0 + assert report["counts"]["candidates"] == 1 + assert report["counts"]["downloads"] == 0 + assert report["analysis_settings"] is None + assert "Analysis settings: not recorded" in format_inspection(report) + + +def test_provenance_over_limit_returns_readable_inspection_error( + demo_inputs: Path, + tmp_path: Path, + monkeypatch, +) -> None: + bundle = _make_bundle(tmp_path / "limited-provenance", demo_inputs, include_excel=False) + monkeypatch.setattr(inspection_module, "_MAX_PROVENANCE_JSON_BYTES", 1) + + report = inspect_bundle(bundle) + + assert report["verification"]["ok"] is True + assert report["ok"] is False + assert "exceeds the 1-byte inspection limit" in report["inspection"]["errors"][0] + assert "counts" not in report + + +def test_oversized_manifest_is_rejected_before_bundle_verification( + tmp_path: Path, + monkeypatch, +) -> None: + bundle = tmp_path / "oversized-manifest" + bundle.mkdir() + (bundle / "manifest.json").write_text("{} ", encoding="utf-8") + monkeypatch.setattr(inspection_module, "_MAX_METADATA_JSON_BYTES", 4) + + def unexpected_verification(_bundle: Path) -> dict[str, object]: + raise AssertionError("oversized manifest must be rejected before verification") + + monkeypatch.setattr(inspection_module, "verify_bundle", unexpected_verification) + + report = inspect_bundle(bundle) + + assert report["ok"] is False + assert report["verification"]["ok"] is False + assert report["verification"]["files_checked"] == 0 + assert "exceeds the 4-byte inspection limit" in report["verification"]["errors"][0] + + +def test_inspect_counts_all_diagnostics_but_limits_preview( + demo_inputs: Path, tmp_path: Path +) -> None: + bundle = _make_bundle(tmp_path / "many-diagnostics", demo_inputs, include_excel=False) + diagnostic_path = bundle / "diagnostics.csv" + with diagnostic_path.open("a", encoding="utf-8", newline="") as handle: + writer = csv.writer(handle) + for index in range(40): + writer.writerow(["test", str(index), "warning", "x" * 900]) + manifest_path = bundle / "manifest.json" + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + manifest["summary"]["diagnostic_count"] = 40 + manifest_path.write_text( + json.dumps(manifest, indent=2, ensure_ascii=False) + "\n", + encoding="utf-8", + ) + _rewrite_manifest_member(bundle, "diagnostics.csv") + + report = inspect_bundle(bundle) + + assert report["ok"] is True + assert report["counts"]["diagnostics"] == 40 + assert report["counts"]["diagnostics_by_level"]["warning"] == 40 + assert len(report["diagnostics_preview"]) == 30 + assert len(report["diagnostics_preview"][0]["message"]) == 500 + assert report["diagnostics_preview_truncated"] is True + + +def test_tampered_bundle_returns_verification_errors_without_conclusions( + demo_inputs: Path, tmp_path: Path +) -> None: + bundle = _make_bundle(tmp_path / "tampered", demo_inputs, include_excel=False) + with (bundle / "phase_summary.csv").open("a", encoding="utf-8") as handle: + handle.write("tampered\n") + + report = inspect_bundle(bundle) + + assert report["ok"] is False + assert report["verification"]["ok"] is False + assert any("hash mismatch" in error for error in report["verification"]["errors"]) + assert "counts" not in report + assert "analysis_settings" not in report + assert "Verification: failed" in format_inspection(report) + + +def test_missing_bundle_member_is_reported_without_conclusions( + demo_inputs: Path, tmp_path: Path +) -> None: + bundle = _make_bundle(tmp_path / "missing", demo_inputs, include_excel=False) + (bundle / "peak_reference.csv").unlink() + + report = inspect_bundle(bundle) + + assert report["ok"] is False + assert report["verification"]["ok"] is False + assert any("missing file: peak_reference.csv" in error for error in report["verification"]["errors"]) + assert "counts" not in report + + +def test_non_bundle_directory_reports_missing_manifest(tmp_path: Path) -> None: + directory = tmp_path / "not-a-bundle" + directory.mkdir() + + report = inspect_bundle(directory) + + assert report["ok"] is False + assert report["verification"]["ok"] is False + assert report["verification"]["errors"] == ["manifest.json not found"] + assert "software_version" not in report + + +def test_manifest_paths_are_rejected_before_inspection( + demo_inputs: Path, tmp_path: Path +) -> None: + bundle = _make_bundle(tmp_path / "unsafe-path", demo_inputs, include_excel=False) + manifest_path = bundle / "manifest.json" + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + manifest["files"].append( + { + "path": "../outside.json", + "role": "provenance", + "sha256": "0" * 64, + "size_bytes": 0, + } + ) + manifest_path.write_text(json.dumps(manifest), encoding="utf-8") + + report = inspect_bundle(bundle) + + assert report["ok"] is False + assert report["verification"]["ok"] is False + assert any("unsafe manifest path" in error for error in report["verification"]["errors"]) + assert "files" not in report + + +def test_inconsistent_manifest_summary_withholds_result_conclusions( + demo_inputs: Path, tmp_path: Path +) -> None: + bundle = _make_bundle(tmp_path / "inconsistent-summary", demo_inputs, include_excel=False) + manifest_path = bundle / "manifest.json" + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + manifest["summary"]["phase_count"] += 1 + manifest_path.write_text(json.dumps(manifest), encoding="utf-8") + + report = inspect_bundle(bundle) + + assert report["verification"]["ok"] is True + assert report["ok"] is False + assert report["inspection"]["ok"] is False + assert any("phase_count" in error for error in report["inspection"]["errors"]) + assert "counts" not in report + assert "analysis_settings" not in report + + +def test_inspection_never_unpickles_bundle_members( + demo_inputs: Path, tmp_path: Path +) -> None: + bundle = _make_bundle(tmp_path / "pickle-member", demo_inputs, include_excel=False) + sentinel = tmp_path / "pickle-executed.txt" + payload = bundle / "opaque.pkl" + payload.write_bytes(pickle.dumps(_PickleCanary(sentinel))) + _rewrite_manifest_member(bundle, "opaque.pkl") + + report = inspect_bundle(bundle) + + assert report["ok"] is True + assert any(item["path"] == "opaque.pkl" for item in report["files"]["members"]) + assert not sentinel.exists() + + +def test_inspect_accepts_manifest_path_argument(demo_inputs: Path, tmp_path: Path) -> None: + bundle = _make_bundle(tmp_path / "manifest-argument", demo_inputs, include_excel=False) + + report = inspect_bundle(bundle / "manifest.json") + + assert report["ok"] is True + assert report["bundle"] == str(bundle) diff --git a/tests/test_joss_readiness.py b/tests/test_joss_readiness.py index e30f9a6..09140e5 100644 --- a/tests/test_joss_readiness.py +++ b/tests/test_joss_readiness.py @@ -52,6 +52,48 @@ def _valid_payload() -> dict[str, object]: } +def _git(repo: Path, *args: str) -> str: + completed = subprocess.run( + ["git", *args], + cwd=repo, + check=True, + text=True, + capture_output=True, + ) + return completed.stdout.strip() + + +def _init_paper_source_repo(repo: Path) -> tuple[str, str]: + repo.mkdir() + _git(repo, "init", "--quiet") + _git(repo, "config", "user.name", "Readiness Test") + _git(repo, "config", "user.email", "readiness-test@example.invalid") + for relative_path in READINESS.PAPER_SOURCE_PATHS: + path = repo / relative_path + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(f"fixture content for {relative_path}\n".encode("utf-8")) + (repo / "notes.txt").write_text("base\n", encoding="utf-8") + _git(repo, "add", "--all") + _git(repo, "commit", "--quiet", "-m", "paper sources") + return _git(repo, "rev-parse", "HEAD"), _git(repo, "branch", "--show-current") + + +def _confirmed_submission(remote_ci_commit: str, official_build_commit: str) -> dict[str, object]: + values: dict[str, object] = {name: True for name in READINESS.SUBMISSION_CONFIRMATIONS} + values.update( + { + "remote_ci_url": "https://github.com/D-sudoasd/DiffractScout/actions/runs/1", + "remote_ci_commit": remote_ci_commit, + "official_joss_build_url": "https://github.com/openjournals/joss-papers/actions/runs/1", + "official_joss_build_commit": official_build_commit, + "paper_source_sha256": "", + "paper_pdf_sha256": "", + "note": "confirmed for test", + } + ) + return values + + def test_add_calendar_months_handles_month_end_and_leap_year() -> None: assert READINESS.add_calendar_months(date(2026, 8, 12), 6) == date(2027, 2, 12) assert READINESS.add_calendar_months(date(2023, 8, 31), 6) == date(2024, 2, 29) @@ -61,6 +103,226 @@ def test_evidence_payload_validation_accepts_empty_honest_ledger() -> None: assert READINESS.validate_evidence_payload(_valid_payload()) == [] +def test_evidence_loader_keeps_the_tracked_ledger_as_its_default() -> None: + tracked = json.loads(READINESS.EVIDENCE_PATH.read_text(encoding="utf-8")) + + assert READINESS._load_evidence() == tracked + + +def test_readiness_cli_reads_and_reports_a_separate_evidence_file(tmp_path: Path) -> None: + payload = _valid_payload() + payload["public_repository"]["public_since"] = "2026-09-01" + payload["public_development_activity"] = [ + { + "date": "2026-09-20", + "title": "External evidence ledger smoke test", + "url": "https://github.com/D-sudoasd/DiffractScout/issues/999999", + "software_version": "0.4.0", + "claim_supported": "sidecar ledger selection", + "activity_type": "issue", + "outcome": "Demonstrates external evidence file selection.", + } + ] + evidence_path = tmp_path / "external-evidence.json" + evidence_path.write_text(json.dumps(payload), encoding="utf-8") + + completed = subprocess.run( + [ + sys.executable, + str(SCRIPT), + "--stage", + "release", + "--evidence-file", + str(evidence_path), + "--json", + ], + cwd=ROOT, + check=False, + text=True, + capture_output=True, + ) + + assert completed.returncode == 0, completed.stderr + report = json.loads(completed.stdout) + assert report["evidence_file"] == str(evidence_path.resolve()) + assert report["evidence_load_error"] is None + assert report["public_since"] == "2026-09-01" + assert report["evidence_counts"]["public_development_activity"] == 1 + integrity = next( + item for item in report["checks"] if item["name"] == "Evidence-ledger integrity" + ) + assert integrity["status"] == "pass", integrity + + +def test_missing_or_non_utf8_evidence_file_is_blocked_without_default_fallback( + tmp_path: Path, +) -> None: + missing_path = tmp_path / "missing.json" + missing_payload = READINESS._load_evidence(missing_path) + assert missing_payload["schema"] == "invalid" + assert missing_payload["error"] + + completed = subprocess.run( + [ + sys.executable, + str(SCRIPT), + "--stage", + "release", + "--evidence-file", + str(missing_path), + "--json", + ], + cwd=ROOT, + check=False, + text=True, + capture_output=True, + ) + assert completed.returncode == 0, completed.stderr + report = json.loads(completed.stdout) + assert report["evidence_file"] == str(missing_path.resolve()) + assert report["evidence_load_error"] + assert report["evidence_counts"]["public_development_activity"] == 0 + integrity = next( + item for item in report["checks"] if item["name"] == "Evidence-ledger integrity" + ) + assert integrity["status"] == "block" + + invalid_encoding_path = tmp_path / "invalid-encoding.json" + invalid_encoding_path.write_bytes(b"\xff\xfe\x00") + invalid_encoding_payload = READINESS._load_evidence(invalid_encoding_path) + assert invalid_encoding_payload["schema"] == "invalid" + assert invalid_encoding_payload["error"] + + for payload in (missing_payload, invalid_encoding_payload): + errors = READINESS.validate_evidence_payload(payload) + assert "Unknown or missing evidence schema." in errors + assert payload != READINESS._load_evidence() + + +def test_official_build_at_current_head_is_accepted(tmp_path: Path, monkeypatch) -> None: + repo = tmp_path / "same-head" + head, _ = _init_paper_source_repo(repo) + monkeypatch.setattr(READINESS, "ROOT", repo) + + assert READINESS._official_build_sources_match(head, head) is True + metadata = READINESS._submission_metadata( + {"submission_metadata": _confirmed_submission(head, head)}, + current_head=head, + ) + assert metadata["ok"] is True + + +def test_ancestor_build_is_accepted_only_when_source_blobs_match_current_head( + tmp_path: Path, + monkeypatch, +) -> None: + repo = tmp_path / "same-paper-inputs" + build_commit, _ = _init_paper_source_repo(repo) + monkeypatch.setattr(READINESS, "ROOT", repo) + (repo / "paper/paper.pdf").write_bytes(b"PDF artifact created after source commit") + _git(repo, "add", "paper/paper.pdf") + _git(repo, "commit", "--quiet", "-m", "record official PDF") + current_head = _git(repo, "rev-parse", "HEAD") + + assert build_commit != current_head + assert READINESS._official_build_sources_match(build_commit, current_head) is True + accepted = READINESS._submission_metadata( + {"submission_metadata": _confirmed_submission(current_head, build_commit)}, + current_head=current_head, + ) + assert accepted["ok"] is True + + stale_ci = READINESS._submission_metadata( + {"submission_metadata": _confirmed_submission(build_commit, build_commit)}, + current_head=current_head, + ) + assert stale_ci["ok"] is False + assert stale_ci["invalid_or_mismatched_commits"] == ["remote_ci_commit"] + + +def test_ancestor_build_is_rejected_when_any_paper_source_blob_changed( + tmp_path: Path, + monkeypatch, +) -> None: + repo = tmp_path / "changed-paper-inputs" + build_commit, _ = _init_paper_source_repo(repo) + monkeypatch.setattr(READINESS, "ROOT", repo) + + for relative_path in READINESS.PAPER_SOURCE_PATHS: + _git(repo, "reset", "--hard", build_commit) + source = repo / relative_path + source.write_bytes(source.read_bytes() + b"changed\n") + _git(repo, "add", relative_path) + _git(repo, "commit", "--quiet", "-m", f"change {relative_path}") + current_head = _git(repo, "rev-parse", "HEAD") + assert READINESS._official_build_sources_match(build_commit, current_head) is False + metadata = READINESS._submission_metadata( + {"submission_metadata": _confirmed_submission(current_head, build_commit)}, + current_head=current_head, + ) + assert metadata["ok"] is False + assert metadata["invalid_or_mismatched_commits"] == ["official_joss_build_commit"] + + +def test_official_build_is_rejected_when_a_paper_source_blob_is_missing( + tmp_path: Path, + monkeypatch, +) -> None: + repo = tmp_path / "missing-paper-input" + build_commit, _ = _init_paper_source_repo(repo) + monkeypatch.setattr(READINESS, "ROOT", repo) + missing_path = repo / "paper/paper.bib" + missing_path.unlink() + _git(repo, "add", "--all") + _git(repo, "commit", "--quiet", "-m", "remove paper bibliography") + current_head = _git(repo, "rev-parse", "HEAD") + + assert READINESS._official_build_sources_match(build_commit, current_head) is False + metadata = READINESS._submission_metadata( + {"submission_metadata": _confirmed_submission(current_head, build_commit)}, + current_head=current_head, + ) + assert metadata["invalid_or_mismatched_commits"] == ["official_joss_build_commit"] + + +def test_official_build_commit_must_exist_and_be_an_ancestor(tmp_path: Path, monkeypatch) -> None: + repo = tmp_path / "commit-ancestry" + base_commit, base_branch = _init_paper_source_repo(repo) + monkeypatch.setattr(READINESS, "ROOT", repo) + + _git(repo, "checkout", "--quiet", "-b", "side-history") + (repo / "notes.txt").write_text("side branch\n", encoding="utf-8") + _git(repo, "add", "notes.txt") + _git(repo, "commit", "--quiet", "-m", "side branch") + side_commit = _git(repo, "rev-parse", "HEAD") + + _git(repo, "checkout", "--quiet", base_branch) + (repo / "notes.txt").write_text("main history\n", encoding="utf-8") + _git(repo, "add", "notes.txt") + _git(repo, "commit", "--quiet", "-m", "main branch") + current_head = _git(repo, "rev-parse", "HEAD") + + unknown_commit = "0" * 40 + assert READINESS._official_build_sources_match(unknown_commit, current_head) is False + unknown_metadata = READINESS._submission_metadata( + {"submission_metadata": _confirmed_submission(current_head, unknown_commit)}, + current_head=current_head, + ) + assert unknown_metadata["invalid_or_mismatched_commits"] == [ + "official_joss_build_commit" + ] + + assert READINESS._official_build_sources_match(side_commit, current_head) is False + nonancestor_metadata = READINESS._submission_metadata( + {"submission_metadata": _confirmed_submission(current_head, side_commit)}, + current_head=current_head, + ) + assert nonancestor_metadata["invalid_or_mismatched_commits"] == [ + "official_joss_build_commit" + ] + assert READINESS._official_build_sources_match(base_commit, current_head) is True + + def test_evidence_payload_validation_rejects_untraceable_and_duplicate_records() -> None: payload = _valid_payload() record = { diff --git a/tests/test_quick_export.py b/tests/test_quick_export.py index b92e65b..e665658 100644 --- a/tests/test_quick_export.py +++ b/tests/test_quick_export.py @@ -6,8 +6,10 @@ import pytest from openpyxl import load_workbook +from diffractscout.cli import main as cli_main +import diffractscout.quick_export as quick_export_module from diffractscout.elasticity_input import parse_cubic_cij -from diffractscout.models import AnalysisSettings +from diffractscout.models import AnalysisSettings, PipelineResult from diffractscout.pipeline import analyze_cifs from diffractscout.quick_export import build_parser as build_quick_export_parser from diffractscout.quick_export import main as quick_export_main @@ -201,6 +203,128 @@ def test_quick_export_directory_mode(demo_inputs: Path, tmp_path: Path) -> None: assert result.analyses[0].metadata.get("export_lab_views") is True +def test_quick_export_rejects_no_excel_for_xlsx_api_output( + demo_inputs: Path, tmp_path: Path +) -> None: + excel = tmp_path / "no-excel.xlsx" + + with pytest.raises(ValueError, match="include_excel=False.*incompatible"): + quick_export([demo_inputs], excel, include_excel=False) + + assert not excel.exists() + assert not (tmp_path / "no-excel_bundle").exists() + + +@pytest.mark.parametrize("entrypoint", ["standalone", "subcommand"]) +def test_quick_export_clis_reject_no_excel_for_xlsx_output( + demo_inputs: Path, tmp_path: Path, capsys, entrypoint: str +) -> None: + excel = tmp_path / f"no-excel-{entrypoint}.xlsx" + if entrypoint == "standalone": + args = [str(demo_inputs), "-o", str(excel), "--no-excel"] + code = quick_export_main(args) + else: + args = ["quick-export", str(demo_inputs), "-o", str(excel), "--no-excel"] + code = cli_main(args) + + assert code == 2 + assert "An .xlsx output path requires Excel output" in capsys.readouterr().err + assert not excel.exists() + assert not (tmp_path / f"no-excel-{entrypoint}_bundle").exists() + + +@pytest.mark.parametrize("failure", ["permission", "runtime"]) +def test_quick_export_reports_bundle_recovery_after_external_excel_failure( + demo_inputs: Path, tmp_path: Path, monkeypatch, failure: str +) -> None: + excel = tmp_path / f"locked-{failure}.xlsx" + excel.write_bytes(b"existing workbook") + + def fail_publish(_source: Path, target: Path, *, overwrite: bool) -> None: + assert overwrite is True + if failure == "permission": + raise PermissionError(errno.EACCES, "workbook is open", str(target)) + raise RuntimeError("simulated source integrity failure") + + monkeypatch.setattr("diffractscout.quick_export._copy_excel_atomic", fail_publish) + error_type = PermissionError if failure == "permission" else RuntimeError + with pytest.raises(error_type) as exc_info: + quick_export([demo_inputs], excel, overwrite=True) + + bundle = tmp_path / f"locked-{failure}_bundle" + recovery_error = str(exc_info.value) + source_workbook = bundle / "results.xlsx" + assert "bundle remains at" in recovery_error + assert str(bundle) in recovery_error + assert "diffractscout verify" in recovery_error + assert str(source_workbook) in recovery_error + assert excel.read_bytes() == b"existing workbook" + assert source_workbook.is_file() + assert verify_bundle(bundle)["ok"] + if failure == "permission": + assert exc_info.value.errno == errno.EACCES + assert exc_info.value.filename == str(excel.resolve()) + + +@pytest.mark.parametrize("entrypoint", ["standalone", "subcommand"]) +def test_quick_export_cli_prints_bundle_recovery_after_publish_failure( + demo_inputs: Path, tmp_path: Path, monkeypatch, capsys, entrypoint: str +) -> None: + excel = tmp_path / f"failed-{entrypoint}.xlsx" + + def fail_publish(_source: Path, _target: Path, *, overwrite: bool) -> None: + assert overwrite is True + raise RuntimeError("simulated workbook verification failure") + + monkeypatch.setattr("diffractscout.quick_export._copy_excel_atomic", fail_publish) + if entrypoint == "standalone": + code = quick_export_main( + [str(demo_inputs), "-o", str(excel), "--overwrite"] + ) + else: + code = cli_main( + ["quick-export", str(demo_inputs), "-o", str(excel), "--overwrite"] + ) + + bundle = tmp_path / f"failed-{entrypoint}_bundle" + error = capsys.readouterr().err + assert code == 2 + assert "simulated workbook verification failure" in error + assert str(bundle) in error + assert "diffractscout verify" in error + assert str(bundle / "results.xlsx") in error + assert (bundle / "results.xlsx").is_file() + + +def test_quick_export_missing_bundle_workbook_error_gives_safe_recovery_guidance( + tmp_path: Path, monkeypatch +) -> None: + excel = tmp_path / "missing-source.xlsx" + bundle = tmp_path / "missing-source_bundle" + bundle.mkdir() + result = PipelineResult( + output_dir=bundle, + discovery=None, + downloads=[], + analyses=[], + manifest_path=bundle / "manifest.json", + ) + monkeypatch.setattr( + "diffractscout.quick_export.analyze_cifs", + lambda *_args, **_kwargs: result, + ) + + with pytest.raises(RuntimeError, match="Expected results.xlsx") as exc_info: + quick_export([tmp_path / "unused.cif"], excel) + + error = str(exc_info.value) + assert str(bundle) in error + assert str(bundle / "results.xlsx") in error + assert "is missing" in error + assert "recover the workbook" not in error + assert not excel.exists() + + def test_quick_export_keyword_overrides_apply_to_explicit_settings( demo_inputs: Path, tmp_path: Path ) -> None: @@ -648,6 +772,91 @@ def test_standalone_quick_export_help_is_legacy_windows_console_safe() -> None: help_text.encode("cp936") +def test_standalone_parser_explicit_tracking_is_per_parse() -> None: + parser = build_quick_export_parser() + explicit = parser.parse_args(["sample.cif", "-o", "first", "--no-excel"]) + defaults = parser.parse_args(["sample.cif", "-o", "second"]) + + assert explicit._diffractscout_explicit == {"no_excel"} + assert not getattr(defaults, "_diffractscout_explicit", set()) + + +def test_cli_and_standalone_quick_export_apply_same_preset_overrides( + tmp_path: Path, monkeypatch, capsys +) -> None: + preset = tmp_path / "quick-preset.json" + assert cli_main( + [ + "preset", + "save", + "-o", + str(preset), + "--energy-keV", + "31", + "--two-theta-max", + "70", + "--no-patterns", + "--no-excel", + "--no-recursive", + ] + ) == 0 + capsys.readouterr() + + captured: list[dict[str, object]] = [] + + def fake_quick_export(inputs, output, **kwargs): + captured.append({"inputs": inputs, "output": output, **kwargs}) + output_dir = Path(output) + return PipelineResult( + output_dir=output_dir, + discovery=None, + downloads=[], + analyses=[], + manifest_path=output_dir / "manifest.json", + ) + + monkeypatch.setattr(quick_export_module, "quick_export", fake_quick_export) + cli_args = [ + "sample.cif", + "-o", + str(tmp_path / "cli-bundle"), + "--preset", + str(preset), + "--excel", + "--patterns", + "--recursive", + "--two-theta-max", + "75", + ] + assert cli_main(["quick-export", *cli_args]) == 2 + assert quick_export_main(cli_args) == 2 + capsys.readouterr() + + assert len(captured) == 2 + for call in captured: + settings = call["settings"] + assert settings.input_mode == "energy" + assert settings.energy_keV == pytest.approx(31) + assert settings.two_theta_max_deg == pytest.approx(75) + assert settings.include_patterns is True + assert call["include_excel"] is True + assert call["recursive"] is True + + +def test_standalone_quick_export_invalid_preset_fails_before_output( + demo_inputs: Path, tmp_path: Path, capsys +) -> None: + preset = tmp_path / "invalid-preset.json" + preset.write_text("{}", encoding="utf-8") + output = tmp_path / "should-not-exist" + + assert quick_export_main( + [str(demo_inputs), "-o", str(output), "--preset", str(preset)] + ) == 2 + assert not output.exists() + assert "Invalid analysis preset" in capsys.readouterr().err + + def test_elastic_override_replaces_sidecar(demo_inputs: Path, tmp_path: Path) -> None: """Override keyed by stem must be used instead of the demo sidecar.""" diff --git a/tests/test_reference_engine_comparison.py b/tests/test_reference_engine_comparison.py new file mode 100644 index 0000000..ad4fd50 --- /dev/null +++ b/tests/test_reference_engine_comparison.py @@ -0,0 +1,170 @@ +from __future__ import annotations + +import json +from pathlib import Path +import sys + +import pytest + + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / "scripts")) + +from compare_reference_engines import ( # noqa: E402 + _group_by_d_spacing, + _validate_expectations, + canonical_cubic_family, + match_d_spacing_groups, +) + +EXPECTATIONS_PATH = ROOT / "src" / "diffractscout" / "benchmark_data" / "expectations.json" + + +def _expectations() -> dict[str, object]: + return json.loads(EXPECTATIONS_PATH.read_text(encoding="utf-8")) + + +def _peak(d_A: float, intensity: float, hkl: tuple[int, int, int]) -> dict[str, object]: + return { + "d_A": d_A, + "two_theta_deg": 12.0, + "intensity": intensity, + "hkls": [hkl], + } + + +def test_grouping_merges_coincident_d_values_and_sums_intensity() -> None: + grouped = _group_by_d_spacing( + [_peak(2.0, 3.0, (2, 0, 0)), _peak(2.0 + 4e-7, 7.0, (1, 1, 0))], + tolerance_A=1e-6, + ) + + assert len(grouped) == 1 + assert grouped[0]["d_A"] == pytest.approx(2.0000002) + assert grouped[0]["intensity"] == pytest.approx(10.0) + assert grouped[0]["family_count"] == 2 + assert grouped[0]["hkls"] == [[1, 1, 0], [2, 0, 0]] + + +def test_grouping_does_not_chain_peaks_beyond_the_declared_window() -> None: + grouped = _group_by_d_spacing( + [ + _peak(3.0, 1.0, (3, 0, 0)), + _peak(3.0 - 0.6e-6, 1.0, (2, 2, 0)), + _peak(3.0 - 1.2e-6, 1.0, (2, 1, 1)), + ], + tolerance_A=1e-6, + ) + + assert len(grouped) == 2 + assert [group["family_count"] for group in grouped] == [2, 1] + + +def test_d_spacing_matching_is_one_to_one_and_reports_unmatched_groups() -> None: + left = [{"d_A": 3.0}, {"d_A": 2.0}] + right = [{"d_A": 3.0 + 0.5e-6}, {"d_A": 1.9}] + + matches, unmatched_left, unmatched_right = match_d_spacing_groups( + left, + right, + tolerance_A=1e-6, + ) + + assert matches == [(0, 0)] + assert unmatched_left == [1] + assert unmatched_right == [1] + + +@pytest.mark.parametrize( + ("hkl", "expected"), + [ + ((1, 0, 0), (1, 0, 0)), + ((-1, 1, 0), (1, 1, 0)), + ((0, -2, 1), (2, 1, 0)), + ], +) +def test_cubic_family_key_ignores_sign_and_permutation( + hkl: tuple[int, int, int], + expected: tuple[int, int, int], +) -> None: + assert canonical_cubic_family(hkl) == expected + + +@pytest.mark.parametrize("hkl", [(1, 0), (1, 0, 0, 0)]) +def test_cubic_family_key_requires_three_indices(hkl: tuple[int, ...]) -> None: + with pytest.raises(ValueError, match="exactly three"): + canonical_cubic_family(hkl) + + +def test_current_benchmark_expectations_define_all_four_required_cases() -> None: + cases = _validate_expectations(_expectations()) + + assert set(cases) == {"simple_cubic_al", "bcc_fe", "fcc_al", "nacl"} + + +@pytest.mark.parametrize("case_name", ["simple_cubic_al", "bcc_fe", "fcc_al", "nacl"]) +def test_missing_benchmark_case_fails_validation(case_name: str) -> None: + expectations = _expectations() + cases = expectations["cases"] + assert isinstance(cases, dict) + cases.pop(case_name) + + with pytest.raises(ValueError, match="requires exactly the packaged synthetic cases"): + _validate_expectations(expectations) + + +def test_empty_case_set_fails_validation() -> None: + expectations = _expectations() + expectations["cases"] = {} + + with pytest.raises(ValueError, match="requires exactly the packaged synthetic cases"): + _validate_expectations(expectations) + + +@pytest.mark.parametrize("field_name", ["first_allowed", "forbidden"]) +def test_required_reflection_list_cannot_be_empty(field_name: str) -> None: + expectations = _expectations() + cases = expectations["cases"] + assert isinstance(cases, dict) + specification = cases["bcc_fe"] + assert isinstance(specification, dict) + specification[field_name] = [] + + with pytest.raises(ValueError, match="must define exactly"): + _validate_expectations(expectations) + + +def test_required_forbidden_reflection_field_cannot_be_missing() -> None: + expectations = _expectations() + cases = expectations["cases"] + assert isinstance(cases, dict) + specification = cases["nacl"] + assert isinstance(specification, dict) + specification.pop("forbidden") + + with pytest.raises(ValueError, match="missing required fields.*forbidden"): + _validate_expectations(expectations) + + +@pytest.mark.parametrize( + ("case_name", "field_name", "value"), + [ + ("bcc_fe", "first_allowed", [[1, 1]]), + ("fcc_al", "forbidden", [[0, 0, 0], [1, 0, 0]]), + ("nacl", "first_allowed", [[1, 1, 1], [1, 1, 1], [2, 0, 0]]), + ], +) +def test_malformed_or_duplicate_reflection_families_fail_validation( + case_name: str, + field_name: str, + value: list[list[int]], +) -> None: + expectations = _expectations() + cases = expectations["cases"] + assert isinstance(cases, dict) + specification = cases[case_name] + assert isinstance(specification, dict) + specification[field_name] = value + + with pytest.raises(ValueError): + _validate_expectations(expectations) diff --git a/validation_cases/README.md b/validation_cases/README.md index bb30b2d..293c97e 100644 --- a/validation_cases/README.md +++ b/validation_cases/README.md @@ -21,5 +21,13 @@ Use `docs/VALIDATION_CASE_TEMPLATE.md` for real-material or independent-package comparisons. Public records are indexed only after completion in `docs/evidence/impact_evidence.json`. +Available numerical cases: + +- [Analytic reference suite](analytic_reference_v1/README.md): closed-form + crystallographic and cubic-elasticity expectations. +- [Independent pymatgen comparison](independent_engines/README.md): the same + synthetic structures evaluated by separate diffraction and tensor + implementations, with versioned numerical reports. + `real_multiphase_case/README.md` is an intake scaffold. Its presence is not research-use evidence and its pending fields must never be cited in the paper. diff --git a/validation_cases/independent_engines/README.md b/validation_cases/independent_engines/README.md new file mode 100644 index 0000000..f4463ec --- /dev/null +++ b/validation_cases/independent_engines/README.md @@ -0,0 +1,63 @@ +# Independent reference-engine comparison + +This case compares the offline Gemmi diffraction implementation with pymatgen's `XRDCalculator` using the project's explicitly synthetic analytic CIF fixtures. It also compares the project's cubic directional Young's-modulus calculation with a fourth-rank compliance-tensor contraction from pymatgen. The case is a numerical software cross-check; it is not experimental validation, a real-material property claim, or evidence of external-user adoption. + +The generated results are in [the Markdown receipt](reference_engine_comparison.md) and [the machine-readable JSON receipt](reference_engine_comparison.json). + +The inputs are the four CIFs already distributed with the package: + +| Fixture | Synthetic cell and basis | Selection rule checked | +|---|---|---| +| `simple_cubic_al.cif` | Pm-3m, a = 3.5 Å, one Al site | Primitive-cubic reflections | +| `bcc_fe.cif` | Im-3m, a = 2.86 Å, one Fe site | h + k + l even; (100) and (111) absent | +| `fcc_al.cif` | Fm-3m, a = 4.0 Å, one Al site | h, k, l all odd or all even; (100) and (110) absent | +| `nacl.cif` | Fm-3m, a = 5.64 Å, Na at (0,0,0), Cl at (1/2,1/2,1/2) | F-centering parity rule; (100) and (110) absent | + +The expected initial allowed families and cubic lattice parameters are maintained in [`expectations.json`](../../src/diffractscout/benchmark_data/expectations.json). The CIFs themselves are hashed in the generated JSON receipt so a changed fixture cannot silently reuse these results. + +## Diffraction comparison + +Both calculations use the same conventional synthetic CIF, an effective single wavelength of 1.5406 Å (Cu Kα), and a 2θ interval of 5–120°. DiffractScout is evaluated with `source_preset="Cu Ka"`; pymatgen reads each structure with `primitive=False` and runs `XRDCalculator(wavelength=1.5406, symprec=0).get_pattern(..., scaled=False, two_theta_range=(5, 120))`. + +Peak families are grouped and matched by d-spacing, with a grouping/matching tolerance of 1 × 10⁻⁶ Å. A position comparison passes only when every group is matched and the maximum absolute differences satisfy both |Δd| ≤ 1 × 10⁻⁶ Å and |Δ2θ| ≤ 1 × 10⁻⁵°. This avoids requiring the two engines to choose identical representative Miller indices. It also accounts for coincident families: the script sums their calculated intensities before normalizing each case to a maximum of 100. + +The tracked receipt was generated with Python 3.12.10, Gemmi 0.7.5, and pymatgen 2026.5.4. All four cases passed the peak-position and selection-rule checks. The maximum observed |Δd| was 1.33 × 10⁻¹⁵ Å and maximum |Δ2θ| was 7.11 × 10⁻¹⁴°. The maximum difference between independently normalized intensities was 0.790 percentage points across the four cases. Intensity differences are reported peak by peak in the JSON receipt but do not have an acceptance threshold: this is not a common-form-factor-model comparison, and agreement in the calculated pattern does not establish absolute experimental intensity accuracy. + +The fixture CIFs omit explicit symmetry-operation loops and provide Hermann–Mauguin symbols and International Tables numbers. Pymatgen records a parser warning that it uses the declared Hermann–Mauguin symbol; these warnings are included in the receipt. + +## Elastic comparison + +The modulus check uses `fcc_al.cif` only to define a cubic Cartesian frame. It pairs that synthetic structure with the explicitly synthetic stiffness matrix (GPa) + +```text +[[250, 150, 150, 0, 0, 0], + [150, 250, 150, 0, 0, 0], + [150, 150, 250, 0, 0, 0], + [ 0, 0, 0, 100, 0, 0], + [ 0, 0, 0, 0, 100, 0], + [ 0, 0, 0, 0, 0, 100]] +``` + +The order is [11, 22, 33, 23, 13, 12], with engineering shear strain. Pymatgen's `ElasticTensor.from_voigt` converts this matrix into its fourth-rank tensor representation; the reference Young's modulus is calculated as the reciprocal of the fourth-rank compliance contraction along each unit direction. This distinction matters because pymatgen's `ElasticTensor.directional_elastic_mod` contracts the stiffness tensor and therefore returns a longitudinal modulus, not the Young's modulus emitted by DiffractScout. + +For [100], [110], and [111], both implementations returned 137.5, 209.5238095238096, and 253.8461538461537 GPa, respectively. The predeclared acceptance rule is |ΔE| ≤ max(1 × 10⁻⁸ GPa, 1 × 10⁻¹⁰ × |E_pymatgen|). The largest observed absolute difference was 5.69 × 10⁻¹⁴ GPa. This is a convention and implementation check on an analytic tensor, not validation of an experimental or database elastic tensor. + +## Reproduction + +The optional `mp` dependency group supplies pymatgen. From a development installation: + +```text +pip install -e ".[mp]" +python scripts/compare_reference_engines.py --output validation_cases/independent_engines +``` + +The script exits with status 0 only when geometry, selection-rule, and elastic checks pass. It writes `reference_engine_comparison.json` with the per-peak values, hashes, software versions, warnings, and tolerances, plus `reference_engine_comparison.md` with a compact result table. Intensity differences remain diagnostic and never affect the exit status. The script and matching helpers are tested by `tests/test_reference_engine_comparison.py`; those tests need only the project's core/test dependencies and do not import pymatgen. + +## Official method references + +- [pymatgen XRDCalculator documentation](https://pymatgen.org/pymatgen.analysis.diffraction.html#pymatgen.analysis.diffraction.xrd.XRDCalculator) describes the reciprocal-point search, atomic scattering factors, structure-factor intensity, Lorentz-polarization correction, configurable wavelength, and unscaled-pattern option. +- [Gemmi scattering documentation](https://gemmi.readthedocs.io/en/latest/scattering.html) describes X-ray form factors and the direct `StructureFactorCalculatorX` summation, including occupancy conversion for small-structure CIF input. +- [pymatgen ElasticTensor documentation](https://pymatgen.org/pymatgen.core.elasticity.html#pymatgen.core.elasticity.elastic.ElasticTensor) describes the fourth-rank elastic tensor, its GPa units, and compliance tensor derived from the Voigt stiffness matrix. +- The calculation quantities and conventions checked here are also stated in [`SCIENTIFIC_CONTRACTS.md`](../../docs/SCIENTIFIC_CONTRACTS.md), especially its diffraction intensity and elastic-tensor sections. + +Documentation pages were checked on 2026-09-26. The JSON receipt records the exact package versions used for its numerical results. diff --git a/validation_cases/independent_engines/reference_engine_comparison.json b/validation_cases/independent_engines/reference_engine_comparison.json new file mode 100644 index 0000000..9c9c4b8 --- /dev/null +++ b/validation_cases/independent_engines/reference_engine_comparison.json @@ -0,0 +1,1409 @@ +{ + "all_acceptance_checks_passed": true, + "diffraction_cases": { + "bcc_fe": { + "gemmi_d_spacing_group_count": 5, + "gemmi_reflection_family_count": 5, + "input_cif": "src/diffractscout/benchmark_data/bcc_fe.cif", + "input_sha256": "eb39f2b2e2a9a29cf8429b3b62da7f5dbea03cc765db8e8f1009c17043c15f6a", + "intensity_comparison": { + "interpretation": "diagnostic_only_no_acceptance_threshold", + "max_abs_delta_relative_intensity_percentage_points": 0.46572887223167214, + "peaks": [ + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.0, + "abs_delta_two_theta_deg": 7.105427357601002e-15, + "d_A_gemmi": 2.022325394193526, + "d_A_pymatgen": 2.022325394193526, + "gemmi_hkl_families": [ + [ + 1, + 1, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 100.0, + "pymatgen_hkl_families": [ + [ + 1, + 1, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 100.0, + "two_theta_deg_gemmi": 44.77865438380817, + "two_theta_deg_pymatgen": 44.778654383808174 + }, + { + "abs_delta_d_A": 2.220446049250313e-16, + "abs_delta_relative_intensity_percentage_points": 0.047089695925629727, + "abs_delta_two_theta_deg": 1.4210854715202004e-14, + "d_A_gemmi": 1.4300000000000002, + "d_A_pymatgen": 1.43, + "gemmi_hkl_families": [ + [ + 2, + 0, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 14.630914795092721, + "pymatgen_hkl_families": [ + [ + 2, + 0, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 14.678004491018351, + "two_theta_deg_gemmi": 65.18647232008216, + "two_theta_deg_pymatgen": 65.18647232008217 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.46572887223167214, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 1.1675901107266482, + "d_A_pymatgen": 1.1675901107266482, + "gemmi_hkl_families": [ + [ + 2, + 1, + 1 + ] + ], + "gemmi_relative_intensity_max_100": 28.021847796210363, + "pymatgen_hkl_families": [ + [ + 2, + 1, + 1 + ] + ], + "pymatgen_relative_intensity_max_100": 28.487576668442035, + "two_theta_deg_gemmi": 82.55932297686478, + "two_theta_deg_pymatgen": 82.55932297686478 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.13577086051555654, + "abs_delta_two_theta_deg": 2.842170943040401e-14, + "d_A_gemmi": 1.011162697096763, + "d_A_pymatgen": 1.011162697096763, + "gemmi_hkl_families": [ + [ + 2, + 2, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 9.648279238460761, + "pymatgen_hkl_families": [ + [ + 2, + 2, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 9.784050098976317, + "two_theta_deg_gemmi": 99.24562508220849, + "two_theta_deg_pymatgen": 99.24562508220852 + }, + { + "abs_delta_d_A": 3.3306690738754696e-16, + "abs_delta_relative_intensity_percentage_points": 0.033953068601988434, + "abs_delta_two_theta_deg": 7.105427357601002e-14, + "d_A_gemmi": 0.9044114108081566, + "d_A_pymatgen": 0.9044114108081562, + "gemmi_hkl_families": [ + [ + 3, + 1, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 18.33027125737359, + "pymatgen_hkl_families": [ + [ + 3, + 1, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 18.2963181887716, + "two_theta_deg_gemmi": 116.79720523036076, + "two_theta_deg_pymatgen": 116.79720523036083 + } + ] + }, + "matched_d_spacing_group_count": 5, + "position_max_abs_delta_d_A": 3.3306690738754696e-16, + "position_max_abs_delta_two_theta_deg": 7.105427357601002e-14, + "position_passed": true, + "pymatgen_d_spacing_group_count": 5, + "pymatgen_parse_warnings": [ + "No _symmetry_equiv_pos_as_xyz type key found. Spacegroup from _space_group_name_H-M_alt used.", + "Issues encountered while parsing CIF: No _symmetry_equiv_pos_as_xyz type key found. Spacegroup from _space_group_name_H-M_alt used." + ], + "pymatgen_pattern_peak_count": 5, + "pymatgen_structure_site_count": 2, + "selection_rules": { + "expected_allowed_families": [ + [ + 1, + 1, + 0 + ], + [ + 2, + 0, + 0 + ], + [ + 2, + 1, + 1 + ] + ], + "expected_forbidden_families": [ + [ + 1, + 0, + 0 + ], + [ + 1, + 1, + 1 + ] + ], + "gemmi_allowed_missing": [], + "gemmi_forbidden_present": [], + "gemmi_rule_violations": [], + "passed": true, + "pymatgen_allowed_missing": [], + "pymatgen_forbidden_present": [], + "pymatgen_rule_violations": [] + }, + "unmatched_gemmi_group_indices": [], + "unmatched_pymatgen_group_indices": [] + }, + "fcc_al": { + "gemmi_d_spacing_group_count": 8, + "gemmi_reflection_family_count": 8, + "input_cif": "src/diffractscout/benchmark_data/fcc_al.cif", + "input_sha256": "c82f082d2c1ed45c2219713cf972ffce67a7df78d00eb8b953541363c8f0ffdf", + "intensity_comparison": { + "interpretation": "diagnostic_only_no_acceptance_threshold", + "max_abs_delta_relative_intensity_percentage_points": 0.7898700632332449, + "peaks": [ + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.0, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 2.3094010767585034, + "d_A_pymatgen": 2.3094010767585034, + "gemmi_hkl_families": [ + [ + 1, + 1, + 1 + ] + ], + "gemmi_relative_intensity_max_100": 100.0, + "pymatgen_hkl_families": [ + [ + 1, + 1, + 1 + ] + ], + "pymatgen_relative_intensity_max_100": 100.0, + "two_theta_deg_gemmi": 38.96873821328758, + "two_theta_deg_pymatgen": 38.96873821328758 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.22428952770014376, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 2.0, + "d_A_pymatgen": 2.0, + "gemmi_hkl_families": [ + [ + 2, + 0, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 48.29121673156484, + "pymatgen_hkl_families": [ + [ + 2, + 0, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 48.066927203864694, + "two_theta_deg_gemmi": 45.306105520893105, + "two_theta_deg_pymatgen": 45.306105520893105 + }, + { + "abs_delta_d_A": 2.220446049250313e-16, + "abs_delta_relative_intensity_percentage_points": 0.3999533907942343, + "abs_delta_two_theta_deg": 1.4210854715202004e-14, + "d_A_gemmi": 1.414213562373095, + "d_A_pymatgen": 1.4142135623730951, + "gemmi_hkl_families": [ + [ + 2, + 2, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 30.513892889111354, + "pymatgen_hkl_families": [ + [ + 2, + 2, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 30.913846279905588, + "two_theta_deg_gemmi": 66.00619219591967, + "two_theta_deg_pymatgen": 66.00619219591965 + }, + { + "abs_delta_d_A": 2.220446049250313e-16, + "abs_delta_relative_intensity_percentage_points": 0.7898700632332449, + "abs_delta_two_theta_deg": 2.842170943040401e-14, + "d_A_gemmi": 1.2060453783110545, + "d_A_pymatgen": 1.2060453783110547, + "gemmi_hkl_families": [ + [ + 3, + 1, + 1 + ] + ], + "gemmi_relative_intensity_max_100": 35.41251088669372, + "pymatgen_hkl_families": [ + [ + 3, + 1, + 1 + ] + ], + "pymatgen_relative_intensity_max_100": 36.20238094992696, + "two_theta_deg_gemmi": 79.38975378111094, + "two_theta_deg_pymatgen": 79.38975378111091 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.21145693021524004, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 1.1547005383792517, + "d_A_pymatgen": 1.1547005383792517, + "gemmi_hkl_families": [ + [ + 2, + 2, + 2 + ] + ], + "gemmi_relative_intensity_max_100": 10.292082591118975, + "pymatgen_hkl_families": [ + [ + 2, + 2, + 2 + ] + ], + "pymatgen_relative_intensity_max_100": 10.503539521334215, + "two_theta_deg_gemmi": 83.68717096748962, + "two_theta_deg_pymatgen": 83.68717096748962 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.017875445143724633, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 1.0, + "d_A_pymatgen": 1.0, + "gemmi_hkl_families": [ + [ + 4, + 0, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 5.47318794292648, + "pymatgen_hkl_families": [ + [ + 4, + 0, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 5.455312497782756, + "two_theta_deg_gemmi": 100.76167247514275, + "two_theta_deg_pymatgen": 100.76167247514275 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.5485054328663317, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 0.917662935482247, + "d_A_pymatgen": 0.917662935482247, + "gemmi_hkl_families": [ + [ + 3, + 3, + 1 + ] + ], + "gemmi_relative_intensity_max_100": 20.675078838275375, + "pymatgen_hkl_families": [ + [ + 3, + 3, + 1 + ] + ], + "pymatgen_relative_intensity_max_100": 20.126573405409044, + "two_theta_deg_gemmi": 114.15678547727177, + "two_theta_deg_pymatgen": 114.15678547727177 + }, + { + "abs_delta_d_A": 1.1102230246251565e-16, + "abs_delta_relative_intensity_percentage_points": 0.7037320782201917, + "abs_delta_two_theta_deg": 4.263256414560601e-14, + "d_A_gemmi": 0.8944271909999159, + "d_A_pymatgen": 0.894427190999916, + "gemmi_hkl_families": [ + [ + 4, + 2, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 21.038988672278386, + "pymatgen_hkl_families": [ + [ + 4, + 2, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 20.335256594058194, + "two_theta_deg_gemmi": 118.90803983023025, + "two_theta_deg_pymatgen": 118.90803983023021 + } + ] + }, + "matched_d_spacing_group_count": 8, + "position_max_abs_delta_d_A": 2.220446049250313e-16, + "position_max_abs_delta_two_theta_deg": 4.263256414560601e-14, + "position_passed": true, + "pymatgen_d_spacing_group_count": 8, + "pymatgen_parse_warnings": [ + "No _symmetry_equiv_pos_as_xyz type key found. Spacegroup from _space_group_name_H-M_alt used.", + "Issues encountered while parsing CIF: No _symmetry_equiv_pos_as_xyz type key found. Spacegroup from _space_group_name_H-M_alt used." + ], + "pymatgen_pattern_peak_count": 8, + "pymatgen_structure_site_count": 4, + "selection_rules": { + "expected_allowed_families": [ + [ + 1, + 1, + 1 + ], + [ + 2, + 0, + 0 + ], + [ + 2, + 2, + 0 + ] + ], + "expected_forbidden_families": [ + [ + 1, + 0, + 0 + ], + [ + 1, + 1, + 0 + ] + ], + "gemmi_allowed_missing": [], + "gemmi_forbidden_present": [], + "gemmi_rule_violations": [], + "passed": true, + "pymatgen_allowed_missing": [], + "pymatgen_forbidden_present": [], + "pymatgen_rule_violations": [] + }, + "unmatched_gemmi_group_indices": [], + "unmatched_pymatgen_group_indices": [] + }, + "nacl": { + "gemmi_d_spacing_group_count": 14, + "gemmi_reflection_family_count": 16, + "input_cif": "src/diffractscout/benchmark_data/nacl.cif", + "input_sha256": "1f9eb32ede4720e3b96394f0145cac0a6ca23eabe52c37a6a76ea2131ce238bc", + "intensity_comparison": { + "interpretation": "diagnostic_only_no_acceptance_threshold", + "max_abs_delta_relative_intensity_percentage_points": 0.5716221965467092, + "peaks": [ + { + "abs_delta_d_A": 1.3322676295501878e-15, + "abs_delta_relative_intensity_percentage_points": 0.06260894102152292, + "abs_delta_two_theta_deg": 1.7763568394002505e-14, + "d_A_gemmi": 3.2562555182294886, + "d_A_pymatgen": 3.25625551822949, + "gemmi_hkl_families": [ + [ + 1, + 1, + 1 + ] + ], + "gemmi_relative_intensity_max_100": 8.139645274047899, + "pymatgen_hkl_families": [ + [ + 1, + 1, + 1 + ] + ], + "pymatgen_relative_intensity_max_100": 8.077036333026376, + "two_theta_deg_gemmi": 27.36720162580169, + "two_theta_deg_pymatgen": 27.367201625801673 + }, + { + "abs_delta_d_A": 4.440892098500626e-16, + "abs_delta_relative_intensity_percentage_points": 0.0, + "abs_delta_two_theta_deg": 7.105427357601002e-15, + "d_A_gemmi": 2.8199999999999994, + "d_A_pymatgen": 2.82, + "gemmi_hkl_families": [ + [ + 2, + 0, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 100.0, + "pymatgen_hkl_families": [ + [ + 2, + 0, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 100.0, + "two_theta_deg_gemmi": 31.704311174001937, + "two_theta_deg_pymatgen": 31.70431117400193 + }, + { + "abs_delta_d_A": 8.881784197001252e-16, + "abs_delta_relative_intensity_percentage_points": 0.20621766037753275, + "abs_delta_two_theta_deg": 2.1316282072803006e-14, + "d_A_gemmi": 1.9940411229460635, + "d_A_pymatgen": 1.9940411229460644, + "gemmi_hkl_families": [ + [ + 2, + 2, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 65.8425359724745, + "pymatgen_hkl_families": [ + [ + 2, + 2, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 65.63631831209696, + "two_theta_deg_gemmi": 45.44905844251734, + "two_theta_deg_pymatgen": 45.44905844251732 + }, + { + "abs_delta_d_A": 6.661338147750939e-16, + "abs_delta_relative_intensity_percentage_points": 0.03827179150902538, + "abs_delta_two_theta_deg": 2.842170943040401e-14, + "d_A_gemmi": 1.7005239834185863, + "d_A_pymatgen": 1.700523983418587, + "gemmi_hkl_families": [ + [ + 3, + 1, + 1 + ] + ], + "gemmi_relative_intensity_max_100": 1.9845099384969627, + "pymatgen_hkl_families": [ + [ + 3, + 1, + 1 + ] + ], + "pymatgen_relative_intensity_max_100": 1.9462381469879373, + "two_theta_deg_gemmi": 53.869824509787506, + "two_theta_deg_pymatgen": 53.86982450978748 + }, + { + "abs_delta_d_A": 6.661338147750939e-16, + "abs_delta_relative_intensity_percentage_points": 0.0011862680548802018, + "abs_delta_two_theta_deg": 2.842170943040401e-14, + "d_A_gemmi": 1.6281277591147443, + "d_A_pymatgen": 1.628127759114745, + "gemmi_hkl_families": [ + [ + 2, + 2, + 2 + ] + ], + "gemmi_relative_intensity_max_100": 21.228664702696925, + "pymatgen_hkl_families": [ + [ + 2, + 2, + 2 + ] + ], + "pymatgen_relative_intensity_max_100": 21.227478434642045, + "two_theta_deg_gemmi": 56.474043221505326, + "two_theta_deg_pymatgen": 56.4740432215053 + }, + { + "abs_delta_d_A": 2.220446049250313e-16, + "abs_delta_relative_intensity_percentage_points": 0.0048705713917573945, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 1.4099999999999997, + "d_A_pymatgen": 1.41, + "gemmi_hkl_families": [ + [ + 4, + 0, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 9.314534004648767, + "pymatgen_hkl_families": [ + [ + 4, + 0, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 9.319404576040524, + "two_theta_deg_gemmi": 66.22874134790135, + "two_theta_deg_pymatgen": 66.22874134790135 + }, + { + "abs_delta_d_A": 4.440892098500626e-16, + "abs_delta_relative_intensity_percentage_points": 0.09011295531453967, + "abs_delta_two_theta_deg": 4.263256414560601e-14, + "d_A_gemmi": 1.293904739029968, + "d_A_pymatgen": 1.2939047390299685, + "gemmi_hkl_families": [ + [ + 3, + 3, + 1 + ] + ], + "gemmi_relative_intensity_max_100": 0.998200732612788, + "pymatgen_hkl_families": [ + [ + 3, + 3, + 1 + ] + ], + "pymatgen_relative_intensity_max_100": 0.9080877772982483, + "two_theta_deg_gemmi": 73.07228727290374, + "two_theta_deg_pymatgen": 73.0722872729037 + }, + { + "abs_delta_d_A": 4.440892098500626e-16, + "abs_delta_relative_intensity_percentage_points": 0.08886556058706319, + "abs_delta_two_theta_deg": 2.842170943040401e-14, + "d_A_gemmi": 1.261142339309881, + "d_A_pymatgen": 1.2611423393098815, + "gemmi_hkl_families": [ + [ + 4, + 2, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 24.834570201530664, + "pymatgen_hkl_families": [ + [ + 4, + 2, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 24.7457046409436, + "two_theta_deg_gemmi": 75.29408255035118, + "two_theta_deg_pymatgen": 75.29408255035115 + }, + { + "abs_delta_d_A": 4.440892098500626e-16, + "abs_delta_relative_intensity_percentage_points": 0.1968619672090277, + "abs_delta_two_theta_deg": 2.842170943040401e-14, + "d_A_gemmi": 1.1512601791080934, + "d_A_pymatgen": 1.1512601791080939, + "gemmi_hkl_families": [ + [ + 4, + 2, + 2 + ] + ], + "gemmi_relative_intensity_max_100": 18.420438665205754, + "pymatgen_hkl_families": [ + [ + 4, + 2, + 2 + ] + ], + "pymatgen_relative_intensity_max_100": 18.223576697996727, + "two_theta_deg_gemmi": 83.99418428895027, + "two_theta_deg_pymatgen": 83.99418428895024 + }, + { + "abs_delta_d_A": 2.220446049250313e-16, + "abs_delta_relative_intensity_percentage_points": 0.06155686886091383, + "abs_delta_two_theta_deg": 2.842170943040401e-14, + "d_A_gemmi": 1.0854185060764963, + "d_A_pymatgen": 1.0854185060764965, + "gemmi_hkl_families": [ + [ + 3, + 3, + 3 + ], + [ + 5, + 1, + 1 + ] + ], + "gemmi_relative_intensity_max_100": 1.1548722014134176, + "pymatgen_hkl_families": [ + [ + 3, + 3, + 3 + ], + [ + 5, + 1, + 1 + ] + ], + "pymatgen_relative_intensity_max_100": 1.0933153325525038, + "two_theta_deg_gemmi": 90.41779958572545, + "two_theta_deg_pymatgen": 90.41779958572542 + }, + { + "abs_delta_d_A": 4.440892098500626e-16, + "abs_delta_relative_intensity_percentage_points": 0.18253840224675866, + "abs_delta_two_theta_deg": 5.684341886080802e-14, + "d_A_gemmi": 0.9970205614730318, + "d_A_pymatgen": 0.9970205614730322, + "gemmi_hkl_families": [ + [ + 4, + 4, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 6.703592061468612, + "pymatgen_hkl_families": [ + [ + 4, + 4, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 6.521053659221853, + "two_theta_deg_gemmi": 101.1762348217296, + "two_theta_deg_pymatgen": 101.17623482172954 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.027403105282516327, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 0.9533339993337666, + "d_A_pymatgen": 0.9533339993337666, + "gemmi_hkl_families": [ + [ + 5, + 3, + 1 + ] + ], + "gemmi_relative_intensity_max_100": 1.9757193200378447, + "pymatgen_hkl_families": [ + [ + 5, + 3, + 1 + ] + ], + "pymatgen_relative_intensity_max_100": 1.9483162147553283, + "two_theta_deg_gemmi": 107.80321779302274, + "two_theta_deg_pymatgen": 107.80321779302274 + }, + { + "abs_delta_d_A": 2.220446049250313e-16, + "abs_delta_relative_intensity_percentage_points": 0.5680433547823025, + "abs_delta_two_theta_deg": 4.263256414560601e-14, + "d_A_gemmi": 0.9399999999999998, + "d_A_pymatgen": 0.9400000000000001, + "gemmi_hkl_families": [ + [ + 4, + 4, + 2 + ], + [ + 6, + 0, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 16.20785772195443, + "pymatgen_hkl_families": [ + [ + 4, + 4, + 2 + ], + [ + 6, + 0, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 15.639814367172127, + "two_theta_deg_gemmi": 110.06316485468736, + "two_theta_deg_pymatgen": 110.06316485468732 + }, + { + "abs_delta_d_A": 3.3306690738754696e-16, + "abs_delta_relative_intensity_percentage_points": 0.5716221965467092, + "abs_delta_two_theta_deg": 7.105427357601002e-14, + "d_A_gemmi": 0.8917623001674827, + "d_A_pymatgen": 0.891762300167483, + "gemmi_hkl_families": [ + [ + 6, + 2, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 13.543001540901981, + "pymatgen_hkl_families": [ + [ + 6, + 2, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 12.971379344355272, + "two_theta_deg_gemmi": 119.49083397824339, + "two_theta_deg_pymatgen": 119.49083397824332 + } + ] + }, + "matched_d_spacing_group_count": 14, + "position_max_abs_delta_d_A": 1.3322676295501878e-15, + "position_max_abs_delta_two_theta_deg": 7.105427357601002e-14, + "position_passed": true, + "pymatgen_d_spacing_group_count": 14, + "pymatgen_parse_warnings": [ + "No _symmetry_equiv_pos_as_xyz type key found. Spacegroup from _space_group_name_H-M_alt used.", + "Issues encountered while parsing CIF: No _symmetry_equiv_pos_as_xyz type key found. Spacegroup from _space_group_name_H-M_alt used." + ], + "pymatgen_pattern_peak_count": 14, + "pymatgen_structure_site_count": 8, + "selection_rules": { + "expected_allowed_families": [ + [ + 1, + 1, + 1 + ], + [ + 2, + 0, + 0 + ], + [ + 2, + 2, + 0 + ] + ], + "expected_forbidden_families": [ + [ + 1, + 0, + 0 + ], + [ + 1, + 1, + 0 + ] + ], + "gemmi_allowed_missing": [], + "gemmi_forbidden_present": [], + "gemmi_rule_violations": [], + "passed": true, + "pymatgen_allowed_missing": [], + "pymatgen_forbidden_present": [], + "pymatgen_rule_violations": [] + }, + "unmatched_gemmi_group_indices": [], + "unmatched_pymatgen_group_indices": [] + }, + "simple_cubic_al": { + "gemmi_d_spacing_group_count": 13, + "gemmi_reflection_family_count": 14, + "input_cif": "src/diffractscout/benchmark_data/simple_cubic_al.cif", + "input_sha256": "a7444474c5eee13163748b2946b8a5fe0f8fac5a57c4ef9b19fc7d56126e5b6a", + "intensity_comparison": { + "interpretation": "diagnostic_only_no_acceptance_threshold", + "max_abs_delta_relative_intensity_percentage_points": 0.4517144092258931, + "peaks": [ + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 2.842170943040401e-14, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 3.5, + "d_A_pymatgen": 3.5, + "gemmi_hkl_families": [ + [ + 1, + 0, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 100.00000000000001, + "pymatgen_hkl_families": [ + [ + 1, + 0, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 99.99999999999999, + "two_theta_deg_gemmi": 25.42813490960536, + "two_theta_deg_pymatgen": 25.42813490960536 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.2563779708508491, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 2.4748737341529163, + "d_A_pymatgen": 2.4748737341529163, + "gemmi_hkl_families": [ + [ + 1, + 1, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 74.69005654200055, + "pymatgen_hkl_families": [ + [ + 1, + 1, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 74.4336785711497, + "two_theta_deg_gemmi": 36.26893808746654, + "two_theta_deg_pymatgen": 36.26893808746654 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.2775465128193808, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 2.0207259421636903, + "d_A_pymatgen": 2.0207259421636903, + "gemmi_hkl_families": [ + [ + 1, + 1, + 1 + ] + ], + "gemmi_relative_intensity_max_100": 26.95950878518283, + "pymatgen_hkl_families": [ + [ + 1, + 1, + 1 + ] + ], + "pymatgen_relative_intensity_max_100": 26.68196227236345, + "two_theta_deg_gemmi": 44.816021759388924, + "two_theta_deg_pymatgen": 44.816021759388924 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.12425766304114205, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 1.75, + "d_A_pymatgen": 1.75, + "gemmi_hkl_families": [ + [ + 2, + 0, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 12.759791203456631, + "pymatgen_hkl_families": [ + [ + 2, + 0, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 12.635533540415489, + "two_theta_deg_gemmi": 52.22963892827337, + "two_theta_deg_pymatgen": 52.22963892827337 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.09600282069863653, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 1.565247584249853, + "d_A_pymatgen": 1.565247584249853, + "gemmi_hkl_families": [ + [ + 2, + 1, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 35.1223210103959, + "pymatgen_hkl_families": [ + [ + 2, + 1, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 35.026318189697264, + "two_theta_deg_gemmi": 58.96090819680111, + "two_theta_deg_pymatgen": 58.96090819680111 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.15666235260220063, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 1.4288690166235207, + "d_A_pymatgen": 1.4288690166235207, + "gemmi_hkl_families": [ + [ + 2, + 1, + 1 + ] + ], + "gemmi_relative_intensity_max_100": 25.645664419073256, + "pymatgen_hkl_families": [ + [ + 2, + 1, + 1 + ] + ], + "pymatgen_relative_intensity_max_100": 25.802326771675457, + "two_theta_deg_gemmi": 65.24447291294112, + "two_theta_deg_pymatgen": 65.24447291294112 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.12675886458547225, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 1.2374368670764582, + "d_A_pymatgen": 1.2374368670764582, + "gemmi_hkl_families": [ + [ + 2, + 2, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 7.812811839943226, + "pymatgen_hkl_families": [ + [ + 2, + 2, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 7.939570704528698, + "two_theta_deg_gemmi": 76.99733036167903, + "two_theta_deg_pymatgen": 76.99733036167903 + }, + { + "abs_delta_d_A": 2.220446049250313e-16, + "abs_delta_relative_intensity_percentage_points": 0.24499058609759672, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 1.1666666666666667, + "d_A_pymatgen": 1.166666666666667, + "gemmi_hkl_families": [ + [ + 2, + 2, + 1 + ], + [ + 3, + 0, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 16.150152710111154, + "pymatgen_hkl_families": [ + [ + 2, + 2, + 1 + ], + [ + 3, + 0, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 16.39514329620875, + "two_theta_deg_gemmi": 82.63897383118324, + "two_theta_deg_pymatgen": 82.63897383118324 + }, + { + "abs_delta_d_A": 2.220446049250313e-16, + "abs_delta_relative_intensity_percentage_points": 0.11491686292123227, + "abs_delta_two_theta_deg": 1.4210854715202004e-14, + "d_A_gemmi": 1.1067971810589328, + "d_A_pymatgen": 1.106797181058933, + "gemmi_hkl_families": [ + [ + 3, + 1, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 11.073631912154664, + "pymatgen_hkl_families": [ + [ + 3, + 1, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 11.188548775075896, + "two_theta_deg_gemmi": 88.2094694370667, + "two_theta_deg_pymatgen": 88.20946943706669 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.02717140666040052, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 1.0552897060221729, + "d_A_pymatgen": 1.0552897060221729, + "gemmi_hkl_families": [ + [ + 3, + 1, + 1 + ] + ], + "gemmi_relative_intensity_max_100": 9.827681516868578, + "pymatgen_hkl_families": [ + [ + 3, + 1, + 1 + ] + ], + "pymatgen_relative_intensity_max_100": 9.854852923528979, + "two_theta_deg_gemmi": 93.76301960107875, + "two_theta_deg_pymatgen": 93.76301960107875 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.019938155420592185, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 1.0103629710818451, + "d_A_pymatgen": 1.0103629710818451, + "gemmi_hkl_families": [ + [ + 2, + 2, + 2 + ] + ], + "gemmi_relative_intensity_max_100": 3.007147570433699, + "pymatgen_hkl_families": [ + [ + 2, + 2, + 2 + ] + ], + "pymatgen_relative_intensity_max_100": 2.987209415013107, + "two_theta_deg_gemmi": 99.35234392983573, + "two_theta_deg_pymatgen": 99.35234392983573 + }, + { + "abs_delta_d_A": 0.0, + "abs_delta_relative_intensity_percentage_points": 0.14393761869152044, + "abs_delta_two_theta_deg": 0.0, + "d_A_gemmi": 0.9707253433941511, + "d_A_pymatgen": 0.9707253433941511, + "gemmi_hkl_families": [ + [ + 3, + 2, + 0 + ] + ], + "gemmi_relative_intensity_max_100": 8.555197994407527, + "pymatgen_hkl_families": [ + [ + 3, + 2, + 0 + ] + ], + "pymatgen_relative_intensity_max_100": 8.411260375716006, + "two_theta_deg_gemmi": 105.03332196120854, + "two_theta_deg_pymatgen": 105.03332196120854 + }, + { + "abs_delta_d_A": 2.220446049250313e-16, + "abs_delta_relative_intensity_percentage_points": 0.4517144092258931, + "abs_delta_two_theta_deg": 4.263256414560601e-14, + "d_A_gemmi": 0.9354143466934853, + "d_A_pymatgen": 0.9354143466934856, + "gemmi_hkl_families": [ + [ + 3, + 2, + 1 + ] + ], + "gemmi_relative_intensity_max_100": 16.743824651662965, + "pymatgen_hkl_families": [ + [ + 3, + 2, + 1 + ] + ], + "pymatgen_relative_intensity_max_100": 16.292110242437072, + "two_theta_deg_gemmi": 110.8704539803576, + "two_theta_deg_pymatgen": 110.87045398035755 + } + ] + }, + "matched_d_spacing_group_count": 13, + "position_max_abs_delta_d_A": 2.220446049250313e-16, + "position_max_abs_delta_two_theta_deg": 4.263256414560601e-14, + "position_passed": true, + "pymatgen_d_spacing_group_count": 13, + "pymatgen_parse_warnings": [ + "No _symmetry_equiv_pos_as_xyz type key found. Spacegroup from _space_group_name_H-M_alt used.", + "Issues encountered while parsing CIF: No _symmetry_equiv_pos_as_xyz type key found. Spacegroup from _space_group_name_H-M_alt used." + ], + "pymatgen_pattern_peak_count": 13, + "pymatgen_structure_site_count": 1, + "selection_rules": { + "expected_allowed_families": [ + [ + 1, + 0, + 0 + ], + [ + 1, + 1, + 0 + ], + [ + 1, + 1, + 1 + ] + ], + "expected_forbidden_families": [], + "gemmi_allowed_missing": [], + "gemmi_forbidden_present": [], + "gemmi_rule_violations": [], + "passed": true, + "pymatgen_allowed_missing": [], + "pymatgen_forbidden_present": [], + "pymatgen_rule_violations": [] + }, + "unmatched_gemmi_group_indices": [], + "unmatched_pymatgen_group_indices": [] + } + }, + "elasticity": { + "comparison_tolerances": { + "absolute_GPa": 1e-08, + "acceptance": "abs(delta) <= max(absolute_GPa, relative * abs(pymatgen))", + "relative": 1e-10 + }, + "directions": [ + { + "abs_delta_GPa": 0.0, + "gemmi_project_modulus_GPa": 137.49999999999997, + "hkl_normal": [ + 1, + 0, + 0 + ], + "passed": true, + "pymatgen_modulus_GPa": 137.49999999999997 + }, + { + "abs_delta_GPa": 5.684341886080802e-14, + "gemmi_project_modulus_GPa": 209.52380952380963, + "hkl_normal": [ + 1, + 1, + 0 + ], + "passed": true, + "pymatgen_modulus_GPa": 209.52380952380958 + }, + { + "abs_delta_GPa": 5.684341886080802e-14, + "gemmi_project_modulus_GPa": 253.84615384615367, + "hkl_normal": [ + 1, + 1, + 1 + ], + "passed": true, + "pymatgen_modulus_GPa": 253.84615384615373 + } + ], + "input_cif": "src/diffractscout/benchmark_data/fcc_al.cif", + "input_sha256": "c82f082d2c1ed45c2219713cf972ffce67a7df78d00eb8b953541363c8f0ffdf", + "passed": true, + "shear_convention": "engineering shear strain", + "synthetic_stiffness_GPa": [ + [ + 250.0, + 150.0, + 150.0, + 0.0, + 0.0, + 0.0 + ], + [ + 150.0, + 250.0, + 150.0, + 0.0, + 0.0, + 0.0 + ], + [ + 150.0, + 150.0, + 250.0, + 0.0, + 0.0, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 100.0, + 0.0, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 100.0, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 100.0 + ] + ], + "voigt_order": [ + "11", + "22", + "33", + "23", + "13", + "12" + ] + }, + "expectations": { + "path": "src/diffractscout/benchmark_data/expectations.json", + "sha256": "7ea211d475382584984748514cc3218b0f6f707f2e81a35b0f1dee98a9780ce0" + }, + "generated_at_utc": "2026-09-26T12:37:13+00:00", + "output_files": [ + "validation_cases/independent_engines/reference_engine_comparison.json", + "validation_cases/independent_engines/reference_engine_comparison.md" + ], + "schema": "diffractscout_independent_reference_engine_comparison_v1", + "settings": { + "d_spacing_merge_and_match_tolerance_A": 1e-06, + "intensity_comparison": "unscaled calculated peaks summed at coincident d then normalized per case to maximum 100; diagnostic only", + "radiation": "Cu Kα effective single wavelength", + "two_theta_acceptance_tolerance_deg": 1e-05, + "two_theta_range_deg": [ + 5.0, + 120.0 + ], + "wavelength_A": 1.5406 + }, + "software": { + "comparison_script_sha256": "e8a8bd9e7742d120347ad543faaa278c0e33b9fb1556e9e2cb0b63fbf2e985b3", + "diffractscout": "0.4.0", + "gemmi": "0.7.5", + "pymatgen": "2026.5.4", + "python": "3.12.10" + }, + "synthetic_inputs_only": true +} diff --git a/validation_cases/independent_engines/reference_engine_comparison.md b/validation_cases/independent_engines/reference_engine_comparison.md new file mode 100644 index 0000000..d450440 --- /dev/null +++ b/validation_cases/independent_engines/reference_engine_comparison.md @@ -0,0 +1,49 @@ +# Independent pymatgen reference comparison + +Generated: `2026-09-26T12:37:13+00:00` + +Overall acceptance: **PASS** + +This receipt records independent software comparisons on explicitly synthetic benchmark fixtures. It is not experimental validation, a real-material property claim, or external-user evidence. + +## Software and settings + +- Python: `3.12.10` +- Comparison script SHA-256: `e8a8bd9e7742d120347ad543faaa278c0e33b9fb1556e9e2cb0b63fbf2e985b3` +- Gemmi: `0.7.5` +- pymatgen: `2026.5.4` +- Radiation: Cu Kα effective wavelength `1.5406 Å` +- 2θ range: `5.0–120.0°` +- Peak grouping/matching: `|Δd| ≤ 1e-06 Å`; geometry acceptance also requires `|Δ2θ| ≤ 1e-05°`. +- Diffraction intensities use each engine's unscaled theoretical pattern, are grouped at coincident d, and are separately normalized to a per-case maximum of 100; differences are diagnostic only. + +## Diffraction peak positions and selection rules + +| Synthetic CIF | Gemmi families | pymatgen groups | Matched d groups | max abs Δd (Å) | max abs Δ2θ (°) | position | selection | max intensity difference (points) | +|---|---:|---:|---:|---:|---:|:---:|:---:|---:| +| `src/diffractscout/benchmark_data/simple_cubic_al.cif` (simple_cubic_al) | 14 | 13 | 13 | 2.22e-16 | 4.26e-14 | PASS | PASS | 0.4517 | +| `src/diffractscout/benchmark_data/bcc_fe.cif` (bcc_fe) | 5 | 5 | 5 | 3.33e-16 | 7.11e-14 | PASS | PASS | 0.4657 | +| `src/diffractscout/benchmark_data/fcc_al.cif` (fcc_al) | 8 | 8 | 8 | 2.22e-16 | 4.26e-14 | PASS | PASS | 0.7899 | +| `src/diffractscout/benchmark_data/nacl.cif` (nacl) | 16 | 14 | 14 | 1.33e-15 | 7.11e-14 | PASS | PASS | 0.5716 | + +The comparison groups lines by d-spacing rather than requiring identical representative hkl labels. Pymatgen may return a single pattern peak containing several hkl families where DiffractScout returns separate family rows; the comparison therefore sums coincident-family intensities before per-case normalization. + +## Cubic directional Young's modulus + +The same synthetic stiffness matrix in GPa is interpreted with Voigt order `['11', '22', '33', '23', '13', '12']` and engineering shear strain. Acceptance is `|ΔE| ≤ max(1e-08 GPa, 1e-10 × |E_pymatgen|)`. + +| Plane normal | DiffractScout (GPa) | pymatgen (GPa) | abs ΔE (GPa) | Result | +|---|---:|---:|---:|:---:| +| `[1, 0, 0]` | 137.5 | 137.5 | 0 | PASS | +| `[1, 1, 0]` | 209.523809524 | 209.523809524 | 5.68e-14 | PASS | +| `[1, 1, 1]` | 253.846153846 | 253.846153846 | 5.68e-14 | PASS | + +## Reproduction + +Install the optional Materials Project dependencies, then run from the repository root: + +```sh +python scripts/compare_reference_engines.py --output validation_cases/independent_engines +``` + +The tracked JSON receipt records relative CIF paths and SHA-256 hashes, software versions, thresholds, per-peak results, selection-rule outcomes, and tensor results. The script exits nonzero if peak geometry/selection or elastic acceptance checks fail. Intensity differences do not affect the exit status.