本書は、CSVの構造確認、規則候補の生成、整形、統制例の実行に使用するコマンドラインを定義します。引数、標準出力、生成ファイル、処理を継続・中止する条件、終了コードを確認できます。
完全なコマンド例と動作契約は以下の英語本文を参照してください。
This document defines the complete command-line workflows and exit semantics. Start with the minimal example in the README, then use the reference below for inspection, schema suggestion, cleaning, and controlled examples.
data-cleaning-toolkit inspect INPUT.csv [--output REPORT.json]
When --output is omitted, the JSON report is written to standard output.
Inspection is structural: it does not infer whether a column is numeric, a
date, an identifier, or a category.
The report contains:
- Input row and column counts
- Empty and non-empty cell counts per column
- Distinct non-empty value counts per column
- Exact duplicate row count
- Rows whose field count differs from the header
data-cleaning-toolkit suggest-schema INPUT.csv \
--output SUGGESTION.json \
[--date-format FORMAT]
The command ignores empty cells while measuring parse coverage. A non-string
type is suggested only when every non-empty value matches that type. Candidate
precedence is boolean, integer, decimal, and date; otherwise the column remains
a string. Textual forms such as true and false may be suggested as boolean,
while 0 and 1 remain eligible for integer inference. Integer-shaped values
with leading zeros are kept as strings to avoid destructive normalization of
identifiers.
The JSON output includes a ready-to-review suggested_schema object plus
per-column counts, whitespace observations, leading-zero counts, and parse
coverage. Copy or extract the candidate only after reviewing it against the
dataset's meaning and downstream requirements.
data-cleaning-toolkit clean INPUT.csv \
--schema SCHEMA.json \
--output CLEAN.csv \
[--report AUDIT.json]
If --report is omitted, clean.csv produces clean.report.json. The input
CSV and schema are protected sources: neither can also be used as the clean
CSV or report path.
Inspect a CSV before defining cleaning rules:
data-cleaning-toolkit inspect examples/demo_dirty.csv \
--output output/inspection.jsonGenerate a reviewable schema candidate with per-column evidence:
data-cleaning-toolkit suggest-schema examples/schema_suggestion_demo.csv \
--output output/schema_suggestion.jsonRun the controlled explicit-value-mapping example:
data-cleaning-toolkit clean examples/value_mapping_demo.csv \
--schema examples/value_mapping_schema.json \
--output output/value_mapping_clean.csv \
--report output/value_mapping_report.jsonRun the controlled cross-column validation example:
data-cleaning-toolkit clean examples/cross_column_demo.csv \
--schema examples/cross_column_schema.json \
--output output/cross_column_clean.csv \
--report output/cross_column_report.jsonRun the controlled conditional-presence example:
data-cleaning-toolkit clean examples/conditional_presence_demo.csv \
--schema examples/conditional_presence_schema.json \
--output output/conditional_presence_clean.csv \
--report output/conditional_presence_report.jsonRun the controlled mapping-coverage example:
data-cleaning-toolkit clean examples/mapping_coverage_demo.csv \
--schema examples/mapping_coverage_schema.json \
--output output/mapping_coverage_clean.csv \
--report output/mapping_coverage_report.jsonRun the controlled unmatched-frequency example:
data-cleaning-toolkit clean examples/unmatched_frequency_demo.csv \
--schema examples/unmatched_frequency_schema.json \
--output output/unmatched_frequency_clean.csv \
--report output/unmatched_frequency_report.jsonCompare raw, redacted, and disabled frequency modes:
data-cleaning-toolkit clean examples/privacy_modes_demo.csv \
--schema examples/privacy_modes_schema.json \
--output output/privacy_modes_clean.csv \
--report output/privacy_modes_report.jsonThe dirty-data demo is intentionally invalid, so inspect and clean write
their outputs and return exit code 1. The value-mapping example also returns 1
because its final row contains deliberately unmapped values. The cross-column
example returns 1 because it includes deliberate relationship violations. The
conditional-presence example returns 1 because it includes deliberately
missing dependent values. The mapping-coverage example returns 1 because it
includes one deliberately unknown value. The unmatched-frequency example
returns 1 because it includes four deliberately unknown values. The controlled
privacy-mode and suggest-schema examples return 0. These semantics make the
CLI usable as a data-quality gate in scripts and CI.
Regenerate the committed reference artifacts with:
python examples/run_demo.pyRun the tests with:
python -m pytestRun the public Python API example:
python examples/public_api_demo.pyVerify the committed reference artifacts without regenerating them:
python examples/run_demo.py --verify-only| Code | Meaning |
|---|---|
0 |
Command completed without validation errors |
1 |
Output was produced, but invalid or malformed rows were detected |
2 |
Input, schema, configuration, or output path prevented normal processing |
Duplicate-key removal is reported as a warning and does not by itself cause exit code 1.