A small Python command-line utility for applying simple, repeatable cleanup rules to CSV files.
Independent demonstration project · Python 3.10+ · Standard library only · MIT license
- Trims leading and trailing whitespace from each cell.
- Skips fully blank rows.
- Optionally removes duplicate rows with
--dedupeafter whitespace is trimmed. - Reads UTF-8 files with an optional UTF-8 BOM (
utf-8-sig) and writes UTF-8 output. - Keeps the source intact: input and output paths must differ.
- Refuses to overwrite an existing output file.
- Python 3.10 or later
- No third-party package installation required
Show command options:
python main.py --helpClean a file without deduplication:
python main.py input.csv cleaned.csvTrim fields, skip blank rows, and remove duplicate rows:
python main.py input.csv cleaned.csv --dedupeChoose a new output filename each time. The tool intentionally stops if the output file already exists.
Input:
name, value
Alice , 10
Alice,10Command:
python main.py input.csv cleaned.csv --dedupeOutput:
name,value
Alice,10Run the unit tests from the repository root:
python -m unittest discover -s tests -vThe current tests cover trimming/blank-row handling with deduplication, rejecting an existing output file, and rejecting identical input/output paths.
This is a deliberately small CSV utility. It does not infer data types, standardize date formats, repair malformed rows, or decide whether two different-looking values represent the same entity. Deduplication compares the complete row after whitespace trimming. Review output before using it in a business workflow.
Use synthetic examples for public demonstrations. Do not commit credentials, private data, or customer files. The output is a convenience tool and should be validated against your actual requirements before production use.
MIT. See LICENSE.