Skip to content

Resolve hardcoded relics and interactive config field preservation issues - #430

Merged
boromir674 merged 36 commits into
releasefrom
dev
Nov 2, 2025
Merged

boromir674 merged 36 commits into
releasefrom
dev

Conversation

@boromir674

@boromir674 boromir674 commented Oct 26, 2025 •

Copy link
Copy Markdown
Owner

Overview

This PR delivers critical bug fixes and a major new observability feature, transforming the cookiecutter Python package generator into an enterprise-ready platform.

🐛 Interactive Config Field Preservation (Critical Bug Fix)

Why: Config file values for derived fields (project_slug, pkg_name, etc.) were completely ignored in interactive mode, forcing users to manually re-enter values they had already specified in YAML config files.

What: The root cause was hardcoded field subsets in the backend helpers and lambda-first dialog defaults that prevented user configuration from being preserved during Jinja2 template rendering.

How: Implemented dynamic field resolution that preserves user config values, modified dialog defaults to check config values before falling back to computation, and prevented Jinja2 re-rendering from overwriting explicit user configuration.

📊 Observability Stack Integration (NEW MAJOR FEATURE)

Why: Modern Python applications require production-grade monitoring and logging capabilities. Users previously had to manually set up complex observability stacks.

What: Complete observability solution with Grafana (visualization), Loki (log aggregation), and Promtail (log collection) integrated via Docker Compose orchestration.

How: Added include_observability parameter to cookiecutter.json that conditionally generates:

  • Pre-configured Docker Compose stack with project-specific container names
  • Grafana dashboards and datasource configurations pointing to Loki
  • Loki configuration optimized for Python application logs
  • Promtail configuration for automatic log collection
  • Comprehensive setup documentation with deployment instructions

🚀 CI/CD Pipeline Enhancements

Why: The existing test pipeline suffered from race conditions between test categories, making builds unreliable and difficult to debug.

What: Enhanced PR validation pipeline with isolated test execution, comprehensive logging, and race condition prevention.

How:

  • Separated flaky test categories into isolated execution streams
  • Added structured logging with automatic artifact collection
  • Implemented cross-worker conflict prevention in pytest-xdist
  • Enhanced build backend testing with extensive environment debugging

🔧 Template Variable Replacement

Why: Hardcoded values throughout template files created inconsistencies and maintenance issues.

What: Systematic replacement of hardcoded relics with proper template variables.

How: Identified and replaced hardcoded values across multiple template files with appropriate cookiecutter template variables, ensuring consistency in generated projects.

🎯 Impact

User Experience: Config file values now display correctly in interactive mode, eliminating the frustrating need to re-enter pre-configured values. Users can now generate projects with enterprise-grade monitoring capabilities out-of-the-box.

Developer Experience: CI pipeline is significantly more reliable with race condition prevention and detailed debugging capabilities. The observability stack provides immediate insights into application behavior in development and production.

Production Readiness: Generated projects now include production-ready monitoring infrastructure that scales with the application, supporting enterprise deployment scenarios.

🔄 Backward Compatibility

Zero breaking changes. All existing functionality is preserved. The observability feature is completely opt-in via the include_observability parameter - existing projects continue to work without modification.

- Fix hardcoded field subsets in helpers.py that ignored derived fields from config files
- Modify dialog defaults in project_name.py to respect config values before falling back to lambda computation
- Prevent Jinja2 re-rendering from overwriting explicit user config values
- Add comprehensive test suite for interactive config bug scenarios
- Ensure derived fields like project_slug, pkg_name show config file values in interactive CLI
- Maintain backward compatibility with dynamic field computation when values not provided

Resolves issue where --config-file values were ignored for fields that come after project_name in cookiecutter.json, causing interactive mode to show computed values instead of user's explicit configuration.
Fixes 2 issues: replaces hardcoded relic ocde with templated cookiecutter variables and preserves the User config variable in Interactive mode
@codecov

codecov Bot commented Oct 26, 2025 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.68%. Comparing base (8dfe749) to head (1a0083e).
⚠️ Report is 52 commits behind head on release.

Additional details and impacted files
@@             Coverage Diff             @@
##           release     #430      +/-   ##
===========================================
+ Coverage    92.54%   93.68%   +1.14%     
===========================================
  Files           53       53              
  Lines          899      919      +20     
===========================================
+ Hits           832      861      +29     
+ Misses          67       58       -9     
Flag Coverage Δ
unittests 93.68% <100.00%> (+1.14%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- Capture all pytest output using tee for debugging CI failures
- Upload combined test logs with clear headers for each invocation
- Include via_build_module, test_build_creates_artifacts, and full suite logs
- Enable artifact download with gh cli for easier debugging
Prevents CI test failures on pytest-xdist worker gw1 by defensively
excluding .tox directories in test snapshots from Poetry's file
discovery process. These directories can contain broken symlinks or
missing paths that cause FileNotFoundError during parallel test
execution.

Resolves worker isolation race condition without modifying test
fixture scopes, providing minimal defensive mitigation.
- Run build_backend_sdist tests in separate invocation to isolate Poetry ValueError
- Keep lint tests isolated to prevent race condition
- Add dedicated logs for easier debugging of specific test failures
- Add .ruff_cache exclusion patterns to pyproject.toml
- Prevents ruff cache files created by lint tests from being included in sdist
- Resolves test failure: 'extra runtime files compared to expectations'
- Add .ruff_cache cleanup to both test_running_lint_passes and test_running_ruff_passes
- Prevents ruff cache files from persisting in snapshot directories
- Resolves sdist build test failure caused by unexpected cache files
- Cache files created by ruff during test execution should not remain in snapshots
Comment thread tests/test_build_backend_sdist.py Fixed
Comment thread tests/test_build_backend_sdist.py Fixed
Comment thread tests/test_build_backend_sdist.py Fixed
Comment thread tests/test_build_backend_sdist.py Fixed
Comment thread tests/test_build_backend_sdist.py Fixed
Comment thread tests/test_build_backend_sdist.py Fixed
Comment thread tests/test_build_backend_sdist.py Fixed
Comment thread tests/test_build_backend_sdist.py Fixed
Comment thread tests/test_build_backend_sdist.py Fixed
Comment thread tests/test_build_backend_sdist.py Fixed
# Python packages information
print("\nPYTHON PACKAGES INFO:")
try:
import subprocess

Check notice

Code scanning / Bandit

Consider possible security implications associated with the subprocess module. Note test

Consider possible security implications associated with the subprocess module.
try:
import subprocess

pip_list_result = subprocess.run(

Check notice

Code scanning / Bandit

subprocess call - check for execution of untrusted input. Note test

subprocess call - check for execution of untrusted input.
# Git repository information (important for setuptools-scm)
print("\nGIT REPOSITORY INFO:")
try:
import subprocess

Check notice

Code scanning / Bandit

Consider possible security implications associated with the subprocess module. Note test

Consider possible security implications associated with the subprocess module.
import subprocess

# Check if git is available
git_version_result = subprocess.run(

Check notice

Code scanning / Bandit

Starting a process with a partial executable path Note test

Starting a process with a partial executable path
import subprocess

# Check if git is available
git_version_result = subprocess.run(

Check notice

Code scanning / Bandit

subprocess call - check for execution of untrusted input. Note test

subprocess call - check for execution of untrusted input.
print(f" Git status failed: {git_status_result.stderr}")

# Check current branch/commit
git_branch_result = subprocess.run(

Check notice

Code scanning / Bandit

subprocess call - check for execution of untrusted input. Note test

subprocess call - check for execution of untrusted input.
if git_branch_result.returncode == 0:
print(f" Current branch: {git_branch_result.stdout.strip()}")

git_commit_result = subprocess.run(

Check notice

Code scanning / Bandit

Starting a process with a partial executable path Note test

Starting a process with a partial executable path
if git_branch_result.returncode == 0:
print(f" Current branch: {git_branch_result.stdout.strip()}")

git_commit_result = subprocess.run(

Check notice

Code scanning / Bandit

subprocess call - check for execution of untrusted input. Note test

subprocess call - check for execution of untrusted input.
print(f" Current commit: {git_commit_result.stdout.strip()}")

# Check if we're in a git repository
git_root_result = subprocess.run(

Check notice

Code scanning / Bandit

Starting a process with a partial executable path Note test

Starting a process with a partial executable path
print(f" Current commit: {git_commit_result.stdout.strip()}")

# Check if we're in a git repository
git_root_result = subprocess.run(

Check notice

Code scanning / Bandit

subprocess call - check for execution of untrusted input. Note test

subprocess call - check for execution of untrusted input.
@boromir674
boromir674 merged commit 70afde2 into release Nov 2, 2025
16 of 17 checks passed

This branch was previously deployed

1 inactive deployment
TEST_DEPLOYMENT — 1a0083e0 Deployed Nov 2, 2025 by boromir674 via pypi_publish / pypi_env_publish #108
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants