Repository navigation
release(v0.1.0-rc.2): framework stabilization, soft-delete, and lookup - #126
Merged
Merged
Conversation
- Added _format_error_location to convert Pydantic loc tuples into dot-separated paths - Added _sanitize_error_item to safely extract field, message, type, and context from Pydantic errors - Updated app_exception_handler return type to ZCoreJSONResponse - Renamed log field from type to error_type for consistency - Added docstrings to ZCoreJSONResponse and app_exception_handler
…rror details - Added request_validation_exception_handler for FastAPI/Pydantic 422 errors - Uses _sanitize_error_item to format field-level validation errors - Returns structured 422 response with summary message and errors list - Logs validation failures with path, method, and error count - Changed Sequence import to collections.abc
- Added http_exception_handler for Starlette/FastAPI HTTPException - Returns structured ResponseWrapper with success=False and error metadata - Preserves original HTTP headers from the exception - Logs HTTP errors with status code, message, path, and method
…d details - Added response_validation_exception_handler for internal response schema mismatches - Returns 500 error with sanitized Pydantic errors only in DEBUG mode - Logs detailed errors in DEBUG; hides internal details in production - Added settings import for DEBUG flag access
…tion - Added unhandled_exception_handler for uncaught runtime exceptions with debug-gated messages - Added register_exception_handlers to attach all handlers with selective inclusion flags - Registered handlers for AppException, RequestValidationError, StarletteHTTPException, ResponseValidationError, and generic Exception - Updated module docstring to reflect unified handler registration
- Exported ZCoreJSONResponse, http_exception_handler, register_exception_handlers - Exported request_validation_exception_handler, response_validation_exception_handler, unhandled_exception_handler - Generalized __getattr__ to lazily resolve all handler names from handlers module - Updated TYPE_CHECKING imports and __all__ list
- Added all exception handlers to TYPE_CHECKING imports and __all__ - Added lazy resolution in __getattr__ for handler names via handlers module - Exported app_exception_handler, http_exception_handler, register_exception_handlers - Exported request_validation_exception_handler, response_validation_exception_handler, unhandled_exception_handler
- Replaced manual app.add_exception_handler with register_exception_handlers(app) - Removed direct import of AppException and app_exception_handler - Bumped version from 0.1.0-rc.1 to 0.1.0-rc.2
- Added tests for request_validation_exception_handler with body and query params - Added test for http_exception_handler preserving custom headers - Added parametrized tests for response_validation_exception_handler (debug on/off) - Added parametrized tests for unhandled_exception_handler (debug on/off) - Added test for register_exception_handlers with all handlers enabled - Added test for selective handler registration - Updated logging integration test to cover both AppException and unhandled exceptions - Added ASGITransport raise_app_exceptions=False to prevent test crashes - Added DuplicateEntity and ForbiddenError parametrized cases for app_exception_handler - Updated strict response wrapper compliance test to validate all handler types
…_all__ - Moved exception handler entries to maintain alphabetical ordering - No functional changes, purely organizational cleanup
- Added _supports_soft_delete() helper to detect SoftDeleteMixin on models - Updated delete() to apply soft deletion when supported, with force flag for hard delete - Updated delete_multi() with atomic soft-delete via UPDATE, dialect-aware RETURNING support, and fallback fetch - Added now() import for consistent soft-delete timestamps - Updated docstrings to reflect soft-delete capabilities
- Added restore() to WriteRepositoryMixin for recovering soft-deleted entities - Supports model instance or primary key lookup - Returns None if model does not support soft delete - Calls record.restore() and refreshes after flush
…eted records - Added restore_multi() to WriteRepositoryMixin for batch soft-delete recovery - Uses atomic UPDATE with deleted_at IS NOT NULL filter - Supports dialect-aware UPDATE RETURNING with fallback fetch - Returns empty list if model doesn't support soft delete or ids is empty
…lete methods - Added pre_restore, post_restore, pre_restore_multi, post_restore_multi lifecycle hooks - Added restore() and restore_multi() orchestration methods - Added on_restore and on_restore_multi core execution methods - Added force parameter to pre_delete, post_delete, pre_delete_multi, post_delete_multi hooks - Propagated force flag through delete() and delete_multi() to repository - Simplified docstrings by removing redundant parameter documentation
- Added force: bool = False parameter to _delete_endpoint wrapper - Propagated force flag to delete_endpoint and service.delete - Added force to delete_endpoint docstring - Enables permanent hard deletion via ?force=true query parameter
- Added RegularTestModel and RegularRepository for non-soft-delete model testing - Added SoftDeleteHookService tracking all delete/restore hooks with force flag - Updated existing tests to use repo.delete() instead of direct soft_delete() - Added test for single soft-delete with restore - Added test for force hard-delete single and multi - Added test for restore_multi with selective restoration - Added test for non-supported model soft-delete behavior - Added test for service orchestration with hooks and force flag propagation - Added test for service multi-delete and multi-restore orchestration
- Updated pre_delete, post_delete, pre_delete_multi, post_delete_multi hooks to track force flag - Added pre_restore, post_restore, pre_restore_multi, post_restore_multi hooks with name mutations - Updated delete tests to assert force-specific hook names - Added test_service_restore_hooks and test_service_restore_multi_hooks - Added test_service_restore_not_found raising EntityNotFound - Updated delete_entity_not_found to assert force-specific hook absence
- Updated MockFullService.delete to accept and track force flag with deleted_ids list - Added assertions verifying force=true query parameter is passed to service.delete - Added raise_app_exceptions=False to all ASGITransport instances for reliable error handling - Cleaned up unused items list in test_repo_cursor_pagination - Added trailing commas to parametrize decorators and dict literals - Fixed quote style consistency in test_repo_search_security_violation
…oft delete queries - Replaced getattr(self.model, 'deleted_at') with self.model.deleted_at for clarity - Removed unused deleted_hard variable assignment in soft delete test
- Added inverted operators: not_like, not_ilike, not_contains, not_startswith, not_endswith, not_in, not_between, is_not_null - Added 'not' logical grouping operator with items or single field support - Refactored _compare_column to detect not_ prefix and wrap expressions in not_() - Updated is_null semantics to accept True/None as null indicator - Updated _validate_filters_recursive to handle not group validation - Added not_ import from sqlalchemy - Updated FilterItem docstring and Literal operator list
- Merged scattered operator tests into parametrized test_search_all_operators - Added tests for inverted operators: not_like, not_ilike, not_contains, not_startswith, not_endswith, not_in, not_between, is_not_null - Added tests for ge, le, lt operators in parametrized test - Added test_search_logical_not_group_operator for not group and not field - Added not_between assertion in datetime between test - Removed redundant individual operator tests (startswith, endswith, contains, lt_le_ge, in, is_null) - Added trailing commas and formatting cleanup throughout
…sthrough - Added _parse_env_file helper for safe .env key-value parsing - Refactored run_server to accept app, host, port, reload, workers, log_level, env_file, and extra_args - Implemented config precedence: CLI arg > .env file > defaults - Added support for APP_MODULE, UVICORN_APP, RELOAD, WORKERS, and LOG_LEVEL from .env - Automatically disables reload when workers > 1 - Forwarded extra_args directly to Uvicorn for full CLI compatibility - Added CalledProcessError handling with exit code reporting - Reformatted gen_secret and gen_env with cleaner line wrapping
- Reformatted DB_DRIVERS dict entries with multiline format - Broke long subprocess.run calls into multi-line format for readability - Reformatted console.print statements with proper indentation and line breaks - Added Any import for future type annotations - No functional changes, purely formatting cleanup
- Added CLI arguments for run: app, --host, --port, --reload, --workers, --log-level, --env-file - Changed parser to parse_known_args to forward extra args to Uvicorn - Propagated all run options to run_server - Bumped CLI version to 0.1.0-rc.2 - Updated run subcommand help text
- Renamed test_cli_run_with_uvicorn_mock to test_cli_run_default_cascading with precise assertions - Added test for CLI args overriding .env values (host, port, no-reload, log-level) - Added test for --workers disabling --reload - Added test for transparent forwarding of arbitrary Uvicorn args - Added test for custom app string and --env-file - Updated version assertion from 0.1.0-rc.1 to 0.1.0-rc.2
- Removed unused Any import from commands.py - Updated UI banner version to 0.1.0-rc.2 - Removed unused run_server import from test_cli
- Added LOOKUP field to Actions dataclass for minimal relational reference queries - Updated docstring to describe the new permission key
- Added fields parameter for selective load_only field loading - Added options parameter for SQLAlchemy executable options - Applied fields and options to base_query before search engine compilation - Updated docstring with new parameter descriptions
- Added fields and options parameters to on_search and search methods - Propagated fields and options to repository.search - Added docstring with parameter descriptions and return documentation
…ries - Added RouteKey.LOOKUP for lightweight relational reference queries - Added lookup_schema, allowed_lookup_fields, and max_lookup_size router attributes - Added _get_effective_lookup_fields to resolve whitelist from config or lookup schema - Added _validate_lookup_request to block unapproved filter/sort fields - Registered /lookup POST endpoint with schema_out response_model - Added lookup_endpoint handler with field projection via service.search(fields=...) - Enforces max_lookup_size cap on pagination size - Mapped LOOKUP action to model.actions().LOOKUP in get_route_action - Updated module and class docstrings
…loading to search (#117) ## Description This PR introduces a dedicated, lightweight `LOOKUP` endpoint designed for high-performance relational lookups, autocomplete selectors, and UI reference dropdowns. It also adds selective column projection (`load_only`) and custom execution options to the underlying search architecture across the Repository and Service layers. ### Key Changes: - **Security & Action Permissions (`zcore.db.setup`)**: - Added `LOOKUP` action to the `Actions` dataclass for fine-grained RBAC/scope permission mapping (`model.actions().LOOKUP`). - **Repository Optimization (`zcore.db.repository`)**: - Updated `SearchRepositoryMixin.search()` to accept `fields` (`list[Any]`) and `options` (`list[ExecutableOption]`). - Applied selective column loading via SQLAlchemy's `load_only(*fields)` and appended executable query options prior to search query compilation. - **Service Layer Propagation (`zcore.service.base`)**: - Updated `search()` and `on_search()` methods to accept and forward `fields` and `options` to the repository layer while preserving `pre_search` and `post_search` lifecycle hooks. - **Declarative Router Scaffolding (`zcore.web.base_router`)**: - Added `RouteKey.LOOKUP` registered as a `POST /lookup` endpoint when `lookup_schema` is defined on `BaseRouter`. - Added router configuration attributes: `lookup_schema`, `allowed_lookup_fields`, and `max_lookup_size` (default: 100). - Added `_get_effective_lookup_fields()` to resolve allowed column whitelists either from explicit definitions or introspected `lookup_schema` fields. - Added `_validate_lookup_request()` to reject any client filters or sort criteria targeting unapproved fields with a 400 `ValidationError`. - Enforced `max_lookup_size` caps on client pagination requests. - Updated route specificity sorting to guarantee static `/lookup` paths are evaluated before parameterized dynamic routes (e.g., `/{id}`). - **Test Suite Coverage**: - Added `test_repo_search_with_fields_and_options` validating SQL `load_only` projection in `tests/test_repository.py`. - Added `test_service_search_with_fields_and_options` confirming parameter forwarding and post-hook execution in `tests/test_service.py`. - Added `test_router_lookup_endpoint_execution` verifying successful field-projected responses and 400 validation rejections for unauthorized fields in `tests/test_web.py`. - Updated route specificity assertions to verify correct priority for `/lookup`. ## Type of change - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [x] Refactoring (architectural or code cleanup with no functional changes) - [ ] Documentation update ## How Has This Been Tested? Please describe the tests that you ran to verify your changes. - [x] Unit/Integration tests (`hatch test`) - Validated repository `search()` with `fields` and `options` parameters using SQLAlchemy `load_only`. - Verified service layer `search()` argument forwarding to the mock repository and post-search hook execution. - Verified `POST /lookup` endpoint returns filtered results conforming to `lookup_schema`. - Verified requests attempting to filter on unauthorized fields raise a 400 `ValidationError`. - Verified route specificity sorting positions `/lookup` ahead of dynamic `/{id:uuid}` endpoints. - [x] Static type check passing (`mypy src/`) ## Checklist: - [x] My code follows the style guidelines and architectural rules of this project - [x] I have performed a self-review of my own code - [x] My changes pass strict static type analysis (`mypy src/` generates no warnings) - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have updated the documentation in `zcore-docs` if applicable - [x] My changes generate no new runtime warnings - [x] I have added tests that prove my fix is effective or that my feature works
…n, cache, streams, and security (#118) ## Description This PR introduces framework-wide tunable configuration parameters in `Settings`, replacing previously hardcoded magic numbers across the pagination, caching, logging, search, streaming, and security subsystems with configurable, environment-driven defaults. ### Key Changes: - **Centralized Configuration Additions (`zcore.config`)**: - `PAGINATION_DEFAULT_SIZE` (default: 20) and `PAGINATION_MAX_SIZE` (default: 100). - `SEARCH_MAX_DEPTH` (default: 3) for relational search traversal limits. - `CACHE_LOCAL_MAXSIZE` (default: 1000), `CACHE_DEFAULT_TTL` (default: 3600), and `CACHE_EVICTION_INTERVAL` (default: 60). - `STREAM_QUEUE_MAXSIZE` (default: 100) for event stream subscriber queues. - `AUTH_CACHE_TTL` (default: 300) for user session/token caching. - `max_bytes` (10MB) and `backup_count` (5) added to `LoggingSettings` for configurable log file rotation. - **Dynamic Pagination & Search Bounding**: - Made `size` optional in `PageNumberParams`, `CursorParams`, and `SearchRequest`. - Added Pydantic `model_validator` across all parameter schemas to automatically clamp `size` between `1` and `PAGINATION_MAX_SIZE`, defaulting to `PAGINATION_DEFAULT_SIZE`. - Updated `SearchEngine` to resolve search depth hierarchy: `model.__max_search_depth__` -> `settings.SEARCH_MAX_DEPTH` -> `3`. - **Cache Layer Flexibility (`zcore.cache`)**: - Updated `BaseCache` and `TTLLRUCache` constructors to accept optional `local_maxsize` and `default_ttl`, falling back gracefully to `Settings`. - Updated `BaseCache.set()` to fall back to the instance's configured default TTL when none is explicitly provided. - Dynamic eviction sweep interval in `_ensure_eviction_task()` backed by `CACHE_EVICTION_INTERVAL`. - **Web Router & Stream Enhancements**: - Added `default_page_size` and `max_page_size` router attributes to `BaseRouter`. - Hierarchical resolution of limit sizes in `/search` and `/lookup` endpoints (`router attribute` -> `settings` -> `fallback`). - Fixed route action mapping for `PATCH` requests in `custom_route_handler` (mapped to `"update"` instead of `"patch"` for consistency with permission models). - Wired `StreamManager.subscribe()` to create bounded queues sized via `STREAM_QUEUE_MAXSIZE`. - **Security & Logging**: - `BaseAuth` resolves user cache TTL from `settings.AUTH_CACHE_TTL`. - Configured `RotatingFileHandler` with dynamic file size thresholds and backup counts from `LoggingSettings`. - **Test Suite Additions**: - Added dynamic setting binding tests for pagination parameter clamping, cache size/TTL resolution, stream queue capacity, and auth cache TTL. ## Type of change - [x] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [x] Refactoring (architectural or code cleanup with no functional changes) - [ ] Documentation update ## How Has This Been Tested? Please describe the tests that you ran to verify your changes. - [x] Unit/Integration tests (`hatch test`) - Validated dynamic size clamping and defaults for `PageNumberParams` and `CursorParams` (`test_dynamic_pagination_params_bounding`). - Verified `TTLLRUCache` and `BaseCache` instantiate with patched `CACHE_LOCAL_MAXSIZE` and `CACHE_DEFAULT_TTL` (`test_dynamic_cache_configuration_bindings`). - Verified `StreamManager` creates subscriber queues sized according to `STREAM_QUEUE_MAXSIZE` (`test_dynamic_stream_queue_maxsize`). - Verified `BaseAuth` resolves cache TTL according to `AUTH_CACHE_TTL` (`test_base_auth_dynamic_cache_ttl`). - Validated router PATCH action mapping and search/lookup size boundary enforcement. - [x] Static type check passing (`mypy src/`) ## Checklist: - [x] My code follows the style guidelines and architectural rules of this project - [x] I have performed a self-review of my own code - [x] My changes pass strict static type analysis (`mypy src/` generates no warnings) - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have updated the documentation in `zcore-docs` if applicable - [x] My changes generate no new runtime warnings - [x] I have added tests that prove my fix is effective or that my feature works
…hip loaders for LOOKUP endpoint (#119) ## Description This PR enhances the automated `/lookup` endpoint by adding intelligent, schema-driven projection resolution (`_resolve_lookup_projections`). Instead of loading entire database models or performing simple column lookups, `BaseRouter` now introspects the defined `lookup_schema` against the SQLAlchemy model metadata to selectively load only required model attributes while automatically configuring optimized relationship eager loaders. ### Key Changes: - **Schema-Driven Column & Relationship Projection (`_resolve_lookup_projections`)**: - Automatically identifies database columns declared in `lookup_schema` and ensures primary keys are included for entity tracking. - Resolves columns using model instrumented attributes via `getattr(self.model, ...)` rather than raw `Column` objects, ensuring full compatibility with SQLAlchemy's `load_only()`. - Automatically detects relationships declared in `lookup_schema` and appends optimal SQLAlchemy loader options: `selectinload` for collection relationships (`rel.uselist is True`) and `joinedload` for scalar relationships. - **Lookup Endpoint Integration**: - Replaced manual column filtering in `lookup_endpoint()` with `_resolve_lookup_projections()`, forwarding both resolved `fields` and `loader_options` directly to `service.search()`. - Promoted SQLAlchemy mapper utilities (`inspect`, `joinedload`, `selectinload`) to module level in `base_router.py`. - **Documentation**: - Fixed the CLI TUI banner version in `README.md` to reflect `0.1.0-rc.2`. - **Test Suite Additions (`tests/test_web.py`)**: - Added `test_router_lookup_projection_with_sqlalchemy_model` verifying that only columns present in `lookup_schema` (plus primary keys) are loaded, while omitted columns remain unloaded. - Added `test_router_lookup_projection_relationship_eager_loading` verifying automatic creation of relationship eager loaders. ## Type of change - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [x] Refactoring (architectural or code cleanup with no functional changes) - [x] Documentation update ## How Has This Been Tested? Please describe the tests that you ran to verify your changes. - [x] Unit/Integration tests (`hatch test`) - Validated selective column projection in `test_router_lookup_projection_with_sqlalchemy_model`. - Validated automatic generation of relationship eager loaders in `test_router_lookup_projection_relationship_eager_loading`. - Verified existing web router, search, and lookup endpoint test suites pass. - [x] Static type check passing (`mypy src/`) ## Checklist: - [x] My code follows the style guidelines and architectural rules of this project - [x] I have performed a self-review of my own code - [x] My changes pass strict static type analysis (`mypy src/` generates no warnings) - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have updated the documentation in `zcore-docs` if applicable - [x] My changes generate no new runtime warnings - [x] I have added tests that prove my fix is effective or that my feature works
…s across context, security, streams, and pagination (#120) ## Description This PR generalizes user identity types across the framework, removing rigid `uuid.UUID` constraints and allowing applications to seamlessly use arbitrary primary key types (such as auto-incrementing integers, external string identifiers like Auth0/Firebase IDs, or UUIDs) across the context store, security protocols, real-time streaming, test runners, and cursor pagination. ### Key Changes: - **Context Management (`zcore.context.context`)**: - Removed strict UUID parsing and validation from the `ZContext.user_id` getter and setter. - Updated `user_id` type annotation to `Any | None`. - Removed unused `uuid` import and streamlined docstrings. - **Security Protocols (`zcore.security.protocols`)**: - Generalized `UserProtocol.id` type from `uuid.UUID` to `Any` to accommodate integer, string, and UUID primary keys. - **Event Streaming (`zcore.web.streams`)**: - Generalized `StreamManager` method signatures (`subscribe`, `unsubscribe`, `subscription`, `publish`, `_local_publish`) to accept `user_id: Any`. - Changed `users_queues` dictionary mapping to accept `Any` key types, adding fallback string-comparison matching across mixed ID representations. - Removed rigid UUID parsing in `_listen_to_redis`, routing messages cleanly by channel user string. - **Cursor Pagination (`zcore.db.pagination`)**: - Added primary key Python type introspection via `pk_col.type.python_type`. - Added safe integer coercion for cursor `last_id` when the underlying primary key is `int`, preserving existing UUID handling. - **Testing Runner (`zcore.testing.client`)**: - Fixed default `user_id` resolution in `BaseZTest.run()` by replacing a falsy `or` check with an explicit `is not None` check, preventing valid falsy identifiers (such as integer `0` or custom identifiers) from being unexpectedly overwritten with a newly generated UUID. - **Repository Maintenance**: - Reorganized `.gitignore` into clear, structured sections (compiled files, environments, IDEs, logs, temp files). - **Test Suite Updates**: - Updated `tests/test_context.py` to assert successful assignment and retrieval of integers, strings, and UUIDs for `ctx.user_id`. ## Type of change - [x] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [x] Refactoring (architectural or code cleanup with no functional changes) - [ ] Documentation update ## How Has This Been Tested? Please describe the tests that you ran to verify your changes. - [x] Unit/Integration tests (`hatch test`) - Parametrized tests verifying `ctx.user_id` supports `str`, `int`, `uuid.UUID`, and `None` (`test_set_current_user_id_validation`). - Tested cursor pagination keyset decoding and type coercion for integer and UUID models. - Verified `StreamManager` routing and unsubscription with diverse identifier types. - Verified `BaseZTest` test client behavior with custom user identifiers. - [x] Static type check passing (`mypy src/`) ## Checklist: - [x] My code follows the style guidelines and architectural rules of this project - [x] I have performed a self-review of my own code - [x] My changes pass strict static type analysis (`mypy src/` generates no warnings) - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have updated the documentation in `zcore-docs` if applicable - [x] My changes generate no new runtime warnings - [x] I have added tests that prove my fix is effective or that my feature works
…ency overrides, event sandboxing, and database setup helper (#121) ## Description This PR significantly expands the ZCore testing subsystem (`zcore.testing`), introducing zero-boilerplate test database initialization, event subscriber sandboxing, custom engine support for transactional rollbacks, multi-dependency overrides (for arbitrary database sessions and auth stubs), and superuser identity simulation. It also updates framework documentation in `README.md` to reflect recent core capabilities. ### Key Changes: - **Test Database Lifecycle Helper (`setup_test_database`)**: - Added `setup_test_database()` to drop and recreate database tables across any test context. - Automatically detects whether an active event loop is running and executes table creation safely using a `ThreadPoolExecutor` or `asyncio.run()`. - **Event Dispatcher Sandboxing (`EventDispatcherSandbox`)**: - Added `EventDispatcherSandbox` fixture to snapshot and restore event handlers before and after test executions, preventing cross-test event subscriber accumulation and side-effect leakage. - Integrated into the default `ZTestClient` fixture pipeline. - **Custom Engine & Session Overrides (`DatabaseRollback`)**: - Enhanced `DatabaseRollback` to accept a custom `engine`, `db_dependency`, and `app`. - Automatically binds session dependency overrides to `app.dependency_overrides` during setup and cleans them up during teardown, enabling seamless testing of vanilla FastAPI apps alongside ZCore apps. - **Expanded `ZTestClient` & `BaseZTest` Capabilities**: - Added support for `engine`, `db_dependency`, `user_dependency`, and `user_model` parameters. - Supports overriding multiple authentication dependency stubs (passed as single dependencies or sequences). - Added `user_model` support using `user_model.model_validate()` for applications using strict Pydantic user schemas. - Added `is_superuser` support with direct context and mock user attribute bindings. - Fixed `GenericMockUser` attribute assignment and preserved polymorphic user identifiers (`user_id is not None`). - **Package Exports & Documentation**: - Exported `EventDispatcherSandbox` and `setup_test_database` from `zcore.testing`. - Added comprehensive docstrings across all testing fixtures and client classes. - Updated `README.md` to document the 8 auto-scaffolded endpoints (including `/lookup`), soft-delete recovery, inverted search operators/NOT groups, and cascading CLI options. ## Type of change - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [x] Refactoring (architectural or code cleanup with no functional changes) - [x] Documentation update ## How Has This Been Tested? Please describe the tests that you ran to verify your changes. - [x] Unit/Integration tests (`hatch test`) - Verified `setup_test_database` execution in both sync fixture lifecycles and active asyncio loops. - Verified `EventDispatcherSandbox` isolates event subscribers across test runs. - Verified `DatabaseRollback` works with custom engines and overrides custom `db_dependency` stubs. - Verified `ZTestClient` mocks user identity with custom Pydantic models, arbitrary user IDs, superuser flags, and multiple auth dependencies. - [x] Static type check passing (`mypy src/`) ## Checklist: - [x] My code follows the style guidelines and architectural rules of this project - [x] I have performed a self-review of my own code - [x] My changes pass strict static type analysis (`mypy src/` generates no warnings) - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have updated the documentation in `zcore-docs` if applicable - [x] My changes generate no new runtime warnings - [x] I have added tests that prove my fix is effective or that my feature works
…uery logging, and isolate background scope context (#122) ## Description This PR refines the framework's logging subsystem to significantly reduce log noise, eliminate redundant access logging during development, introduce status-code-aware logging for exception handlers, support standalone slow-query tracking, and ensure structlog context isolation across background task scopes. ### Key Changes: - **SQL Query & Slow-Query Logging (`zcore.config`, `zcore.db.setup`)**: - Changed `LoggingSettings.log_sql_queries` default to `False` to prevent unintentional SQL dumping in standard environments. - Added flat `LOG_SQL_QUERIES: bool | None = None` setting in `Settings` with bidirectional synchronization against `LOGGING.log_sql_queries`. - Decoupled slow-query threshold logging: queries exceeding `slow_query_threshold_ms` are logged even when `log_sql_queries` is disabled, while fast queries are cleanly skipped when a threshold is defined. - **Framework Lifecycle Log Level Downgrades**: - Downgraded startup and teardown lifecycle logs from `INFO` to `DEBUG` across `DatabaseManager` (initialization and connection close), `BaseCache` (Redis initialization, cache eviction loop stop, and cache close), and `StreamManager` (PubSub connection release). - **Status-Code-Aware Exception Logging (`zcore.exceptions.handlers`)**: - Updated `app_exception_handler` and `http_exception_handler` to log at `DEBUG` for client errors (HTTP < 500) and `ERROR` for internal server failures (HTTP >= 500). - Updated `request_validation_exception_handler` to log 422 errors at `DEBUG` instead of `WARNING`. - Standardized event messages to `"AppException handled"` and `"HTTPException handled"`. - **Non-Intrusive Request Metrics Logging (`zcore.web.middleware`)**: - `RequestLogMiddleware` now logs HTTP metrics at `INFO` only in production mode (`DEBUG=False` and `json_format=True`). - In development environments, request metrics fall back to `DEBUG` to eliminate noisy duplicate access logs alongside Uvicorn's native console output. - **Unified Structlog Pipeline (`zcore.logging.config`)**: - Unified JSON and console output through `structlog.stdlib.ProcessorFormatter`. - Applied human-readable timestamps (`%Y-%m-%d %H:%M:%S`) with `ConsoleRenderer(colors=True)` and Rich traceback support for development mode, while preserving ISO timestamps and `JSONRenderer` for production JSON mode. - **Background Scope Context Isolation (`zcore.kernel.di`)**: - `background_scope` now binds `task_id=scope_id` into structlog contextvars. - Clears structlog context when `inherit_context=False` and safely restores prior context variables upon exiting the scope. - **Test Suite Updates**: - Updated configuration synchronization tests in `tests/test_config.py`. - Adjusted SQL query timing tests in `tests/test_db_lifecycle.py`. - Updated exception handling logging assertions in `tests/test_exceptions.py`. - Added test cases verifying `RequestLogMiddleware` logs at `DEBUG` in dev mode and `INFO` in production JSON mode in `tests/test_web.py`. ## Type of change - [x] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [x] Refactoring (architectural or code cleanup with no functional changes) - [ ] Documentation update ## How Has This Been Tested? Please describe the tests that you ran to verify your changes. - [x] Unit/Integration tests (`hatch test`) - Verified `LOG_SQL_QUERIES` bidirectional synchronization in `test_settings_sync_validator_logging_bidirectional`. - Verified SQL logger execution and timing threshold suppression in `test_db_manager_sql_logger_timing`. - Verified `RequestLogMiddleware` emits `DEBUG` logs in default dev mode and `INFO` logs when `DEBUG=False` with `json_format=True`. - Verified status-code-aware logging in `test_exceptions.py`. - [x] Static type check passing (`mypy src/`) ## Checklist: - [x] My code follows the style guidelines and architectural rules of this project - [x] I have performed a self-review of my own code - [x] My changes pass strict static type analysis (`mypy src/` generates no warnings) - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have updated the documentation in `zcore-docs` if applicable - [x] My changes generate no new runtime warnings - [x] I have added tests that prove my fix is effective or that my feature works
…pection, DI scope hardening, and schema pruning optimizations (#123) ## Description This PR brings significant architectural enhancements and performance optimizations across the core framework, specifically targeting the Unit of Work transaction lifecycle, cursor pagination, dependency injection scope isolation, schema data pruning performance, and logger propagation controls. ### Key Changes: - **Nested & Re-entrant Unit of Work (`zcore.db.uow`)**: - Implemented depth-aware transactional coordination (`uow_depth` in `session.info`) to support nested `UnitOfWork` blocks across modular domain services. - Buffered domain events in a shared `session.info['uow_events']` collection across nested scopes; events are dispatched atomically only when the outermost root transaction commits. - Nested scopes (`depth > 1`) invoke `session.flush()` during commit, delegating the physical `session.commit()` to the root boundary (`depth == 0`). - Exceptions at any depth reset the active depth to `0`, clear pending events, and trigger a complete transaction rollback. - **Dynamic Primary Key Inspection in Cursor Pagination (`zcore.db.pagination`)**: - Made `cursor_field` optional in `CursorPagination`, dynamically defaulting to the model's inspected primary key column. - Renamed the cursor payload identifier key from `'id'` to `'pk'` with backward-compatible fallback for legacy cursors. - Simplified SQL WHERE clauses and ORDER BY expressions to direct single-column comparisons (`col < val` / `col > val`) when the cursor field matches the primary key. - Added dynamic primary key Python type introspection (`int` vs `uuid.UUID`) for safe type coercion of `last_pk`. - **Hardened Scoped DI Isolation (`zcore.kernel.di`, `zcore.web.middleware`, `zcore.testing.fixtures`)**: - Changed `_scoped_instances` `ContextVar` default from an empty dictionary `{}` to `None`, preventing accidental cross-scope or out-of-scope registrations. - `ScopedDependencyMiddleware`, `background_scope`, and `DatabaseRollback` now explicitly initialize and reset their own `_scoped_instances` tokens. - `clear_scope()` now purges the dictionary in-place rather than reallocating. - Optimized `background_scope` and `ScopedDependencyMiddleware` to yield directly when no database session factory is configured. - **Zchema Pruning Performance Optimizations (`zcore.web.projection`)**: - Added `STANDARD_ACTIONS` frozenset for rapid action-level restriction resolution. - Added early termination guards to `_prune_data`, `secure_serializer`, and `prune_schema` when no relative paths or nested restrictions exist, eliminating costly recursive dictionary and list traversals on unconstrained payloads. - **Granular Logger Categorization (`zcore.config`, `zcore.logging.config`)**: - Split third-party loggers into three distinct categories: `muted_loggers` (default: `sqlalchemy.engine`), `passthrough_loggers` (default: Uvicorn loggers for dev mode), and `intercept_loggers` (for production JSON formatting). - Preserved native Uvicorn console formatting in dev mode (`propagate=False` with untouched handlers) while cleanly clearing handlers and enabling propagation in JSON production mode. - **Test Suite Updates**: - Added test coverage for `passthrough_loggers` in development mode (`test_passthrough_loggers_in_development_mode`). - Updated logging configuration tests for the new logger classifications in `tests/test_config.py` and `tests/test_logging.py`. ## Type of change - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [x] Refactoring (architectural or code cleanup with no functional changes) - [ ] Documentation update ## How Has This Been Tested? Please describe the tests that you ran to verify your changes. - [x] Unit/Integration tests (`hatch test`) - Verified nested `UnitOfWork` scope depth tracking, nested flushing, root commit, and exception rollback behavior. - Verified cursor pagination with dynamic model primary keys, composite ordering vs direct PK ordering, and integer/UUID coercion. - Verified scoped DI isolation and token lifecycle across middleware, background tasks, and test fixtures. - Verified field pruning logic with early termination against single and nested restriction schemas. - Verified logging behavior in both JSON production and colored development modes. - [x] Static type check passing (`mypy src/`) ## Checklist: - [x] My code follows the style guidelines and architectural rules of this project - [x] I have performed a self-review of my own code - [x] My changes pass strict static type analysis (`mypy src/` generates no warnings) - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have updated the documentation in `zcore-docs` if applicable - [x] My changes generate no new runtime warnings - [x] I have added tests that prove my fix is effective or that my feature works
…caffolding (#124) ## Description This PR updates the CLI app generator (`zc startapp`) to generate test files following standard Pytest module naming conventions (`test_<app_name>.py`) instead of the generic static filename `tests.py`. ### Key Changes: - **Dynamic File Pattern Support**: - Updated `COMPONENT_TEMPLATES` to treat filenames as format patterns (changing `"tests.py"` to `"test_{app_name}.py"`). - Updated `start_app` in `commands.py` to interpolate filename patterns using `filename_pattern.format(**context)` prior to writing to disk. - **Interactive TUI Label Update**: - Updated the layer choice prompt in `src/zcore/cli/__init__.py` to display `Pytest Async Test Suite (test_<app_name>.py)` instead of the hardcoded `tests.py` label. ## Type of change - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Refactoring (architectural or code cleanup with no functional changes) - [ ] Documentation update ## How Has This Been Tested? Please describe the tests that you ran to verify your changes. - [x] Unit/Integration tests (`hatch test`) - Verified `zc startapp` domain module generation creates `test_<app_name>.py` matching the target app name. - Verified template rendering and syntax validity of generated test suites. - [x] Static type check passing (`mypy src/`) ## Checklist: - [x] My code follows the style guidelines and architectural rules of this project - [x] I have performed a self-review of my own code - [x] My changes pass strict static type analysis (`mypy src/` generates no warnings) - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have updated the documentation in `zcore-docs` if applicable - [x] My changes generate no new runtime warnings - [x] I have added tests that prove my fix is effective or that my feature works
- Updated test file existence checks to expect test_{app_name}.py instead of tests.py
- Added auto_error parameter to BaseAuth controlling whether missing or invalid tokens raise - Made oauth2_scheme return None instead of raising when token is absent - Return None from __call__ for missing, invalid, or inactive users when auto_error is False - Preserved original AuthError messages when auto_error is True - Added get_optional_user_stub dependency for routes supporting optional authentication
- Added get_optional_user_stub to security package exports
…ntexts - Added __private__ ClassVar to Zchema for declaring always-pruned fields - Included private fields in relative restricted paths when no user is authenticated - Initialized relative_paths once and returned early when no model is mapped
- Added get_current_user_stub and get_optional_user_stub to zcore lazy exports and __all__ - Registered get_optional_user_stub in ZTestClient dependency overrides - Fixed minor formatting in zcore __getattr__
- Simplified AuthError handling in BaseAuth to a single re-raise path - Added tests for optional missing, invalid, and valid tokens, and required missing token - Added test for get_optional_user_stub default None - Added optional-me route and tests for authenticated and guest mocking in ZTestClient - Added Zchema private field tests for guest pruning, authenticated visibility, JSON schema pruning, and input filtering
…ate fields (#125) ## Description This PR introduces support for optional authentication workflows and schema-level private field pruning for unauthenticated contexts across `zcore`. Key changes: - **Optional Authentication Support**: - Added `auto_error: bool = True` configuration to `BaseAuth`. When set to `False`, unauthenticated requests (missing/invalid tokens, inactive/missing users) return `None` instead of raising `AuthError`. - Configured `OAuth2PasswordBearer` with `auto_error=False` to allow FastAPI dependency resolution to proceed without headers. - Added and exported `get_optional_user_stub` dependency anchor across `zcore` and `zcore.security`. - Updated `ZTestClient` to mock `get_optional_user_stub` alongside `get_current_user_stub`. - **Schema-level Private Fields for Guests**: - Added `__private__` `ClassVar` attribute to `Zchema` (`src/zcore/web/projection.py`) allowing schemas to declare sensitive fields that are automatically pruned from outputs, inputs, and generated JSON schemas when no authenticated user (`ctx.user_id`) is present. - **Test Coverage**: - Added unit and integration tests covering optional auth, test client guest/authenticated dependency overrides, and private schema field pruning. ## Type of change - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Refactoring (architectural or code cleanup with no functional changes) - [ ] Documentation update ## How Has This Been Tested? - [x] Unit/Integration tests (`hatch test`) - [x] Static type check passing (`mypy src/`) **Test Details:** - `tests/test_security.py`: Tested `BaseAuth` with `auto_error=False` (missing token, invalid token, valid token), verified `auto_error=True` raises `AuthError`, and verified `get_optional_user_stub` default behavior. - `tests/test_testing_subsystem.py`: Verified `ZTestClient` handles `/optional-me` routes correctly for both guest and authenticated states. - `tests/test_web.py`: Validated that `Zchema.__private__` fields are pruned for unauthenticated contexts (serialization, JSON schema, input filtering) and visible for authenticated users. ## Checklist: - [x] My code follows the style guidelines and architectural rules of this project - [x] I have performed a self-review of my own code - [x] My changes pass strict static type analysis (`mypy src/` generates no warnings) - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have updated the documentation in `zcore-docs` if applicable - [x] My changes generate no new runtime warnings - [x] I have added tests that prove my fix is effective or that my feature works
…owcase - Restructured README with Overview, Honest Comparison, and Core Patterns sections - Added ZShop reference implementation showcase section - Replaced feature table with framework comparison table including Django - Added deep-dive details sections with problem/solution narratives for each pattern - Updated quick start with 60-second flow and 8 endpoint listing - Added Community Testing call-to-action for rc.2 feedback - Improved badge alt text and license badge - Updated footer attribution
- Added concurrency group with cancel-in-progress to avoid redundant runs - Renamed test job to quality-check running Ruff lint, Ruff format check, and mypy - Added matrix-tests across ubuntu, windows, and macos with Python 3.11–3.13 and hatch test - Added smoke-install-test building the wheel and verifying the zc CLI and zcore import - Collapsed branch filters to inline arrays for pull_request and push
- Added trailing newlines to all source and test files - Reformatted long function signatures, decorators, and expressions to multi-line - Consolidated short expressions into single lines where appropriate - Applied consistent parenthesized context managers in test patches - Reorganized imports and added blank lines between test classes/functions - Fixed quote escaping in env file parser - Normalized parametrize trailing commas across test files
- Replaced strict = true with granular mypy options (check_untyped_defs, no_implicit_optional, pydantic plugin) - Added ignore_errors overrides for base_router, testing fixtures, repository, and handlers - Added .mypy_cache to .gitignore - Added int() coercion for settings-driven numeric values in cache, pagination, and search - Made Inject class generic (Inject(Generic[T])) for proper type inference - Added cast() calls in security auth, storage provider, and BaseAuth cache - Annotated async_results, response_payload, and BaseZTest.app with Any where mypy inference fails - Added type: ignore comments for session.info assignment and get_settings default - Changed send_wrapper message type to Any
- Replaced hatch install with pip install -e '.[all]' - Added explicit pip upgrade step
- Expanded matrix tests to include macOS and Python 3.13 without include overrides - Added fail-fast: false to smoke install test - Added macos-latest to smoke install matrix - Moved sqlalchemy[asyncio] from optional extras to core dependencies - Removed sqlalchemy from db, postgres, and mysql extras - Added blank line between CI jobs
- Added all-checks-passed job depending on quality-check, matrix-tests, and smoke-install-test - Fails if any upstream job result is not success - Enables single required status check for branch protection rules
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This release candidate (
v0.1.0-rc.2) introduces major architectural stabilization, performance optimizations, and core feature additions to prepare the framework for the upcomingv1.0.0stable milestone.Summary
AppException,RequestValidationError,StarletteHTTPException,ResponseValidationError, and genericException) wrapping responses inResponseWrapper[None], with Pydantic error sanitization (_sanitize_error_item,_format_error_location), debug-gated response details, and one-call registration viaregister_exception_handlers(app).restore()/restore_multi()with atomic SQL dialect-awareRETURNINGupdates, lifecycle hooks (pre/post_restore), andforce: bool = Falseflag propagation for physical deletion viaDELETE /resource/{id}?force=true.LOOKUPEndpoint & Query Projections: IntroducedRouteKey.LOOKUP(POST /lookup) with automatic schema-driven column projection (load_only), relationship eager-loading (selectinload/joinedload), allowed field validation, andActions.LOOKUPRBAC permission integration.not_like,not_ilike,not_contains,not_startswith,not_endswith,not_in,not_between,is_not_null) and logicalnotgrouping to the AST query builder.uow_depth) for nestedUnitOfWorkscopes, buffering domain events insession.info['uow_events']to ensure atomic root-level dispatching.ZContext,UserProtocol,StreamManager, and keysetCursorPaginationto seamlessly supportint,str, andUUIDidentifiers.auto_error=Falsesupport toBaseAuth, exportedget_optional_user_stub, and introduced__private__class attributes onZchemato prune sensitive fields for unauthenticated requests automatically.<500, ERROR for>=500), decoupled slow-query logging, preserved native Uvicorn console formatting in dev mode vs JSON intercept in production, and hardened Structlog contextvar isolation across background scopes.setup_test_database,EventDispatcherSandbox, custom engine/dependency overrides inDatabaseRollback, and guest/superuser mocking inZTestClient.zc runwith cascading configuration (CLI > .env > Defaults) and transparent Uvicorn argument forwarding, added dynamic test suite naming (test_<app_name>.py), restructuredREADME.mdwith the ZShop showcase, and split GitHub Actions CI into quality check, cross-platform matrix, and smoke installation jobs.Impact
Standardizes error handling, transaction lifecycles, and data exposure mechanisms across all layers of the framework. These changes resolve multi-tenancy, typing, and background execution edge cases while boosting serialization performance and providing developer tooling for the stable
v1.0.0release.Type of change
How Has This Been Tested?
hatch test)force=True), and restoration lifecycles.ZTestClientsavepoint rollbacks, polymorphic user IDs, and optional auth mocking.Zchemaguest pruning (__private__) and role-based field restrictions.mypy src/)ruff check src/,ruff format --check src/).pip install dist/*.whland CLI verification).Checklist:
mypy src/generates no warnings)zcore-docsif applicable