* fix(introspection): preserve schema integrity during db pull
- Retain data validation attributes (e.g., @email) on fields after
introspection (#670)
- Preserve original declaration order of enums instead of moving them
to the end of the schema file (#669)
- Preserve triple-slash comments above enum declarations (#669)
Fixes#669, fixes#670
* fix: address PR comments
* fix(cli): improve db pull for relations and defaults
Prevents field name collisions during introspection by refining the naming strategy for self-referencing relations with multiple foreign keys.
Extends support for JSON and Bytes default values across MySQL, PostgreSQL, and SQLite providers to ensure consistent schema restoration.
Adds test cases for self-referencing models to verify the avoidance of duplicate fields.
* fix: address PR comments
* feat: initial db pull implementation
* fix: generate imports and attributes for zmodel-code-generator
* fix: add option to not exclude imports in loadDocument
* fix: continue work on db pull
* fix: missing import
* fix: rewrite model generation
generate model from ground up and diff later
* feat: add ast factory
* fix: ast factory import order
* fix: some runtime bugs
* fix: lint fix
* fix: update zmodel code generator
- include imports in output
- fix indentaions
- include comments in output
* feat: add exclude schemas option
* feat: implement initial diff update
* fix: update format in zmodel code generator
* fix: typo
* feat: progress on database introspection and syncing
* fix: make ignore behave it does in prisma with no index models
* fix: lint fix
* feat: make all format options configurable
* fix: lint fix
* feat: Handle the database type mapping
* fix: catch up with feature updates
- improve code styling
- enable schema support for db pull
* fix: add sqlite e2e test and fix some bugs
* fix: lint fix
* fix: formatting for e2e test schemas
* test: run db pull e2e test also for postgres
* fix: postgres instorspection schema filter
* test: update cli tests
* feat(cli): Improves database introspection and syncing
Enhances the `db pull` command with a spinner for better UX.
Adds color-coded logging to highlight important steps.
Provides more detailed output on schema changes, including deleted models, enums, added fields, and deleted attributes.
Also includes minor improvements to enum mapping and constraint handling.
* fix(cli): fixes field casing and sort issues
* chore(cli): remove temporary test script
Deletes an unused script used for experimenting with URI path resolution. Cleans up the codebase by removing development-only artifacts.
* chore: update pnpm-lock.yaml
* feat(cli): add MySQL support for schema introspection
Introduces a MySQL-specific introspection provider to support pulling existing database schemas into ZenStack.
The implementation includes logic for mapping MySQL data types to ZenStack types, handling auto-incrementing fields, and parsing MySQL-specific enum definitions. It utilizes dynamic imports for database drivers to minimize the CLI footprint for users not targeting MySQL.
* fix(cli): improve field matching logic during db pull
* feat(cli): enhance SQLite introspection with autoincrement support
* fix(cli): refine attribute generation in db pull
* test(cli): update db pull tests for SQLite specific behavior
* refactor(language): export ZModelServices type
* fix(cli): improve sqlite introspection for autoincrement and fk names
* feat(cli): enhance field matching logic during pull by using relation fields
* refactor(cli): refine relation name generation and table syncing
* test(cli): update pull tests to reflect improved schema generation
* test(cli): add MySQL support to test utility helpers
Extends the testing infrastructure to support MySQL databases.
Adds MySQL configuration defaults and environment variable overrides.
Updates the prelude generation logic to handle MySQL connection strings and provider types, enabling broader database integration testing across the CLI.
* fix(cli): omit default constraint names in table sync
Avoids explicitly declaring unique constraint names when they match the default database naming convention. This results in cleaner generated schema code by removing redundant mapping arguments.
* fix: correctly handle default values for 'text' type in PostgreSQL
* fix: sort table indexes to ensure stable schema generation
* refactor: dynamically determine supported db providers in CLI
* test: fix typo in pull test description
* chore(cli): remove debug artifacts and silence test logs
Removes hardcoded file system path debugging and unnecessary console logging from the introspector and test suites. Silences CLI command output during tests to provide a cleaner test execution environment.
* fix(cli): ensure MySQL column and index ordering
Wraps JSON_ARRAYAGG calls in subqueries with explicit ORDER BY clauses to maintain correct metadata ordering.
This addresses a limitation in MySQL versions prior to 8.0.21, where ORDER BY is not supported directly within the JSON_ARRAYAGG function, ensuring consistent introspection results across different database versions.
* fix(cli): preserve column order during MySQL pull
Ensures database columns are sorted by their ordinal position during the introspection process. This maintains the original schema structure and provides a consistent output that matches the physical database layout.
* refactor(cli): remove schema fields from MySQL queries
Eliminates redundant schema and database name fields from the MySQL introspection query. Since MySQL does not support multi-schema architectures internal to a single connection in this context, removing these fields simplifies the data structure and avoids unnecessary metadata overhead.
* fix(cli): improve MySQL introspection and index mapping
Refines the database pull process to better handle MySQL-specific patterns. Improves unique constraint detection to prevent redundant mapping attributes when default naming conventions are used.
Updates the MySQL introspection logic to correctly identify boolean types, handle timestamp precision in default values, and normalize numeric defaults. Also ensures auto-incrementing columns and primary key indexes are correctly mapped to prevent schema duplication.
* test(cli): pass provider to default prelude in tests
Ensures that the default schema prelude correctly reflects the database provider specified in test options. This prevents inconsistencies when generating test projects with non-default providers.
* fix(cli): improve MySQL introspection for types and defaults
Disables NativeEnum support for MySQL to prevent loss of schema-level enums since MySQL enums are column-specific.
Refines boolean and numeric type mapping to better handle synthetic boolean types and preserve decimal precision in default values.
Updates default value parsing logic to correctly identify unquoted strings and avoid misinterpreting numeric literals as booleans.
* fix(cli): improve MySQL default value introspection
Refines how default values are handled during database introspection for MySQL by considering the specific field type. This ensures that boolean variants and numeric literals for Float and Decimal types are correctly formatted and preserved.
Also clarifies unsupported features in the SQLite provider to improve codebase maintainability.
* test(cli): expand and reorganize db pull tests
Enhances the test suite for the database pull command by adding comprehensive coverage for common schema features and PostgreSQL-specific functionality.
Includes new test cases for:
- Restoring complex schemas from scratch, including relations and indexes
- Preserving existing imports in multi-file schema setups
- Handling PostgreSQL-specific features like multi-schema support and native enums
- Verifying schema preservation for field and table mappings
The tests are restructured for better clarity across different database providers.
* refactor: restructure introspection provider interface and attribute generation
* feat: modernize MySQL introspection provider
* feat: modernize PostgreSQL introspection provider
* feat: modernize SQLite introspection provider
* fix: improve relation field naming and default action handling
* feat: track imports and auto-format during db pull
* test: update pull tests to reflect naming and formatting improvements
* fix(cli): refactor PostgreSQL type casting and fix index order
Extracts PostgreSQL type casting logic into a reusable helper function to improve maintainability and ensure consistent attribute handling across all field types.
Adjusts the table index sorting logic to better preserve the original database creation order while maintaining the priority of unique indexes.
* fix(cli): filter out auto-generated MySQL indexes
Prevents foreign key indexes created automatically by MySQL from appearing in the introspected schema. This ensures the output reflects manually defined indexes and avoids redundancy in schema definitions.
* test(cli): support datasource extras in test utils
Enhances the test utility helpers to allow passing extra datasource properties, such as multi-schema configurations for PostgreSQL.
Refactors existing database pull tests to use these extra properties, ensuring the generated ZModel schema correctly reflects multi-schema environments while simplifying assertions.
* fix: address PR comments
* fix: address PR comments
* fix: address PR comments
* fix: address PR comments
* fix(cli): improve file path resolution in pull action
* refactor(cli): extract and enhance name casing logic
* refactor(cli): consolidate default value normalization
* feat(cli): improve enum syncing and relation naming during pull
* docs(cli): add documentation comments to SQL introspection queries
* test(cli): refactor test utilities and modernize test suites
* fix(cli): improve db pull for composite FKs and MySQL uniqueness
Enhances database introspection to correctly handle composite foreign keys by mapping columns by position rather than name alone.
Improves MySQL introspection by checking statistics tables for single-column unique indexes, ensuring accurate model generation even when column keys are ambiguous.
Ensures MySQL synthetic enum names respect requested model casing to prevent unnecessary schema mapping.
Adds comprehensive tests for composite relations and database-specific uniqueness detection.
* fix: address PR comments
* fix(cli): improve SQLite introspection for untyped columns and composite FKs
Ensures columns with no declared type are correctly mapped to Bytes following SQLite affinity rules, preventing them from being marked as Unsupported.
Updates the DDL parser to correctly identify and map constraint names for composite foreign keys. This ensures that multi-column relations are properly restored during the pull process.
Adds regression tests for both untyped columns and composite foreign key restoration.
* feat(cli): pull generated/computed columns as Unsupported type
Improves database introspection by identifying generated columns in MySQL, PostgreSQL, and SQLite. These columns are now pulled as `Unsupported` types containing their full DDL definition, preventing issues where read-only database fields were incorrectly treated as writable application-level fields.
Includes normalization for expression formatting and a fix for string literal escaping in the code generator to ensure stable schema output.
Relates to ZModel introspection consistency.
* fix(cli): Use parameterized queries for MySQL introspection
Switches from template literal interpolation to parameterized queries in MySQL introspection functions. This improves security by preventing potential SQL injection and ensures better handling of database names containing special characters.
* fix(cli): use nullish coalescing for precision check
* feat: custom procs
* chore: cleanup
* fix: remove $procedures from client
* fix: failing test due to previous alias
* feat(custom-procs)!: make procedures envelope-only via $procs
- Switch procedure calls to `db.$procs.name({ args: {...} })` (no positional args)
- Remove legacy `$procedures` alias entirely (client API + server routing/logging)
- Validate procedure envelope input (`args` object, required/unknown keys)
- Keep TanStack Query procedure hooks as `(args, options)` (with conditional args optionality)
- Update server/ORM/client tests for the envelope API
* fix: code review feedback
* fix: code review comments
* fix: coderabbit review comments
* fix: remove useless proxy method
* test: add a couple of e2e tests that verify both typing and runtime
* test: improve e2e tests
* test: add missing mutation flag
* regenerate test schema
* refactor: procedure params generation fix and type refactors
- Simplified procedure's params definition from a tuple an object, since procs are now called with an envelop now
- Refactored procedure related typing to make them more consistent with other CURD types (that usually takes the schema as the first type parameter, and a name as the second)
- Moved detailed procedure's types to "crud-types" where other ORM client detailed types are defined
- Removed some type duplication from hooks side
- Updated the "orm" sample to demonstrate procedures
* fix: disable infinite custom proc queries for now
---------
Co-authored-by: ymc9 <104139426+ymc9@users.noreply.github.com>
