feat(mongodb-schema): add mongodb-schema package and update dependencies COMPASS-10911 by dudaschar · Pull Request #842 · mongodb-js/devtools-shared

dudaschar · 2026-07-28T14:03:40Z

Introduced the @mongodb-js/mongodb-schema package moving from original https://github.com/mongodb-js/mongodb-schema
Packages who depends on mongodb-schema were also updated to use the local package now.

Description

Open Questions

Checklist

I have signed the Contributor License Agreement (https://www.mongodb.com/legal/contributor-agreement)
New tests and/or benchmarks are included
Documentation is changed or added

- Introduced the @mongodb-js/mongodb-schema package to infer probabilistic schemas for MongoDB collections. - Updated package.json files across multiple packages to replace direct dependencies on 'mongodb-schema' with the new '@mongodb-js/mongodb-schema'. - Added ESLint, Prettier, and Mocha configuration files for consistent code quality and formatting. - Included a comprehensive Apache 2.0 License file. - Updated tests and autocompletion logic to utilize the new mongodb-schema package.

Copilot

Pull request overview

This PR introduces a new first-party @mongodb-js/mongodb-schema package (schema inference + converters + TS type generation + CLI) and migrates existing packages to consume it instead of mongodb-schema, along with associated tooling/config updates.

Changes:

Added new packages/mongodb-schema package with schema analysis, converters, CLI, and extensive test suite.
Updated @mongodb-js/mql-typescript and @mongodb-js/mongodb-ts-autocomplete imports/dependencies to use @mongodb-js/mongodb-schema.
Added/updated formatting, linting, and dependency-alignment ignores for the new package and example fixture.

Reviewed changes

Copilot reviewed 66 out of 70 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
packages/mql-typescript/src/testGenerator/test-generator.ts	Switched schema type imports to `@mongodb-js/mongodb-schema`.
packages/mql-typescript/src/driverSchema/static-schemas.ts	Switched schema type import to `@mongodb-js/mongodb-schema`.
packages/mql-typescript/src/driverSchema/docs-crawler.ts	Switched schema inference imports to `@mongodb-js/mongodb-schema`.
packages/mql-typescript/package.json	Replaced `mongodb-schema` with `@mongodb-js/mongodb-schema` (currently in devDependencies).
packages/mongodb-ts-autocomplete/src/autocompletion-context.ts	Switched `JSONSchema` type import to `@mongodb-js/mongodb-schema`.
packages/mongodb-ts-autocomplete/src/autocompleter.ts	Switched schema->TS conversion imports to `@mongodb-js/mongodb-schema`.
packages/mongodb-ts-autocomplete/src/autocompleter.spec.ts	Updated tests to use `@mongodb-js/mongodb-schema`.
packages/mongodb-ts-autocomplete/package.json	Switched dependency to `@mongodb-js/mongodb-schema`.
packages/mongodb-schema/tsconfig.json	Added TS config for the new package.
packages/mongodb-schema/tsconfig-lint.json	Added lint TS config for the new package.
packages/mongodb-schema/test/wrapper.test.ts	Added tests for default export wrapper behavior.
packages/mongodb-schema/test/stats.test.ts	Added tests for schema statistics helpers.
packages/mongodb-schema/test/simplified-schema.test.ts	Added tests for simplified schema output.
packages/mongodb-schema/test/semantic-types.test.ts	Added tests for semantic type detection options.
packages/mongodb-schema/test/schema-accessor.test.ts	Added tests for lazy schema conversion accessor.
packages/mongodb-schema/test/regression-strings-have-same-probability.test.ts	Added regression test for value sampling behavior.
packages/mongodb-schema/test/promise.test.ts	Added tests ensuring promise-based API contract.
packages/mongodb-schema/test/no-node.test.ts	Added test to ensure runtime works in constrained (non-Node builtins) context.
packages/mongodb-schema/test/nested-document-path.test.ts	Added tests for nested path handling with dotted field names.
packages/mongodb-schema/test/mixed-type-probability.test.ts	Added tests for mixed-type probability calculations.
packages/mongodb-schema/test/mixed-type-order.test.ts	Added tests for mixed-type ordering behavior.
packages/mongodb-schema/test/mixed-type-nested.test.ts	Added tests for nested mixed-type handling.
packages/mongodb-schema/test/mixed-type-evolving-schema.test.ts	Added tests for evolving-schema scenarios.
packages/mongodb-schema/test/integration/generate-and-validate.test.ts	Added integration tests: generate schema + validate documents (incl. MongoDB validation).
packages/mongodb-schema/test/get-schema-paths.test.ts	Added tests for schema path extraction.
packages/mongodb-schema/test/field-order.test.ts	Added tests for field ordering rules.
packages/mongodb-schema/test/bloated.test.ts	Added tests for value cropping and complexity abort threshold.
packages/mongodb-schema/test/basic.test.ts	Added baseline schema inference tests.
packages/mongodb-schema/test/basic-unique.test.ts	Added tests for uniqueness/duplicates behavior.
packages/mongodb-schema/test/basic-probability.test.ts	Added tests for field/type probabilities incl. Undefined.
packages/mongodb-schema/test/basic-embedded-documents.test.ts	Added tests for embedded document inference.
packages/mongodb-schema/test/basic-embedded-array.test.ts	Added tests for embedded array inference.
packages/mongodb-schema/test/array-object-types.test.ts	Added tests for array/object polymorphism behavior.
packages/mongodb-schema/test/all-bson-types.test.ts	Added tests covering all BSON types.
packages/mongodb-schema/test/all-bson-types-fixture.ts	Added BSON fixture documents (incl. edge cases).
packages/mongodb-schema/src/util.ts	Added abort/yield utility.
packages/mongodb-schema/src/types.ts	Added schema type definitions (standard/MongoDB/expanded).
packages/mongodb-schema/src/to-typescript.ts	Added JSON-schema-to-TypeScript type literal generator.
packages/mongodb-schema/src/to-typescript.spec.ts	Added tests for TypeScript type generation.
packages/mongodb-schema/src/stats.ts	Added schema statistics implementation.
packages/mongodb-schema/src/semantic-types/index.ts	Added semantic type registry.
packages/mongodb-schema/src/semantic-types/email.ts	Added email semantic type detector.
packages/mongodb-schema/src/schema-converters/internal-to-standard.ts	Added converter from internal schema to standard JSON Schema.
packages/mongodb-schema/src/schema-converters/internal-to-mongodb.ts	Added converter from internal schema to MongoDB `$jsonSchema`.
packages/mongodb-schema/src/schema-converters/internal-to-expanded.ts	Added converter to expanded JSON schema with metadata/sample values.
packages/mongodb-schema/src/schema-converters/internal-to-expanded.test.ts	Added converter tests for expanded schema output and abort behavior.
packages/mongodb-schema/src/schema-analyzer.ts	Added main schema analysis implementation and options.
packages/mongodb-schema/src/schema-accessor.ts	Added lazy conversion accessor with memoization.
packages/mongodb-schema/src/index.ts	Added public API surface for parsing/analyzing and exports.
packages/mongodb-schema/scripts/time-testing.ts	Added simple timing script for manual perf testing.
packages/mongodb-schema/README.md	Added package documentation and usage examples.
packages/mongodb-schema/package.json	Added new package manifest and scripts.
packages/mongodb-schema/LICENSE	Added Apache 2.0 license file.
packages/mongodb-schema/examples/parse-schema.ts	Added example: parse schema from MongoDB cursor.
packages/mongodb-schema/examples/parse-from-file.ts	Added example: parse schema from line-delimited JSON file.
packages/mongodb-schema/examples/fanclub.json	Added example dataset fixture.
packages/mongodb-schema/bin/mongodb-schema	Added CLI entrypoint for schema inference.
packages/mongodb-schema/.prettierrc.json	Added Prettier config for the new package.
packages/mongodb-schema/.prettierignore	Added Prettier ignore rules for generated/build artifacts and fixture.
packages/mongodb-schema/.mocharc.js	Added Mocha config for the new package.
packages/mongodb-schema/.eslintrc.js	Added ESLint config for the new package.
packages/mongodb-schema/.eslintignore	Added ESLint ignore rules for build artifacts/fixture.
packages/mongodb-schema/.depcheckrc	Added depcheck config for the new package.
.prettierignore	Ignored the new fanclub fixture at repo root.
.depalignrc.json	Updated dependency alignment ignore list (added `js-yaml`).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

dudaschar · 2026-07-28T14:33:24Z

+  allowAbortDuringAnalysis(): Promise<void> {
+    // Allow aborting the analysis.
+    if (this.fieldAndTypeAnalysisCounter++ % ALLOW_ABORT_INTERVAL_COUNT === 0) {
+      void allowAbort();
+    }
+    return Promise.resolve();
+  }


This is the same the previous repo does, https://github.com/mongodb-js/mongodb-schema/blob/main/src/schema-analyzer.ts#L520. Change around the code is to comply with typechecking.

dudaschar · 2026-07-28T14:34:55Z

+  if (type.bsonType === 'Array') {
+    const items = await parseTypes((type as ArraySchemaType).types);
+    // Don't include empty bson type arrays (it's invalid).
+    if (!items.bsonType || items.bsonType?.length > 0) {
+      schema.items = items;
+    }


Same code as original package https://github.com/mongodb-js/mongodb-schema/blob/main/src/schema-converters/internalToMongoDB.ts#L51

dudaschar · 2026-07-28T14:35:46Z

+    return 1;
+  }
+  // Otherwise sort case-insensitively.
+  return aName.toLowerCase() < bName.toLowerCase() ? -1 : 1;


Same code as original package https://github.com/mongodb-js/mongodb-schema/blob/main/src/schema-analyzer.ts#L216

dudaschar · 2026-07-28T14:36:43Z

+  if (types.length === 1) {
+    return `${types[0]}[]`;
+  }
+  return `${types.join(' | ')})[]`;


Same code as original package: https://github.com/mongodb-js/mongodb-schema/blob/main/src/to-typescript.ts#L104

dudaschar · 2026-07-28T14:38:44Z

+  const eachFieldDefinition = Object.entries(properties).map(
+    ([propertyName, schema]) => {
+      switch (getBSONType(schema)) {
+        case 'array':
+          assertIsDefined(schema.items, 'schema.items must be defined');
+          return `${indentSpaces(indent)}${propertyName}?: ${arrayType([
+            ...uniqueTypes(schema.items),
+          ])}`;
+        case 'object':
+          assertIsDefined(
+            schema.properties,
+            'schema.properties must be defined',
+          );
+          return `${indentSpaces(indent)}${propertyName}?: ${toTypescriptType(
+            schema.properties as Record<string, JSONSchema>,
+            indent + 1,
+          )}`;
+        default:
+          return `${indentSpaces(indent)}${propertyName}?: ${[
+            ...uniqueTypes(schema),
+          ].join(' | ')}`;
+      }
+    },
+  );


Same code as original package, only lint changes: https://github.com/mongodb-js/mongodb-schema/blob/main/src/to-typescript.ts#L111

dudaschar · 2026-07-28T14:29:12Z

    "@mongodb-js/eslint-config-devtools": "^0.11.8",
    "@mongodb-js/prettier-config-devtools": "^1.0.4",
+    "@mongodb-js/mongodb-schema": "^0.1.0",
    "@mongodb-js/shell-bson-parser": "^1.5.14",
    "@mongodb-js/tsconfig-devtools": "^1.1.3",


The previous package was also a devDepencency, should we really need to dependencies, or ignore this comment? I'm not sure if I'm missing out something here.

dudaschar · 2026-07-28T14:24:23Z

+    "email": "compass@mongodb.com"
+  },
+  "homepage": "https://github.com/mongodb-js/devtools-shared",
+  "version": "0.1.0",


@gribnoysup should we match the current version of the original package?

Doesn't matter too much, but I think we matched it for other packages just so it's easier to track when exactly we moved them, so probably can do the same here

dudaschar · 2026-07-28T14:24:44Z

+`mongodb-schema` can be used as a command line tool or programmatically in your application as a node module.
+
+### Command line
+
+To install mongodb-schema for command line use, run `npm install -g @mongodb-js/mongodb-schema`. This will add a new
+shell script which you can run directly from the command line.
+
+The command line tool expects a MongoDB connection URI and a namespace in the form `<database>.<collection>`.
+Without further arguments, it will sample 100 random documents from the collection and print a schema of
+the collection in JSON format to stdout.
+
+```
+mongodb-schema mongodb://localhost:27017 mongodb.fanclub
+```


Needs to update the documentation after the package is published.

dudaschar · 2026-07-28T14:43:15Z

  "ignore": {
-    "node-fetch": ["^3.3.2", "^2.6.11"]
+    "node-fetch": ["^3.3.2", "^2.6.11"],
+    "js-yaml": ["^5.2.2", "^4.1.1"]


mql-typescript package will need to be updated to js-yaml separated, as it was not straightforward.

gribnoysup

Some of those GH bot comments look kinda legit so worth taking a closer look, but to avoid mixing too much changes into one patch, we can maybe do a follow-up to this PR, what do you think?

Otherwise LGTM

dudaschar · 2026-07-28T15:24:49Z

Some of those GH bot comments look kinda legit so worth taking a closer look, but to avoid mixing too much changes into one patch, we can maybe do a follow-up to this PR, what do you think?

Otherwise LGTM

Make sense, we do already have follow up work from this migration.

coveralls · 2026-07-28T15:56:19Z

Coverage is 80.283% — COMPASS-10911 into main. No base build found for main.

Copilot AI review requested due to automatic review settings July 28, 2026 14:03

Copilot started reviewing on behalf of dudaschar July 28, 2026 14:04 View session

Copilot AI reviewed Jul 28, 2026

View reviewed changes

chore: fix dependency order in package.json

197e441

dudaschar commented Jul 28, 2026

View reviewed changes

dudaschar changed the title ~~feat: add mongodb-schema package and update dependencies~~ feat: add mongodb-schema package and update dependencies COMPASS-10911 Jul 28, 2026

dudaschar changed the title ~~feat: add mongodb-schema package and update dependencies COMPASS-10911~~ feat(mongodb-schema): add mongodb-schema package and update dependencies COMPASS-10911 Jul 28, 2026

dudaschar commented Jul 28, 2026

View reviewed changes

dudaschar marked this pull request as ready for review July 28, 2026 14:56

gribnoysup approved these changes Jul 28, 2026

View reviewed changes

dudaschar enabled auto-merge (squash) July 28, 2026 15:25

dudaschar merged commit 75433d8 into main Jul 28, 2026
31 of 33 checks passed

dudaschar deleted the COMPASS-10911 branch July 28, 2026 15:55

dudaschar mentioned this pull request Jul 29, 2026

fix(mongodb-schema): follow up migration with small bug fixes COMPASS-10911 #843

Open

3 tasks

Uh oh!

Conversation

dudaschar commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Open Questions

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dudaschar Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gribnoysup left a comment

Choose a reason for hiding this comment

Uh oh!

dudaschar commented Jul 28, 2026

Uh oh!

Uh oh!

coveralls commented Jul 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

dudaschar commented Jul 28, 2026 •

edited

Loading

dudaschar Jul 28, 2026 •

edited

Loading