Deferred loading of NumPy and SciPy dependencies

[josemmo]

Problem

import compas.geometry takes several seconds due to top-level NumPy and SciPy imports in compas.geometry and compas.data. This triggers a cascade of around 470 transitive imports at startup, even when those libraries are not used.

While the COMPAS codebase already defers heavy imports in several places¹²³, some modules don't follow this practice, which defeats the effort made elsewhere.

Impact of the PR

	Import time
Before	~2.21s
After	~0.14s
Speedup	~15x

How to reproduce

1. Setup a blank project:

uv init
uv add compas==2.15.1 snakeviz

2. Profile with SnakeViz:

# Clear cache first for a cold-start measurement
Get-ChildItem -Directory -Recurse -Force -Filter "__pycache__" -ErrorAction SilentlyContinue | Remove-Item -Recurse -Force -ErrorAction SilentlyContinue

uv run python -c 'import cProfile; cProfile.run("import compas.geometry", "before.prof")'
uv run snakeviz before.prof

The profile will show _io.open_code dominating runtime across 500+ file reads. This is Python pulling in source/bytecode files from NumPy/SciPy from disk.

For my particular device, it took 2.21 seconds to run import compas.geometry. Note this time may vary significant depending on the machine hardware and cache status.

3. Capture an import time log:

uv run python -X importtime -c "import compas.geometry" 2>&1 | Out-File -FilePath before.txt -Encoding utf8

The log will contain around 470 NumPy/SciPy imports with high cumulative times. For example:

import time: self [us] | cumulative | imported package
import time:      1271 |      63580 | numpy
import time:     27278 |      39595 | numpy.testing._private.utils
import time:       507 |      49062 | numpy.testing
import time:       727 |     160679 | scipy.linalg
import time:       694 |      94077 | scipy.optimize
import time:       439 |      20286 | scipy.spatial.transform
import time:       539 |      79419 | scipy.spatial

How to verify fix

Repeat the steps above after switching to this branch:

uv add "git+https://github.com/josemmo/compas@performance/deferred-loading"

The profile should be significantly shorter (0.14 seconds in my case) and the import log should contain no NumPy or SciPy entries.

Alternatives considered

• lazy_loader (SPEC 1): The approach used by NumPy, scikit-image, NetworkX, and other libraries. Compatible with Python 3.9+, but would require restructuring large parts of the codebase.

• lazy keyword (PEP 810): Native language support, but requires Python 3.15+, which is outside COMPAS's current support range.

Footnotes

1. https://github.com/compas-dev/compas/blob/e03e40975ebec56ab98b4cfaead1eedf70241cb4/src/compas/geometry/polyhedron.py#L395 ↩

2. https://github.com/compas-dev/compas/blob/e03e40975ebec56ab98b4cfaead1eedf70241cb4/src/compas/geometry/_core/distance.py#L595 ↩

3. https://github.com/compas-dev/compas/blob/e03e40975ebec56ab98b4cfaead1eedf70241cb4/src/compas/datastructures/mesh/mesh.py#L404 ↩

[tomvanmele]

i think the PR makes sense. however, before merging can we add a benchmark to check the impact on repeated calls to functions like normrow, transform_points_numpy, trimesh_pull_points_numpy, ...?

[tomvanmele]

i guess this is included by accident?

[josemmo]

No, this is intentional. The linter of the repository applied those corrections during the pre-commit hook.

I put them in a separate commit (2f59322) in case you want to cherry pick them.

[tomvanmele]

i guess this is included by accident?

[josemmo]

(See previous comment)

[josemmo]

can we add a benchmark to check the impact on repeated calls to functions like normrow, transform_points_numpy, trimesh_pull_points_numpy, ...?

Thanks for raising this point!

I don't think a benchmark is necessary here because Python caches all previously imported modules in sys.modules. On subsequent calls to any of these functions, an import statement is effectively a dictionary lookup that takes tenths of nanoseconds, which is completely negligible compared to the time spent on the underlying NumPy/SciPy operations that those functions perform.

Unless functions like normrow/transform_points_numpy/etc are called millions of times in a "for" loop or 60 times per second in an infinite loop, this change should not have any negative impact at all. That said, if there's a specific case you're worried about, please let me know and I'd be happy to take a look :)

[chenkasirer]

So where is the heavy import time deferred to? would it be on the first call to a numpy-backed function?

In apps that make no use of numpy features this is a clear improvement, but in apps that do use numpy it's hard to anticipate when that might be - especially since a user might not even know.

I'd maybe add an explicit warm-up function and documentation for this behavior.
also I'd try to add some test for this; seems easy to re-introduce an import to numpy somewhere and re-introduce this import delay.

[josemmo]

Hi @chenkasirer,

would it be on the first call to a numpy-backed function?

Yep, it will load the module on the first call to a function importing NumPy/SciPy.

---

seems easy to re-introduce an import to numpy somewhere and re-introduce this import delay

Luckily for us, there's a Ruff rule precisely for this! https://docs.astral.sh/ruff/rules/banned-module-level-imports/

If you're happy with this approach, I can update the PR to modify the pyproject.toml file:

[tool.ruff.lint]
select = ["E", "F", "I", "TID253"]

[tool.ruff.lint.flake8-tidy-imports]
banned-module-level-imports = ["numpy", "scipy"]