Marq/fix/python install paths

[MarqRazz]

Generate Python parameter modules from setuptools' build_py/develop commands instead of inferring paths from sys.argv. Alternative to #348 but also vibe coded.

setup(..., cmdclass=parameter_cmdclass('my_parameters', 'my_package/parameters.yaml'))

• Regular builds generate into the build tree, so the module installs with the package wherever it goes (merge-install, overlays, custom --install-base) and the user's __init__.py is kept.
• --symlink-install and editable pip install -e generate into the source package, which is the copy Python imports there.
• Supports several modules via modules=[...]; generate_parameter_module() is unchanged.

Tested on Lyrical across regular, symlink, merge, overlay, and --install-base builds plus pip install -e. The example now has a non-empty __init__.py, so the --symlink-install break in #348 shows up as a test failure. Windows and the buildfarm are untested.

Fixes #192, #255, #257. The README documents a test conftest.py workaround for #233.

[christophfroehlich]

I really don't care how the fix looks like as long as it works ;)

It seems that this one has problems on distros older than lyrical. Is there a reason for that, can we fix it? I applied the same strategy we have in ros-controls to make the rolling version usable on all active distros.. and only want to drop that if really necessary

you guys have a suggestion how we could test all the possible colcon settings for the install paths with github actions? tokens welcome!

[MarqRazz]

The older-distro failures weren't in parameter_cmdclass; the generated module builds and installs correctly on Humble, Jazzy and Kilted. With no testpaths set, pytest < 8 also collects generate_parameter_module_example/ as a package, which puts the source tree back ahead of the installed copy on sys.path, so the tests imported the source package without the generated module. Scoping collection to test/ fixes it on all distros. Humble's setuptools (59.6) has no PEP 660 editable support, so that one test is skipped there. Verified locally in the Humble/Jazzy/Kilted/Lyrical docker images; Windows is untested.

[christophfroehlich]

CI seems to be happy, how can we test the build options from this repo?
At least "no option" (normal workflows) and the "buildfarm settings" are tested (pre-release job)

[MarqRazz]

Added a small Python Install Paths workflow for this. It builds and tests only generate_parameter_library_py and example_python (sparse checkout, --packages-up-to), so each job finishes in about a minute and only runs when those paths change. The matrix is Humble × Rolling (oldest and newest setuptools/pip/pytest, which is what actually varies across distros), crossed with a regular build and --symlink-install, the two code paths the helper has. --merge-install, --install-base and overlays only change where setuptools copies the build output, which the helper no longer computes, so they aren't in the matrix. When Humble goes EOL the matrix becomes [jazzy, rolling].

[christophfroehlich]

Is it desired behavior or just not possible otherwise as that the generated python module is generated in the source folder with --symlink-install?
I see that it isn't symlinked into the install folder actually, which is wrong?

[MarqRazz]

It's not avoidable for setup.py packages. With --symlink-install, colcon doesn't install a Python package's files at all; it puts the source package on PYTHONPATH (via a symlink in build/). Python only looks for submodules in that one directory, so the generated module has to go there. pip install -e works the same way.

C++ is different because ament_cmake installs each file, so it can symlink the generated header from the build dir. Python packages built with ament_cmake get the same behavior through the existing CMake generate_parameter_module() (see example_cmake_python).