From 3a61145e182568e1082511637e761777f42268ae Mon Sep 17 00:00:00 2001 From: Julien Marrec Date: Tue, 13 Oct 2026 11:39:11 +0200 Subject: [PATCH 1/4] Customizing conan settings: Make glibc version part of the Conan package_id on CentOS 9 Make glibc version part of the Conan package_id on CentOS 9 The gcc 14 binaries on nrel-v2 are built on ubuntu-24.04 (glibc 2.39), and get downloaded on AlmaLinux 9 (glibc 2.34) since the package_id matches, which fails to link (undefined reference to `__isoc23_sscanf`). Add a shared Conan config in developer/conan/config with an optional `os.libc` / `os.libc.version` sub-setting and a glibc profile that sets it from the detected glibc. CentOS 9 CI installs it and passes `-pr:a glibc`, so it gets distinct package IDs and builds from source or pulls the glibc 2.34 binaries from nrel-v2. Other platforms are unaffected. Also refresh binary_incompatility_glibc.md for Conan 2. - Fixes #5652 reliably - Fixes NatLabRockies/openstudio-cmake-tools#6 --- .github/workflows/full-build.yml | 13 ++++- developer/conan/binary_incompatility_glibc.md | 51 +++++++++++-------- developer/conan/config/.conanignore | 23 +++++++++ developer/conan/config/README.md | 33 ++++++++++++ developer/conan/config/global.conf | 8 +++ developer/conan/config/profiles/glibc | 11 ++++ developer/conan/config/settings_user.yml | 10 ++++ 7 files changed, 128 insertions(+), 21 deletions(-) create mode 100644 developer/conan/config/.conanignore create mode 100644 developer/conan/config/README.md create mode 100644 developer/conan/config/global.conf create mode 100644 developer/conan/config/profiles/glibc create mode 100644 developer/conan/config/settings_user.yml diff --git a/.github/workflows/full-build.yml b/.github/workflows/full-build.yml index ed6d3f6b649..f8aaf9ce439 100644 --- a/.github/workflows/full-build.yml +++ b/.github/workflows/full-build.yml @@ -283,6 +283,16 @@ jobs: wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | gpg --dearmor - | tee /etc/apt/trusted.gpg.d/kitware.gpg >/dev/null apt-get update + - name: Add libc setting to Conan (CentOS) + # nrel-v2's gcc 14 binaries are built on ubuntu-24.04 with GLIBC 2.39 (AlmaLinux 9 has 2.34). Making the libc version part of the + # package_id means they never match, so we build from source (or pull the glibc 2.34 ones once uploaded to nrel-v2). + # See https://github.com/NatLabRockies/openstudio-cmake-tools/issues/6 + if: matrix.platform == 'centos-9-x64' + run: | + conan config install developer/conan/config + conan profile show -pr:a glibc | grep "os.libc" + echo "CONAN_EXTRA_ARGS=-pr:a glibc" >> $GITHUB_ENV + - name: Conan install env: CONAN_BUILD_POLICY: ${{ contains(inputs.conan_build_all, matrix.platform) && '*' || 'missing' }} @@ -292,7 +302,8 @@ jobs: --build="$CONAN_BUILD_POLICY" \ -c tools.cmake.cmaketoolchain:generator=Ninja \ -s compiler.cppstd=20 \ - -s build_type=${{ env.BUILD_TYPE }} + -s build_type=${{ env.BUILD_TYPE }} \ + ${CONAN_EXTRA_ARGS:-} - name: Locate Ruby run: | diff --git a/developer/conan/binary_incompatility_glibc.md b/developer/conan/binary_incompatility_glibc.md index 878eeace3e3..5998c649f48 100644 --- a/developer/conan/binary_incompatility_glibc.md +++ b/developer/conan/binary_incompatility_glibc.md @@ -1,36 +1,47 @@ # Binary (in)compatibility of Conan package -The GLIBC version that is used to build the conan packages matter, and that is NOT part of the conan `package_id`. +The GLIBC version that is used to build the conan packages matters, and that is NOT part of the conan `package_id`. -* Ubuntu 20.04 has GLIBC 2.31 -* Ubuntu 18.04 has GLIBC 2.27 -* Centos7 has GLIBC 2.17 +glibc is backward compatible but not forward compatible: a binary built against an older glibc runs on a newer one, but a binary built against a newer glibc may reference symbols that don't exist on an older one. -The conan docker images that are used: -* gcc7 for eg is based on Artful (17.10), GLIBC 2.26 -* gcc8 is Bionic (18.04), -* gcc9 is Eoan (19.10), GLIC 2.30 -* gcc10 and gcc11 are Focal (20.04) +| Distribution | GLIBC | +|------------------------|-------| +| CentOS 7 | 2.17 | +| Ubuntu 18.04 | 2.27 | +| Ubuntu 20.04 | 2.31 | +| CentOS 9 (AlmaLinux 9) | 2.34 | +| Ubuntu 22.04 | 2.35 | +| Ubuntu 24.04 | 2.39 | -cf for eg: https://github.com/conan-io/conan-docker-tools/blob/3f59cb379c0e43097a7685c74b44ae7a3676c0f7/legacy/gcc_7/Dockerfile#L1 +So if a remote (conancenter or `nrel-v2`) has binaries built on a newer distro than yours, the `package_id` still matches (same os / arch / compiler / build_type), Conan downloads it, and you get a link error. +For example, on CentOS 9 with gcc 14 (gcc-toolset-14), the boost binary was built against glibc >= 2.38 [^1], where `sscanf` is redirected to the ISO C23 `__isoc23_sscanf`, which doesn't exist in glibc 2.34: -On Centos7, you'll always get issues, every binary package downloaded will be binary incompatible. +``` +libboost_filesystem.a(operations.o): in function `boost::filesystem::detail::(anonymous namespace)::syscall_initializer::syscall_initializer()': +operations.cpp:(.text.startup+0x5f): undefined reference to `__isoc23_sscanf' +``` + +See https://github.com/NatLabRockies/openstudio-cmake-tools/issues/6 -On Ubuntu 18.04, if you use the system compiler which is gcc-7, it's all good because it'll download a binary package that was made with a 17.10 machine, so an older GLIBC. +[^1]: conan-center-index only provides Linux binaries for gcc 11 and gcc 13, so there is no gcc 14 boost on conancenter. The gcc 14 binaries on `nrel-v2` are ones we build ourselves on `ubuntu-24.04` (glibc 2.39), see [conan-recipes/.github/workflows/conan_v2.yml](https://github.com/jmarrec/conan-recipes/blob/3b0898bc298335efce8396b49d4d69f593c8a490/.github/workflows/conan_v2.yml#L51). -Now on that Ubuntu 18.04 machine you upgrade to gcc-10 (because you want C++20 for example) -> It gets a binary package that was built on 20.04 => **Problem** -So now you can't link against boost/1.79.0 for eg, because that boost is looking for a GLIBC provided symbol like `fnctl64` or `statx` and that doesn't exist in GLIBC 2.27. +## Rebuilding all dependencies -For example: `fnctl` was purely redefined to `fnctl64` in GLIBC 2.28. +The brute-force solution is to rebuild all of the dependencies on the target machine (`conan install . --build="*"`) to **guarantee** binary compatibility. And this is what you should do if developing locally. -## The solution: rebuild all dependencies +But nothing stops Conan from downloading an incompatible binary again next time (e.g. after a cache eviction in CI), and the rebuilt binaries can't be shared since they have the same `package_id` as the incompatible ones on the remotes. -The only safe solution in that case is to just rebuild all of the dependencies on the target machine to **guarantee** binary compatibility. +## The solution: make glibc part of the `package_id` -Our `ConanInstall.cmake` allows getting a `-DCONAN_FIRST_TIME_BUILD_ALL:BOOL=ON` to achieve that. That boolean is automatically switched to OFF after the first pass. +[config/](config/README.md) declares an optional `os.libc` / `os.libc.version` sub-setting (defaulting to `null`, so nothing changes unless you pass it), and a `glibc` profile that sets it from the detected glibc: ```shell -/bin/rm -Rf ~/.conan/data -cmake -DCONAN_FIRST_TIME_BUILD_ALL:BOOL=ON .. +conan config install developer/conan/config +conan install . --build=missing -pr:a glibc -s compiler.cppstd=20 -s build_type=Release +conan upload "*" -r nrel-v2 -c -p "os.libc.version=2.34 ``` + +Since `os.libc` is a sub-setting of `os`, every recipe that declares `os` in its `settings` gets a different `package_id` when you pass the `glibc` profile. No existing conancenter or `nrel-v2` binary matches, so `--build=missing` builds them from source against the local glibc, and they can be uploaded to `nrel-v2` and reused by other machines with the same glibc. + +We use this for CentOS 9 (AlmaLinux 9, glibc 2.34) only. diff --git a/developer/conan/config/.conanignore b/developer/conan/config/.conanignore new file mode 100644 index 00000000000..3139588c52a --- /dev/null +++ b/developer/conan/config/.conanignore @@ -0,0 +1,23 @@ +README.md + +.idea/* +.github/* +.venv/* +venv/* +tests/* +.gitignore +LICENSE +*README.md* +*.DS_Store* +.AppleDouble +.LSOverride +*.pytest_cache* +.*cache +tmp/* +tmp2/* +.vscode/* +.idea/* +demos/ + +*.swp +*.swo diff --git a/developer/conan/config/README.md b/developer/conan/config/README.md new file mode 100644 index 00000000000..19ff9e2a7e1 --- /dev/null +++ b/developer/conan/config/README.md @@ -0,0 +1,33 @@ +# Conan configuration + +Shared Conan 2 configuration for OpenStudio, installed into your Conan home with: + +```shell +conan config install developer/conan/config +``` + +* `settings_user.yml`: declares an optional `os.libc` / `os.libc.version` sub-setting on Linux +* `profiles/glibc`: includes your `default` profile, sets `os.libc` / `os.libc.version` from the detected glibc, and sets `CMAKE_POLICY_VERSION_MINIMUM=3.5` for packages built from source +* `global.conf`: non-interactive mode, parallel downloads, and a source download cache next to the Conan home + +## Why: CentOS 9 / AlmaLinux 9 + +Conan doesn't include the glibc version in the `package_id`, so a binary built against a newer glibc can get downloaded on a machine with an older one, and then fail to link. + +This is a problem on CentOS 9 (AlmaLinux 9), which has glibc 2.34: the gcc 14 binaries on `nrel-v2` are built on Ubuntu 24.04 against glibc 2.39 (conancenter has no gcc 14 Linux binaries), and you get errors like `undefined reference to '__isoc23_sscanf'`. (It could be a problem if we were building for an older Ubuntu release too) +See [binary_incompatility_glibc.md](../binary_incompatility_glibc.md) and https://github.com/NatLabRockies/openstudio-cmake-tools/issues/6 + +## How it works + +Installing this config does **not** change anything by itself: `os.libc` defaults to `null`, so package IDs stay the same, and Ubuntu, macOS and Windows keep using the existing conancenter / `nrel-v2` binaries. + +It only kicks in when you pass the `glibc` profile (`-pr:a glibc`), i.e. on CentOS 9. Since `os.libc` is a sub-setting of `os`, every recipe that declares `os` in its `settings` (nearly all of them) then gets a **different `package_id`**. No existing conancenter or `nrel-v2` binary matches, so `--build=missing` builds them from source against the local glibc. Header-only packages are still downloaded, since their `package_id` ignores settings. + +## Usage (CentOS 9) + +```shell +conan install . --build=missing -pr:a glibc -s compiler.cppstd=20 -s build_type=Release +conan upload "*" -r nrel-v2 -c +``` + +Once uploaded to `nrel-v2`, subsequent CentOS builds download those binaries instead of rebuilding them. CI (`full-build.yml`) passes `-pr:a glibc` for the CentOS 9 build only. diff --git a/developer/conan/config/global.conf b/developer/conan/config/global.conf new file mode 100644 index 00000000000..d698c20ae93 --- /dev/null +++ b/developer/conan/config/global.conf @@ -0,0 +1,8 @@ +# Core configuration (type 'conan config list' to list possible values) +# e.g, for CI systems, to raise if user input would block +# core:non_interactive = True +# some tools.xxx config also possible, though generally better in profiles +# tools.android:ndk_path = my/path/to/android/ndk +core:non_interactive = True +core.download:parallel = {{ [os.cpu_count() - 2, 1]|max }} +core.sources:download_cache={{ os.path.join(os.path.dirname(conan_home_folder), ".conan-download-cache") }} diff --git a/developer/conan/config/profiles/glibc b/developer/conan/config/profiles/glibc new file mode 100644 index 00000000000..8fce2dd39bd --- /dev/null +++ b/developer/conan/config/profiles/glibc @@ -0,0 +1,11 @@ +include(default) + +{% set libc, libc_version = detect_api.detect_libc() %} + +[settings] +os.libc={{ libc }} +os.libc.version={{ libc_version }} + +[buildenv] +# CMake 4 removed compatibility with cmake_minimum_required(VERSION < 3.5), which many older recipes still use +CMAKE_POLICY_VERSION_MINIMUM=3.5 diff --git a/developer/conan/config/settings_user.yml b/developer/conan/config/settings_user.yml new file mode 100644 index 00000000000..dabfcdedab6 --- /dev/null +++ b/developer/conan/config/settings_user.yml @@ -0,0 +1,10 @@ +os: + Linux: + libc: + null: + gnu: + version: ["2.0", "2.1", "2.2", "2.3", "2.4", "2.5", "2.6", "2.7", "2.8", "2.9", + "2.10", "2.11", "2.12", "2.13", "2.14", "2.15", "2.16", "2.17", "2.18", + "2.19", "2.20", "2.21", "2.22", "2.23", "2.24", "2.25", "2.26", "2.27", + "2.28", "2.29", "2.30", "2.31", "2.32", "2.33", "2.34", "2.35", "2.36", + "2.37", "2.38", "2.39", "2.40", "2.41", "2.42", "2.43", "2.44"] From 75c5f6484344afbc27356917d004158cf1e025c5 Mon Sep 17 00:00:00 2001 From: Julien Marrec Date: Tue, 13 Oct 2026 14:27:46 +0200 Subject: [PATCH 2/4] Fix new compiler errors on GCC 14 --- CMakeLists.txt | 4 ++++ src/epjson/epJSONTranslator.cpp | 25 +++++++++++++++++-------- 2 files changed, 21 insertions(+), 8 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index 1f6c58d56e2..82d7af8b4b6 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -562,6 +562,10 @@ if(UNIX) if (CMAKE_BUILD_TYPE STREQUAL "RelWithDebInfo") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-uninitialized") endif() + # GCC >= 12 emits false positive -Warray-bounds and -Wstringop-overflow in libstdc++ on `const char* + std::string&&` (GCC bugs 105329, 107852) + if (CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 12) + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-array-bounds -Wno-stringop-overflow") + endif() endif() endif() diff --git a/src/epjson/epJSONTranslator.cpp b/src/epjson/epJSONTranslator.cpp index 5caddb071df..7e5daa45af3 100644 --- a/src/epjson/epJSONTranslator.cpp +++ b/src/epjson/epJSONTranslator.cpp @@ -21,6 +21,13 @@ #include #include +// GCC >= 14: tells -Wdangling-reference that the returned reference does not bind to any temporary argument +#if defined(__has_cpp_attribute) && __has_cpp_attribute(gnu::no_dangling) +# define OS_NO_DANGLING [[gnu::no_dangling]] +#else +# define OS_NO_DANGLING +#endif + namespace openstudio::epJSON { template @@ -61,7 +68,7 @@ enum class JSONValueType NumberOrString }; -const std::string& toJSONFieldName(std::map& fieldNames, const std::string& fieldNameInput) { +OS_NO_DANGLING const std::string& toJSONFieldName(std::map& fieldNames, const std::string& fieldNameInput) { if (const auto& cached_value = fieldNames.find(fieldNameInput); cached_value != fieldNames.end()) { return cached_value->second; @@ -160,8 +167,9 @@ JSONValueType schemaPropertyTypeDecode(const Json::Value& type) { * else: * getSchemaObjectProperties > [group_name] > items > properties > [field_name] > [field_property_name] */ -const Json::Value& getSchemaObjectFieldProperty(const Json::Value& schema, const std::string& type_description, const std::string& group_name, - const std::string& field_name, const std::string& field_property_name) { +OS_NO_DANGLING const Json::Value& getSchemaObjectFieldProperty(const Json::Value& schema, const std::string& type_description, + const std::string& group_name, const std::string& field_name, + const std::string& field_property_name) { const auto& objectProperties = getSchemaObjectProperties(schema, type_description); @@ -250,9 +258,9 @@ std::string fixupEnumerationValue(const Json::Value& schema, const std::string& } /* Find name of an extensible group and cache it */ -auto getGroupName(std::map, std::pair>& group_names, - std::map& /*field_names*/, const Json::Value& schema, const std::string& type_description, - const std::string& group_name) -> const auto& { +OS_NO_DANGLING auto getGroupName(std::map, std::pair>& group_names, + std::map& /*field_names*/, const Json::Value& schema, const std::string& type_description, + const std::string& group_name) -> const auto& { const auto key = std::pair{type_description, group_name}; @@ -444,12 +452,13 @@ Json::Value toJSON(const openstudio::IdfFile& idf, const openstudio::path& schem }; std::size_t cur_group_number = 0; + const auto idd_eg = obj.iddObject().extensibleGroup(); for (const auto& g : obj.extensibleGroups()) { ++cur_group_number; // get first field and try to make a group name out of it const auto& [group_name, is_array_group] = - openstudio::epJSON::getGroupName(group_names, field_names, schema, type_description, obj.iddObject().extensibleGroup()[0].name()); + openstudio::epJSON::getGroupName(group_names, field_names, schema, type_description, idd_eg[0].name()); auto& containing_json = [&json_object, &group_name = group_name, is_array_group = is_array_group]() -> auto& { if (is_array_group) { @@ -461,7 +470,7 @@ Json::Value toJSON(const openstudio::IdfFile& idf, const openstudio::path& schem }(); for (unsigned int idx = 0; idx < g.numFields(); ++idx) { - const auto& iddField = obj.iddObject().extensibleGroup()[idx]; + const auto& iddField = idd_eg[idx]; const auto fieldName = getFieldName(is_array_group, obj.iddObject(), schema, type_description, cur_group_number, idx, toJSONFieldName(field_names, iddField.name())); From ac9b31b8f0a6148b1e03eb46e9783455e7a9507d Mon Sep 17 00:00:00 2001 From: Julien Marrec Date: Tue, 13 Oct 2026 16:19:45 +0200 Subject: [PATCH 3/4] Missing headers for GCC 14 --- python/engine/PythonEngine.cpp | 1 + src/scriptengine/ScriptEngine.hpp | 3 +++ 2 files changed, 4 insertions(+) diff --git a/python/engine/PythonEngine.cpp b/python/engine/PythonEngine.cpp index 17f5f8fca48..34b441b464c 100644 --- a/python/engine/PythonEngine.cpp +++ b/python/engine/PythonEngine.cpp @@ -8,6 +8,7 @@ #include "../../src/utilities/core/Filesystem.hpp" #include +#include #include #include #include diff --git a/src/scriptengine/ScriptEngine.hpp b/src/scriptengine/ScriptEngine.hpp index 164528b8eac..b28ce017eaf 100644 --- a/src/scriptengine/ScriptEngine.hpp +++ b/src/scriptengine/ScriptEngine.hpp @@ -8,14 +8,17 @@ #include "../utilities/core/ApplicationPathHelpers.hpp" #include "../utilities/core/DynamicLibrary.hpp" +#include #include #include +#include #include #include #include #include #include #include +#include namespace openstudio { From 5b56b51e3f314271fb932f41890af5d033390d11 Mon Sep 17 00:00:00 2001 From: Alex Chapin Date: Thu, 15 Oct 2026 09:56:20 -0400 Subject: [PATCH 4/4] Fix conan upload command Updated the conan upload command to fix syntax. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- developer/conan/binary_incompatility_glibc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developer/conan/binary_incompatility_glibc.md b/developer/conan/binary_incompatility_glibc.md index 5998c649f48..99047abc0be 100644 --- a/developer/conan/binary_incompatility_glibc.md +++ b/developer/conan/binary_incompatility_glibc.md @@ -39,7 +39,7 @@ But nothing stops Conan from downloading an incompatible binary again next time ```shell conan config install developer/conan/config conan install . --build=missing -pr:a glibc -s compiler.cppstd=20 -s build_type=Release -conan upload "*" -r nrel-v2 -c -p "os.libc.version=2.34 +conan upload "*" -r nrel-v2 -c -p "os.libc.version=2.34" ``` Since `os.libc` is a sub-setting of `os`, every recipe that declares `os` in its `settings` gets a different `package_id` when you pass the `glibc` profile. No existing conancenter or `nrel-v2` binary matches, so `--build=missing` builds them from source against the local glibc, and they can be uploaded to `nrel-v2` and reused by other machines with the same glibc.