howto cross-compile to get to work for powerpc e500v2

This relates to SynoCommunity spksrc project to build and package various open source software to run on Synology NAS.

I've been running into an issue where rust code cross-compiled for powerpc arch segfault (more specifically qoriq). The exact same code builds perfectly fine for all other archs (armv5, v7, aarch64, x86_64, i686). Ref: SynoCommunity/spksrc#5847 SynoCommunity/spksrc#5684

I've took two different approaches:

Using RUSTFLAGS to reproduce our CFLAGS used -mcpu=8548 -mhard-float -mfloat-gprs=double SynoCommunity/spksrc@90ad41d
Building a tier 3 powerpc-unknown-linux-gnuspe <<-- testing code only in my local branch for now

option 1

Using default powerpc-unknown-linux-gnu along with RUSTFLAGS = -Ctarget-cpu=e500 lead to the exact same result, seftault at startup. I may not be using the right RUSTFLAGS?

option 2

As for option 2 I feel I'm digging my own hole as I'm unable to build a powerpc-unknown-linux-gnuspe target using Synology provided toolchain and toolkit. I'm able to build up to stage1 & stage2 but unable to create a fully working target using cargo along with either my stage1 or stage2 builds... (clearly there is something I'm not fully understanding). Here's what I have so far:

git clone --depth 1 https://github.com/rust-lang/rust.git
./x setup compiler
PATH="$(WORK_DIR)/$(TC_TARGET)/bin:$${PATH}" ./x build --target $(RUST_TARGET)
rustup toolchain link powerpc-stage1 $(WORK_DIR)/rust/build/host/stage1
PATH="$(WORK_DIR)/$(TC_TARGET)/bin:$${PATH}" ./x build --stage 2 --target $(RUST_TARGET))
rustup toolchain link pwoerpc-stage2 $(WORK_DIR)/rust/build/host/stage2
--->> Up to this point all working OK <<---

Where it then fails:

rustup override set nightly
echo "[llvm]" >> $(WORK_DIR)/rust/config.toml
echo "allow-old-toolchain = true" >> $(WORK_DIR)/rust/config.toml   <<-- Presuming as `powerpc-e500v2-linux-gnuspe-gcc` is v4.9.3
PATH="$(WORK_DIR)/$(TC_TARGET)/bin:$${PATH}" POWERPC_UNKNOWN_LINUX_GNUSPE_OPENSSL_DIR=$(WORK_DIR)/../../../toolkit/syno-$(ARCH)-$(TCVERSION)/work/usr/ RUST_BACKTRACE=full cargo +$(firstword $(subst -, ,$(RUST_TARGET)))-stage1 build -Zbuild-std=core,alloc --target powerpc-unknown-linux-gnuspe

The error relates to llvm (enven though it did built it succesfully during either stage1 or stage2):

error: failed to run custom build command for `rustc_llvm v0.0.0 (/home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/rust/compiler/rustc_llvm)`
Caused by:
  process didn't exit successfully: `/home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/rust/target/debug/build/rustc_llvm-dc7ac9c0f6cecf54/build-script-build` (exit status: 101)
  --- stdout
  cargo:rustc-check-cfg=values(llvm_component,"ipo")
  cargo:rustc-check-cfg=values(llvm_component,"bitreader")
  cargo:rustc-check-cfg=values(llvm_component,"bitwriter")
  cargo:rustc-check-cfg=values(llvm_component,"linker")
  cargo:rustc-check-cfg=values(llvm_component,"asmparser")
  cargo:rustc-check-cfg=values(llvm_component,"lto")
  cargo:rustc-check-cfg=values(llvm_component,"coverage")
  cargo:rustc-check-cfg=values(llvm_component,"instrumentation")
  cargo:rustc-check-cfg=values(llvm_component,"x86")
  cargo:rustc-check-cfg=values(llvm_component,"arm")
  cargo:rustc-check-cfg=values(llvm_component,"aarch64")
  cargo:rustc-check-cfg=values(llvm_component,"amdgpu")
  cargo:rustc-check-cfg=values(llvm_component,"avr")
  cargo:rustc-check-cfg=values(llvm_component,"loongarch")
  cargo:rustc-check-cfg=values(llvm_component,"m68k")
  cargo:rustc-check-cfg=values(llvm_component,"csky")
  cargo:rustc-check-cfg=values(llvm_component,"mips")
  cargo:rustc-check-cfg=values(llvm_component,"powerpc")
  cargo:rustc-check-cfg=values(llvm_component,"systemz")
  cargo:rustc-check-cfg=values(llvm_component,"jsbackend")
  cargo:rustc-check-cfg=values(llvm_component,"webassembly")
  cargo:rustc-check-cfg=values(llvm_component,"msp430")
  cargo:rustc-check-cfg=values(llvm_component,"sparc")
  cargo:rustc-check-cfg=values(llvm_component,"nvptx")
  cargo:rustc-check-cfg=values(llvm_component,"hexagon")
  cargo:rustc-check-cfg=values(llvm_component,"riscv")
  cargo:rustc-check-cfg=values(llvm_component,"bpf")
  cargo:rerun-if-env-changed=RUST_CHECK
  cargo:rerun-if-env-changed=REAL_LIBRARY_PATH_VAR

  --- stderr
  thread 'main' panicked at compiler/rustc_llvm/build.rs:51:59:
  REAL_LIBRARY_PATH_VAR
  stack backtrace:
     0: rust_begin_unwind
     1: core::panicking::panic_fmt
     2: core::option::expect_failed
     3: core::option::Option<T>::expect
               at /home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/rust/library/core/src/option.rs:888:21
     4: build_script_build::restore_library_path
               at ./build.rs:51:15
     5: build_script_build::main
               at ./build.rs:113:5
     6: core::ops::function::FnOnce::call_once
               at /home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/rust/library/core/src/ops/function.rs:250:5
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
warning: build failed, waiting for other jobs to finish...
make[2]: *** [../../mk/spksrc.tc-rust.mk:64: rustc_target] Error 101
/home/spksrc/qoriq-debug/spksrc/cross/bat/work-qoriq-6.2.4/tc_vars.mk:1: *** An error occured while setting up the toolchain, please check the messages above.  Stop.
make[1]: Leaving directory '/home/spksrc/qoriq-debug/spksrc/cross/bat'

option 3

Now, further reading and as option 3 I may be able to "rebuild" tier 1 powerpc-unknown-linux-gnu by adding the proper target features such as #117347 . But guessing I'd have to run through the same procession as option 2 ?

Using default powerpc-unknown-linux-gnu along with RUSTFLAGS = -Ctarget-cpu=e500 lead to the exact same result, seftault at startup. I may not be using the right RUSTFLAGS?

You are using a tier 2 target. Can you compile any programs successfully? It's not clear to me from your description that this is not a miscompile.

@saethlin

You are using a tier 2 target. Can you compile any programs successfully? It's not clear to me from your description that this is not a miscompile.

@saethlin nope, no programs seems to runs on the qoriq platform using powerpc-unknown-linux-gnu with or without RUSTFLAGS = -Ctarget-cpu=e500. Which is why I'm wondering if I'm using the right target? Or missing build flags... ?

Try -Ctarget-cpu=e500mc

I'm confused, you said QoriQ, but then you said you're targeting MPC8548? Which is it? No, they're not necessarily guaranteed to be compatible: ignore the vendor's claims otherwise and approach this empirically.

You use a vendor-provided (and thus possibly patched?) gcc, so it seems plausible to me that LLVM might have a different model of the CPU capabilities. Can you use clang to build, instead of gcc, for this target, and get working binaries? Or even a stock gcc?

You also made a typo:
rustup toolchain link pwoerpc-stage2 $(WORK_DIR)/rust/build/host/stage2

You use a vendor-provided (and thus possibly patched?) gcc, so it seems plausible to me that LLVM might have a different model of the CPU capabilities.

Indeed, Synology provide a GCC toolchain for each arch. The qoriq (e.g. powerpc) is using a version 4.9.3 of gcc.

Can you use clang to build, instead of gcc, for this target, and get working binaries? Or even a stock gcc?

That would require me to build a clang (or gcc) cross-compiler for this arch? Probably feasible but, I did try in the past using gcc in order to have a newer (and identical) version for all targets but I did not managed to assemble all the pieces together properly...

LLVM-based compilers are always cross-compilers, that I've seen? You can use an x86_64-linux clang, or likewise for rustc, to emit code for PowerPC. The usual clang builds include codegen components for all their supported targets, I think.

Indeed, Synology provide a GCC toolchain for each arch. The qoriq (e.g. powerpc) is using a version 4.9.3 of gcc.

Regarding this: do they patch it?

Indeed, Synology provide a GCC toolchain for each arch. The qoriq (e.g. powerpc) is using a version 4.9.3 of gcc.

Regarding this: do they patch it?

No clue. And I was reviewing our own native llvm build that we provide part of synocli-devel package, we're dropping down to version 9 of llvm for archs that uses gcc <= 5.1... Which includes qoriq whereas it won't be able to build it anyhow without another version of GCC (or clang for that matter).

Rust ships LLVM 17.

GCC is a GPL project, so if you have been given the software, you also have the right to demand the source code, including its patches. ( also if you saw any messages between this and "Rust ships LLVM 17" from me: I'm blaming them all on the cat finding a few macro-commands and the enter key. )

@workingjubilee

@workingjubilee so while my cycles where rather limited I did manage to make a bit of mileage over the last month...

In my current PR SynoCommunity/spksrc#5879 I was able to automate building a rustc compiler supporting powerpc-unknown-linux-gnuspe.

Why you might ask? because it hapens that resulting binaries contains lwsync instead of sync (ref: #96394 (comment)). It was confirmed below in the thread that using -Ctarget-cpu=e500 in conjunction with tier 3 powerpc-unknown-linux-gnuspe target support solves the issue for exactly the the Synology NAS model and CPU type I'm having issues with (ref: #96394 (comment)).

So now in one hand I do have a rustc compiler with tier 3 powerpc-unknown-linux-gnuspe target support... And in the other I'm passing -Ctarget-cpu=e500 using RUSTFLAGS variable... and I can see it being used in the build output (and compiler doesn't complain).

But sadly the end-result binaries still contains lwsync (ref: SynoCommunity/spksrc#5879 (comment)) and ends-up segfaulting (as to be expected, ref: SynoCommunity/spksrc#5847 (comment))

Question I have, what am I missing?

If someone else stumbles on this and wonder howto:

	@(cd $(WORK_DIR) && [ ! -d rust ] && git clone --depth 1 https://github.com/rust-lang/rust.git || true)
	@(cd $(WORK_DIR)/rust && ./x setup compiler)
	@(cd $(WORK_DIR)/rust && \
	    CFLAGS_$(subst -,_,$(RUST_TARGET))="$(TC_EXTRA_CFLAGS)" \
	    CXXFLAGS_$(subst -,_,$(RUST_TARGET))="$(TC_EXTRA_CFLAGS)" \
	    LDFLAGS_$(subst -,_,$(RUST_TARGET))="--sysroot=$(WORK_DIR)/$(TC_TARGET)/$(TC_SYSROOT)" \
	    RUST_BACKTRACE=full \
	    ./x build --config $(TC_LOCAL_VARS_RUST))
	@rustup toolchain link $(TC_RUSTUP_TOOLCHAIN) $(WORK_DIR)/rust/build/host/stage$(RUSTUP_DEFAULT_TOOLCHAIN_STAGE)

Where my config file refers to Synology toolchain tools:

profile = "compiler"

[build]
target = ["powerpc-unknown-linux-gnuspe"]
build-stage = 2
docs = false
docs-minification = false
compiler-docs = false

[target.powerpc-unknown-linux-gnuspe]
cc = "/home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-l
inux-gnuspe-gcc"
cxx = "/home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-
linux-gnuspe-g++"
ar = "/home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-l
inux-gnuspe-ar"
ranlib = "/home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500
v2-linux-gnuspe-ranlib"
linker = "/home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500
v2-linux-gnuspe-gcc"

@workingjubilee

@workingjubilee and @saethlin and @wycats I believe I have found where the issue is ... When building my powerpc-unknown-linux-gnuspe rust toolchain support I am using Synology provided toolchain and thus pointing to its relevant cc and all as per the config.toml above.

The rust toolchain do build just fine, but I noticed that lwsync do appear in the resulting libraries rust/build/host/stage1/lib/rustlib/powerpc-unknown-linux-gnuspe/lib/libstd-4b4243768d8e8241.so whereas objdump -d gives a few of those which are incompatible with this platform:

   ac98c:	7c 20 04 ac 	lwsync
   ac9a8:	7c 20 04 ac 	lwsync

My theory is that when building any software afterward and linking to these libraries, it will necessarely fail as result holds incompatible lwsync ...

Now why? I am providing the necessary CFLAGS and CXXFLAGS at rust toolchain build time... so issue must be how the flags are being passed in here somewhere where flags are not being honored:

(cd /home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/rust \
   CFLAGS_powerpc_unknown_linux_gnuspe="-mcpu=8548 -mhard-float -mfloat-gprs=double" \
   CXXFLAGS_powerpc_unknown_linux_gnuspe="-mcpu=8548 -mhard-float -mfloat-gprs=double" \
   LDFLAGS_powerpc_unknown_linux_gnuspe="--sysroot=/home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/powerpc-e500v2-linux-gnuspe/sysroot" \
   RUST_BACKTRACE=full \
   ./x build --config /home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/qoriq.toml)

Help would be much appreciated, thnx in advance!

Support for powerpc-unknown-linux-gnuspe was added to Rust by me and always worked fine until the port was removed from Debian due to gcc upstream dropping support for this target.

@glaubitz

@glaubitz perhaps you can assist in providing me hints as to why things are not working out?

quick background, synocommunity is spksrc framework github project to build package for Synology NAS on multiple archs (armv5, v7, v8, x86_64, ppc, etc). All other archs are well supported from a rust point of view with the exception of qoriq being a powerpc gnuspe tier 3 target. My current PR in order to support rust for qoriq is SynoCommunity/spksrc#5879

To build rust support I'm using Synology DSM 6.2.4 toolchain for that specific arch. Toolchain bundle file is located here https://sourceforge.net/projects/dsgpl/files/Tool%20Chain/DSM%206.2.4%20Tool%20Chains/PowerPC%20QorIQ%20Linux%202.6.32/qoriq-gcc493_glibc220_hard_qoriq-GPL.txz/download

gcc provided part of the toolchain is gcc version 4.9.3 20150311 (prerelease) (crosstool-NG 1.20.0)

Configuration file I'm using is the following:

profile = "compiler"

[build]
target = ["x86_64-unknown-linux-gnu", "powerpc-unknown-linux-gnuspe"]
build-stage = 1
doc-stage = 2
docs = false
docs-minification = false
compiler-docs = false

[rust]
channel = "stable"
lto = "off"

[llvm]
download-ci-llvm = "if-unchanged"

[install]

[dist]

[target.x86_64-unknown-linux-gnu]

[target.powerpc-unknown-linux-gnuspe]
cc = "/home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-linux-gnuspe-gcc"
cxx = "/home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-linux-gnuspe-g++"
ar = "/home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-linux-gnuspe-ar"
ranlib = "/home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-linux-gnuspe-ranlib"
linker = "/home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-linux-gnuspe-gcc"

Arch gcc specific flags we've been using since ever are -mcpu=8548 -mhard-float -mfloat-gprs=double. Working equivalent to be used with RUSTFLAGS for the llvm backend is -Ctarget-cpu=e500 based on #96394 (comment)

Hopefully what I'm missing is obvious... but your help would be more than welcomed ...

Rollup merge of rust-lang#137860 - taiki-e:powerpcspe-msync, r=workin…

Unrolled build for rust-lang#137860

@taiki-e

@taiki-e with the fix now merged upstream #137860 is the updated ppc toolchain available for download (part of 1.85.0 or nightly)? and if so, would I need to use -Ctarget-cpu=e500 flag or similar to use?

Also noting that something else changed around that same time, I can no longer link my custom built toolchain for my ppc arch. Nothing changed in our framwork with this regards but the error started about the same time as you fix got merged upstream, no clue if this is related:

===>  Processing dependencies of syno-qoriq
make[1]: Entering directory '/home/spksrc/wheel-meson/spksrc/toolchain/syno-qoriq-6.2.4-rust'
===>  Installing for rust-qoriq
find /home/spksrc/wheel-meson/spksrc/toolchain/syno-qoriq-6.2.4-rust/work-native/install//usr/local/ \! -type d -printf '%P\n' | sed 's?^target/??g' | sort > /home/spksrc/wheel-meson/spksrc/toolchain/syno-qoriq-6.2.4-rust/work-native/rust-qoriq.plist.tmp
===>  Rustup 1.82.0-powerpc-unknown-linux-gnuspe
===>  PATH=/home/spksrc/wheel-meson/spksrc/distrib/cargo/bin:/home/spksrc/wheel-meson/spksrc/distrib/cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games RUSTUP_HOME=/home/spksrc/wheel-meson/spksrc/distrib/rustup /home/spksrc/wheel-meson/spksrc/distrib/cargo/bin/rustup toolchain link 1.82.0-powerpc-unknown-linux-gnuspe /home/spksrc/wheel-meson/spksrc/toolchain/syno-qoriq-6.2.4-rust/work-native/install/usr/local
/home/spksrc/wheel-meson/spksrc/distrib/cargo/bin/rustup
error: invalid value '1.82.0-powerpc-unknown-linux-gnuspe' for '<TOOLCHAIN>': invalid custom toolchain name '1.82.0-powerpc-unknown-linux-gnuspe'

For more information, try '--help'.
make[1]: *** [Makefile:29: rust-qoriq-postinstall] Error 1
make[1]: Leaving directory '/home/spksrc/wheel-meson/spksrc/toolchain/syno-qoriq-6.2.4-rust'
make: *** [../../mk/spksrc.depend.mk:54: depend_target] Error 2

EDIT: Something must have changed upstream not allowing our filenaming anymore. Anyhow, now solved with a rebuild of our custom ppc/qoriq toolchain SynoCommunity/spksrc#6477. What`s a bit sad is that 1.82 is the latest version we can build with that older synology toolchain still using gcc-4.9 and glibc-2.20

the updated ppc toolchain available for download (part of 1.85.0 or nightly)?

The fix is available since nightly-2025-03-04 (1.87-nightly). Since powerpc-unknown-linux-gnuspe is tier 3 target, cross compiling with -Z build-std or building rustc on your end is needed.

Navigation Menu

Repository navigation

howto cross-compile to get to work for powerpc e500v2 #117361

option 1

option 2

option 3

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

howto cross-compile to get to work for powerpc e500v2 #117361

Description

option 1

option 2

option 3

Activity

saethlin commented on Oct 29, 2023

th0ma7 commented on Oct 29, 2023

workingjubilee commented on Oct 29, 2023

workingjubilee commented on Oct 29, 2023

workingjubilee commented on Oct 29, 2023

th0ma7 commented on Oct 29, 2023

workingjubilee commented on Oct 29, 2023

workingjubilee commented on Oct 29, 2023

th0ma7 commented on Oct 29, 2023

workingjubilee commented on Oct 29, 2023

workingjubilee commented on Oct 29, 2023

th0ma7 commented on Dec 4, 2023

th0ma7 commented on Jan 11, 2024

glaubitz commented on Jan 19, 2024

th0ma7 commented on Jan 21, 2024

th0ma7 commented on Mar 8, 2025

taiki-e commented on Mar 8, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions