/lib64/libc.so.6: version `GLIBC_2.18' not found #17

Oh man, I'm not sure, and I don't have an Ubuntu instance handy. Can you try building this using musl-libc instead of GNU libc? This would entail adding the x86_64-unknown-linux-musl target using rustup.

@Dgame I would probably use a docker image to build in local. AWS Lambda uses Amazon Linux 2017.03.1. We are working to add Rust support to SAM build.

I get the same error when compiling on Arch Linux. And it seems that using amazonlinux doesn't help, building with a docker image yields the same result. Here's a Dockerfile that I used: https://gist.github.com/drogus/e16260baa0210a30922a3808a9eed8ff

After building it I just ran the command from the README:

cargo build -p lambda_runtime --example basic --release 

That said, building with musl works correctly (on Arch I installed the musl package and added a x86_64-unknown-linux-musl target)

Mh, I was able to cargo build successfully and run the basic example with this Dockerfile:

FROM amazonlinux:2017.03.1.20170812
RUN curl https://sh.rustup.rs -sSf | /bin/sh -s -- -y
ENV PATH="/root/.cargo/bin:${PATH}"
RUN rustup install stable
RUN yum install -y gcc gcc-c++ make openssl openssl-devel
RUN mkdir /code
WORKDIR /code

I will try to replicate with Nightly next, not sure what's causing your issue yet.

@Dgame I would probably use a docker image to build in local. AWS Lambda uses Amazon Linux 2017.03.1. We are working to add Rust support to SAM build.

Well, we are already doing that, but there was the slight hope that building it locally would be faster.

Oh man, I'm not sure, and I don't have an Ubuntu instance handy. Can you try building this using musl-libc instead of GNU libc? This would entail adding the x86_64-unknown-linux-musl target using rustup.

I've tried but I get this:

rustup install stable-musl
info: syncing channel updates for 'stable-x86_64-unknown-linux-musl'
info: latest update on 2018-11-08, rust version 1.30.1 (1433507eb 2018-11-07)
error: target not found: 'x86_64-unknown-linux-musl'

I've tried

cargo build --release --target=x86_64-unknown-linux-musl

but then I get those strange OpenSSL errors which I don't get if I drop --target=x86_64-unknown-linux-musl:

Compiling openssl-sys v0.9.39
error: failed to run custom build command for openssl-sys v0.9.39
process didn't exit successfully: /mnt/lambda/target/release/build/openssl-sys-a16ee2b07ae0e6c5/build-script-main (exit code: 101)
--- stdout
cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_LIB_DIR
cargo:rerun-if-env-changed=OPENSSL_LIB_DIR
cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_INCLUDE_DIR
cargo:rerun-if-env-changed=OPENSSL_INCLUDE_DIR
cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_DIR
cargo:rerun-if-env-changed=OPENSSL_DIR
run pkg_config fail: "Cross compilation detected. Use PKG_CONFIG_ALLOW_CROSS=1 to override"

--- stderr
thread 'main' panicked at '

Could not find directory of OpenSSL installation, and this -sys crate cannot
proceed without this knowledge. If OpenSSL is installed and this crate had
trouble finding it, you can set the OPENSSL_DIR environment variable for the
compilation process.

Make sure you also have the development packages of openssl installed.
For example, libssl-dev on Ubuntu or openssl-devel on Fedora.

If you're in a situation where you think the directory should be found
automatically, please open a bug at https://github.com/sfackler/rust-openssl
and include information about your system as well as this message.

$HOST = x86_64-unknown-linux-gnu
$TARGET = x86_64-unknown-linux-musl
openssl-sys = 0.9.39

', /home/dgame/.cargo/registry/src/github.com-1ecc6299db9ec823/openssl-sys-0.9.39/build/main.rs:269:9
note: Run with RUST_BACKTRACE=1 for a backtrace.

warning: build failed, waiting for other jobs to finish...
error: build failed

Any ideas?

@Dgame Are you compiling on a Mac or in a container? In a Mac, you get these errors because the version of OpenSSL and source you need are not available or the compiler/linker cannot find them. There's a pretty good explanation on StackOverflow here. I would recommend using a docker container for building with the Dockerfile from my previous answer.

@Dgame I'm on ARCH so I'm not sure if it helps you in any way, but I had to specify openssl paths and allow cross compilation:

OPENSSL_LIB_DIR=/usr/lib/openssl-1.0
OPENSSL_INCLUDE_DIR=/usr/include/openssl-1.0
PKG_CONFIG_ALLOW_CROSS=1

@sapessi
I'm using Ubuntu 18.04 and only get those pesky ssl errors if I append --target=x86_64-unknown-linux-musl. Without it, it compiles fine but will crash with the error message you can see in my initial post.

@drogus
Thanks, I'll try that next. :)

On thing I should have added @Dgame, is that if you use the Amazon Linux Docker container to build then you don't have to specify a --target option in the build command.

Until some of the command line tools finish, you may want to use the docker and cargo_config from https://github.com/srijs/rust-aws-lambda/tree/master/docker

More information on setting up environment here: https://github.com/srijs/rust-aws-lambda/blob/master/docs/getting-started.md#Deploy

@Dgame you could also try using the C++ runtime. It deals with this issue seamlessly ;) you can build on Ubuntu, ArchLinux, Alpine, etc.

@marcomagdy No, thanks. ;) I've already written the code in rust. It's just about the build time. With Docker it takes ~15 min, without <4min.

This is quite painful. I've spent a good few hours trying to get all of the required musl tools and musl Rust target to compile, and now I'm having issues getting openssl-sys to compile on musl. I'd rather not build inside Docker if possible.
@sapessi You mentioned SAM build - will that resolve these difficulties? Thanks.

Hi everyone,

Having the same issue.

• My build environment is AWS EC2 instance (Amazon Linux 2 AMI 2.0.20190313 x86_64 HVM ebs).

• Lambda fails (after successful deployment) with the same error /var/task/bootstrap: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /var/task/bootstrap)

I think the takeaway of this issue is that this project has to built against musl, as the glibc on the Lambda sandbox is too old for Rust.

@davidbarsky sorry for my ignorance. What your statement "has to built against musl" means? Can I achieve that by using Ubuntu, Debian or Amazon Linux by installing the special packages? Or/And I need to change some configuration of a system or a project?

Thank you.

@konstantinvlasenko Sorry, that's my bad! Nothing to apologize for: I was curt in my previous message. You'll need to build against the x86_64-unknown-linux-musl target, which statically links against the musl-libc. You can install musl-libc on Ubuntu/Debian/Amazon Linux; some of the instructions on https://wiki.debian.org/musl#Installation cover that. You'll also need to add the x86_64-unknown-linux-musl target via Rustup: https://doc.rust-lang.org/edition-guide/rust-2018/platform-and-target-support/musl-support-for-fully-static-binaries.html.

All that being said, I am primarily a macOS user (with the occasional Amazon Linux usage), so I can't really tell you how to install Musl on a Debian-based system. Personally, I build everything via a Docker image.

(I also apologize if I over-explained anything to you!)

Works! Works! Works!
@davidbarsky Thank you very much!

So in my case after following all the steps from here https://aws.amazon.com/blogs/opensource/rust-runtime-for-aws-lambda/ I was missing this one command in my Ubuntu instance:

apt-get install musl

@davidbarsky I am currently facing this issue. I initially tried building with musl but the problem is that some crates rely heavily on C++ libraries. Static linking to the libraries definitely require some extra work (time) as described in rust-lang/rust#36710 (comment).

Is there another way to resolve this?

@chumaumenze That makes sense. Trying to build crates with C/C++ dependencies isn't trivial. Unfortunately, there isn't anything that you can do at the moment. To properly resolve this issue, the runtime environment needs to be updated to use Amazon Linux 2, which has a significantly more recent glibc. I'm unable to share a date for when that work will be completed, but I'll let you know when that update has launched.

@softprops , would it be better to replace

Since Lambda uses Amazon Linux, you'll need to target your executable for an x86_64-linux platform.

with

Since Lambda uses Amazon Linux, you'll need to target your executable for x86_64-unknown-linux-musl platform.

in the ReadMe (https://github.com/awslabs/aws-lambda-rust-runtime#aws-cli)?

I managed to get my Lambda going after targeting -musl.

That doc is still true but when expanded, you've got options.

For local builds musl seems reliable with the tradeoff I'm warming up to of making sure your os or the one your building on has all of the musl prerequisites manually installed. This varies by os.

This is the approach I'm trying to get sam team to accept aws/aws-lambda-builders#174 (available someday maybe)

And one I'm experimenting with in the serverless framework https://github.com/softprops/serverless-rust/blob/master/README.md#-experimental-local-builds (available today)

The alternative is to build on a container that matches the lambda runtime. The tradeoff here is that assumes docker. Docker is most likely already present on many dev envs but comes with some limitations of what is preinstalled in that docker image https://github.com/softprops/lambda-rust. (available today)

I'd actually close this issue and direct folks to provided.al2, all the MUSL tooling is not required anymore... unless there's an edge usecase I'm not aware of nowadays?

I'm looking into building a docker container for rust and lambda that can work well with SAM CLI (and Apple Silicon trying to cross-compile ring) in aws/aws-sam-cli#3132 (comment).... so that local builds and sam local start-api works reliably for Rust.

Hey, I've added references to provided.al2 in the README now. Closing this issue.