Packaging

Zip vs layers vs container images. arm64 vs x86_64. Native wheels.

Three deployment formats

Format	Size limit	Best for	Caveats
Zip (direct)	50 MB upload / 250 MB unzipped	Most Python/Node functions with pure-Python or pre-built wheels	Must match Lambda's architecture; no custom runtime
Zip via S3	250 MB unzipped	Same as above but when zip exceeds 50 MB	S3 bucket must be in the same region
Layers	250 MB total (function + all layers)	Shared dependencies across functions (e.g. a company-wide logging layer)	Max 5 layers per function; later layers overwrite earlier ones
Container image	10 GB	ML models, native binary deps, custom runtimes	Slower first cold start (image pull); larger attack surface

Layers in practice

A layer is a zip file that Lambda extracts into /opt before running your function. Your code in /var/task can import from /opt/python (for Python) without any path manipulation. Use cases:

Shared internal libraries deployed independently of business logic
Large dependencies that change rarely (numpy, pandas) — cache them in a layer so deployments of the business logic are fast
AWS-provided layers: Lambda Insights extension, X-Ray SDK

Layers count toward the 250 MB unzipped limit. If you have 5 layers at 40 MB each and your function zip is 50 MB, you're at 250 MB — no room left.

Container images

Container images must be based on AWS-provided base images (public.ecr.aws/lambda/python:3.13) or implement the Lambda Runtime Interface. They must be stored in ECR (Elastic Container Registry) in the same region. The Lambda service caches images on the underlying host after the first pull, so subsequent cold starts on the same host are fast — but the very first invocation after a new image is deployed can be slow for large images.

Container images bypass the 250 MB unzipped limit, which is why they're the standard choice for Python ML workloads that bundle PyTorch or TensorFlow.

arm64 vs x86_64

Graviton2-based arm64 is ~20% cheaper per GB-second than x86_64 and typically faster at compute-heavy work. The decision tree:

Check all your dependencies for arm64 wheels: pip download --platform manylinux2014_aarch64 --only-binary :all: -r requirements.txt. If any fail, you either build from source (needs Dockerfile) or stay on x86.
For pure-Python deps and most modern packages, arm64 works out of the box.
Native extensions (cryptography, numpy, psycopg2) have arm64 wheels on PyPI since ~2022. Check the exact version you need.

Building for Lambda (the common foot-gun)

Lambda runs on Amazon Linux 2023. pip install on macOS produces wheels compiled for macOS, which will segfault or import-error on Lambda. The correct approach:

# build inside the Lambda runtime image
docker run --rm \
  -v "$PWD":/var/task \
  public.ecr.aws/lambda/python:3.13 \
  pip install -r requirements.txt -t python/

zip -r layer.zip python/

This is also where architecture matters: use the :3.13-arm64 tag when building for arm64.

This project uses a zip deployment. aioboto3 and aiofiles are pure-Python and have no native extensions, so they build cleanly on any architecture. The Makefile's install target creates a local .venv for development; a real CI pipeline would build the deployment zip inside the Lambda image.

3.4 KiB Raw Blame History