Build same images only once, even they are used in multiple services

[adaamz]

Is your feature request related to a problem? Please describe.
I'm always frustrated when same image is built multiple times.

Describe the solution you'd like
If one image (with same tag) is used in multiple services, docker-compose should be smart enough to not build it multiple times.

Describe alternatives you've considered
No existing alternative, docker-compose up --build also build it multiple times. Also I'm unable to use proper caching, because of #7342 problem.

Additional context
This docker-compose.yml should build only one image web:${TAG} once, not twice, when running command docker-compose build/docker-compose up --build.
Also when I run docker-compose up my-web/my-web-saver, it should build the image if not exists.

version: '3.2'
services:
    my-web:
        image: web:${TAG}
        build:
            cache_from:
                - web:${TAG}
                - web:latest
            context: web
            dockerfile: web/Dockerfile

    my-web-saver:
        image: web:${TAG}
        build:
            cache_from:
                - web:${TAG}
                - web:latest
            context: web
            dockerfile: web/Dockerfile

[ndeloof]

Both docker-compose latest version and Compose v2 use BuildKit by default, and the later will manage building services in parallel and avoid duplicate builds for services with equivalent Dockerfiles.

[Arcitec]

While it's true that BuildKit detects identical build instructions/dockerfiles and uses a shared cache for all services, you should always specify an image name!

When a service in a Compose file specifies a build: block but does not specify an image: name, Docker Compose automatically invents a name and tag for the resulting image using the default naming convention: <project-name>-<service-name>:latest.

Consider a compose.yml located in a directory named myapp:

services:
  web:
    build: .
    command: ["python", "app.py"]

  worker:
    build: .
    command: ["python", "worker.py"]

  cron:
    build: .
    command: ["python", "scheduler.py"]

Even though all three services use the exact same Dockerfile in .:

1. BuildKit deduplicates the layers: BuildKit is smart enough to run the Dockerfile steps only once and pull from the internal cache for the other two.
2. Docker creates separate image references: Because Compose needs a target reference for each service, it tags three separate entries in your local image store:

• myapp-web:latest
• myapp-worker:latest
• myapp-cron:latest

Running docker images shows:

REPOSITORY     TAG       IMAGE ID       SIZE
myapp-web      latest    a1b2c3d4e5f6   450MB
myapp-worker   latest    a1b2c3d4e5f6   450MB
myapp-cron     latest    a1b2c3d4e5f6   450MB

To fix this, you must specify an image name:

If you provide both the identical build: definition and the exact same image: name across every service, Compose will build and tag that image only once, then reuse it for all services. It's also a very good idea to add pull_policy: never to ensure that it won't try to pull the named image from a public registry.

# Reusable YAML anchor to avoid repeating the build configuration
x-build-config: &base-build
  image: my-app:latest
  pull_policy: never
  build:
    context: .
    dockerfile: Dockerfile

services:
  web:
    <<: *base-build
    command: ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
    ports:
      - "8000:8000"

  worker:
    <<: *base-build
    command: ["celery", "-A", "tasks", "worker", "--loglevel=info"]

  scheduler:
    <<: *base-build
    command: ["celery", "-A", "tasks", "beat", "--loglevel=info"]

This solution:

• Deduplicates the final image tag and build cache.
• Automatically builds the image before starting with docker compose up.
• Supports granular execution. So docker compose up worker builds the image and runs automatically.
• Multiple build streams are still initialized, though BuildKit merges the deduplicated steps.