Problem: Docker build cache not working (install layers re-run on every code change) - COPY order and .dockerignore —— Trove problem library

What you will run into

You change only one business file, yet the whole npm install / pip install layer re-runs every time:

Step 3/4 : COPY . /app
Step 4/4 : RUN npm install        # executes on every build, minutes of reinstall

Builds get slower and slower, and it's worse on CI (full reinstall every time).

Why this happens

Docker build cache matching is "parent chain + instruction + inputs":

• Each layer's cache key contains the parent layer ID, the instruction, and that instruction's inputs (checksums for COPY/ADD, the command string for RUN);
• COPY . . counts the whole build context as input - changing one file = the COPY layer checksum changes = the cache for that layer and every layer after it is invalidated;
• RUN npm install's command string didn't change, but its parent layer (the COPY layer) did, so it can't hit the cache either;
• Stuff that shouldn't be in the build context (node_modules, .git, cache directories) makes the COPY layer invalidate constantly.

Essentially: the dependency-install layer gets dragged down by business-file changes that shouldn't affect it. The fix direction = make the COPY layer contain only what the install layer really needs (the dependency manifest), and put the install layer first.

The invalidation chain is "input change → layer checksum change → parent chain breaks": Docker records only the parent layer ID, the instruction, and an input digest per layer.

COPY . . takes the whole build context as input, so touching any business file changes the COPY digest;

RUN npm install's cache key includes its parent layer ID, so it loses its cache too — the cache is not "same command, same reuse" but "verify the whole parent chain layer by layer".

Three deterministic takeaways: ① put dependency installs before COPY of business code so they only depend on the lockfile digest; ② tighten the context with .dockerignore to stop unrelated files from jittering the digest;

③ when debugging, find the first line where ---> Using cache disappears — that is the break point, and the changed input is somewhere before it.

Step by step

1. Move dependency install before COPY — Required

COPY package.json package-lock.json ./
RUN npm ci
COPY . /app

Only dependency-manifest changes re-run the install layer; business code changes no longer trigger it.

2. Tighten the context with .dockerignore — Required

Exclude node_modules/.git/build artifacts/cache directories; otherwise COPY . .'s checksum jitters with unrelated files (also shrinks the upload size).

3. Watch the cache markers while debugging — Required

Wherever ---> Using cache appears in the build output is a hit layer; after changing a file, if the COPY layer shows no Using cache, every layer after it re-runs - walk the output back to the first invalidation point.

4. Cache shared across layers (BuildKit) — Required

RUN --mount=type=cache,target=/root/.npm npm ci

Mount the package-manager cache as a standalone cache volume for higher network-layer hit rates.

5. Use --cache-from in CI — Required

Push a remote image as the cache source (buildx --cache-from), working best with the "install-first" structure; with the order reversed, --cache-from can't save the frequently-invalidated install layer either.

How to confirm it works

After changing a business file, the install layer no longer re-runs (measured output shape):

Change app.cfg again and rebuild -> Step 3/4 RUN -- Using cache (install layer doesn't re-run, fix effective)

Before the fix: after changing a file, neither COPY nor RUN shows Using cache (symptom reproduced).

Other environments

Symptom

Build cache "always invalidates" when building an image:

# Only one business file changed, yet the whole npm install / pip install layer re-runs every time
Step 3/4 : COPY . /app
Step 4/4 : RUN npm install        # executes on every build, minutes of reinstall

Signs: the Dockerfile structure has COPY . . before RUN npm install (or pip/go mod); changing any source file changes the COPY layer and the whole dependency-install layer after it re-runs; builds get slower, worse on CI (full reinstall every time).

Cause

Docker build cache matching is "parent chain + instruction + inputs":

• Each layer's cache key contains the parent layer ID, the instruction, and that instruction's inputs (checksums for COPY/ADD, the command string for RUN);
• COPY . . counts the whole build context (including any changed business file) as input - changing one file = the COPY layer checksum changes = the cache for that layer and every layer after it is invalidated;
• RUN npm install's command string didn't change, but its parent layer (the COPY layer) did, so it can't hit the cache either;
• Likewise, stuff that shouldn't be in the build context (node_modules, .git, cache directories) makes the COPY layer invalidate frequently.

Essentially: the dependency-install layer gets dragged down by business-file changes that shouldn't affect it. The fix direction = make the COPY layer contain only what the install layer really needs (the dependency manifest), and put the install layer first.

Solution

1. Move dependency install before COPY: COPY package.json /app/ + RUN npm install first, then COPY . /app - only package.json changes re-run the install layer; business code changes no longer trigger it.

• npm: COPY package.json package-lock.json ./ -> RUN npm ci;
• pip: COPY requirements.txt ./ -> RUN pip install -r requirements.txt;
• go: COPY go.mod go.sum ./ -> RUN go mod download.

1. Tighten the context with .dockerignore: exclude node_modules/.git/build artifacts/cache directories; otherwise COPY . .'s checksum jitters with unrelated files (also shrinks the upload size).
2. Watch the cache markers while debugging: wherever ---> Using cache appears in the build output is a hit layer; after changing a file, if the COPY layer shows no Using cache, every layer after it re-runs - walk the output back to the first invalidation point.
3. Cache shared across layers (BuildKit): RUN --mount=type=cache,target=/root/.npm etc., mounting the package-manager cache as a standalone cache volume for higher network-layer hit rates.
4. Use --cache-from in CI: push a remote image as the cache source (buildx --cache-from), working best with the "install-first" structure; with the order reversed, --cache-from can't save the frequently-invalidated install layer either.

Related canonicals

Read in order: precondition -> same root / same domain / related:

• Precondition (rule out first):
• Cannot connect to the Docker daemon (precondition: build depends on the daemon)