How to reduce build time and avoid duplicate artifacts in CI for large PHP monoliths

By · Published on · 6 min read

Panel with electrical terminal blocks and colored wires connected on rails
Technical visual evoking infrastructure and CI pipelines Photo: Taken / Pixabay

Is your pipeline slow? Learn how to cut build time and avoid duplicate artifacts in CI for large PHP monoliths. The path includes effective Composer caching, deterministic Docker layers, and producing a single artifact to feed tests and deploys. Below is a practical plan to diagnose bottlenecks and apply improvements without rewriting everything.

Common symptoms and root causes in PHP monolith builds

Large PHP monoliths suffer from long checkout times, dependency installs, and repeated artifact rebuilds. In many pipelines, every job rebuilds everything, producing identical artifacts multiple times and wasting cache.

Quickly map what consumes the most time. It’s almost always Composer, asset builds, and Docker layer downloads. Fixing those areas usually yields immediate improvements.

  • Repeated Composer installs across parallel jobs.
  • No cache or cache invalidated on every commit.
  • Dockerfile that invalidates layers too early.
  • Artifact produced in more than one pipeline job.
  • Lack of controlled concurrency and cancellation of stale runs.
Symptom Likely cause Practical action
composer install is slow throughout the job Composer cache not persisted or keyed incorrectly Persist $COMPOSER_CACHE_DIR using the composer.lock hash as the cache key
Docker rebuilds everything on every commit Invalid COPY order; no BuildKit remote cache Reorder the Dockerfile and enable cache-from/cache-to
duplicate artifacts "build" happens in multiple stages Create a single artifact and fan-out to consumers
heavy checkout Huge repository; full history Use shallow/partial clone and clean media blobs

Data-driven pipeline diagnosis

Measure before you change anything. Record durations per step and identify what varies between commits. Focus on the two or three largest time blocks and on work that’s repeated unnecessarily.

Also check runner I/O. Slow disks and caches stored on high-latency network storage can sabotage pipelines even when configured correctly.

  • Enable detailed Composer logs to see cache hits/misses.
  • Review checkout time and enable shallow clone.
  • Separate image build time from test execution time.
  • Measure cache hit rate per key (commit, composer.lock).

Fast, deterministic Composer (no wasted work)

The biggest gains come from treating dependencies as deterministic and cacheable. Always use composer.lock and non-interactive flags. Avoid reinstalling vendor/ in multiple jobs: build it once and reuse it.

Persist the Composer cache directory between jobs. Key the cache by the composer.lock hash so it’s only invalidated when dependencies change.

  • Use --prefer-dist, --no-interaction and optimized autoloader.
  • Cache $COMPOSER_CACHE_DIR, not the raw vendor/ across branches.
  • Produce vendor/ once and package it into the final artifact.
  • Keep access tokens for private repos in secrets.

If you use containers: BuildKit, layer ordering and remote cache

With Docker, structure the Dockerfile to maximize cache: copy composer.json and composer.lock before the application code. Install dependencies first, then copy the rest. Enable BuildKit and remote cache to share layers between runners.

For practical details on remote cache and BuildKit optimizations, see the specific guide below.

  • Multi-stage builds to separate build and runtime.
  • COPY order: dependency files before application code.
  • --mount=type=cache for Composer inside the build.
  • Use cache-to/cache-from to reuse layers across builds.

Single artifact: build once, fan-out to tests and deploy

Avoid duplicate artifacts by producing a single versioned package per commit (e.g., zip/tar with code, vendor/, and assets). Subsequent jobs only consume that artifact. This reduces environment variation and eliminates wasted work.

Name the artifact with the commit SHA and optionally a content hash to enable deduplication in storage.

  • Create the artifact after fast lint + unit checks pass.
  • Run integration and e2e tests using the same artifact.
  • Deploy exactly the artifact that was tested.
  • Define retention time and cleanup policy.

Pipeline strategy: ordering, matrices and concurrency

Structure the pipeline into clear stages: light checkout, artifact build, test fan-out, publish, and deploy. Use matrices to parallelize tests, never the artifact build.

Control concurrency: cancel obsolete runs on the same branch and limit simultaneous builds that compete for caches. This avoids long queues and unnecessary duplication.

  • Gate: only build the artifact if quick checks pass.
  • Use matrix only for tests; artifact build remains unique.
  • Converge to a single publish/deploy step.
  • Define rules to cancel older runs.

Cache hygiene, retention and security

Cache speeds things up, but without hygiene it becomes clutter. Clean caches when branches are deleted. Rotate keys when composer.lock changes. Monitor hit rates and adjust partitioning.

Pay attention to secrets and private package providers. Store tokens securely and restrict their scope to the pipeline.

  • Set different TTLs for branches and main.
  • Avoid sharing vendor/ across different branches.
  • Rotate tokens periodically and isolate permissions.
  • Audit who can access artifacts and sensitive logs.

Frequently asked questions

How do I avoid reinstalling vendor/ in every CI job?

Produce vendor/ once in the build step and package it into the final artifact. Downstream jobs consume that artifact. For extra speed, persist the Composer cache keyed by the composer.lock hash.

Does BuildKit help even for PHP projects without much C/C++?

Yes. Even without heavy compilation, deterministic layers and remote cache reduce downloads and reinstalls. Correct COPY ordering and reusing Composer layers typically yield significant gains.

What's the best cache key for Composer in CI?

Use a key derived from the composer.lock hash and the OS/PHP version. That way the cache is only invalidated when dependencies actually change, avoiding cross-platform breakage.

How do I eliminate duplicate artifacts across parallel pipelines?

Centralize artifact build in a single job per commit and reuse that package in a fan-out pattern. Name it with the commit SHA and optionally a content hash for storage deduplication. Cancel older runs on the same branch.

Should I cache the vendor/ directory directly?

Prefer caching Composer's cache directory and producing vendor/ inside the single artifact. Caching vendor/ across branches can introduce subtle inconsistencies and complicate debugging.

Need help designing your pipeline?

If you want a CI review for your PHP monolith—implementing a single artifact, efficient caching, and BuildKit—we can discuss your context and priorities.

Discuss CI optimization

See also