Caching Dependencies in CircleCI

Ryan Kienstra on August 18, 2022

It’s usually really easy to cache dependencies in CircleCI.

Most of the time…

You don’t even need to set cache keys.

Or know how its caching works.

Below are examples of caching Node, Composer, and Gradle dependencies.

CircleCI takes care of it under the hood when you use orbs for caching.

Orbs For Caching Dependencies

There’s usually a certified CircleCI orb that will handle caching for you.

An orb is packaged YAML for a CircleCI config.

Like an npm package, but for CircleCI.

So caching dependencies in CircleCI usually means finding the right orb for the environment:

Node Example

Here’s how to do caching with the Node orb:

diff --git a/.circleci/config.yml b/.circleci/config.yml
index 74a030f..a6e0f84 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -1,27 +1,31 @@
 version: 2.1
 
+orbs:
+  node: circleci/node@5.0
+
 jobs:
   lint-js:
     docker:
       - image: cimg/node:14.19
     steps:
       - checkout
-      - run: npm ci
+      - node/install-packages
       - run: npm run lint:js
   test-js:
     docker:
       - image: cimg/node:14.19
     steps:
       - checkout
-      - run: npm ci
+      - node/install-packages
       - run: npm run test:js
Code language: Diff (diff)

We reference that orb on lines 18 and 26 by starting with node/, then adding the command name of install-packages.

The resulting node/install-packages command will cache npm ci.

So every time it runs, it’ll first check if there’s a cache for the same lockfile.

If there is, it restores that cache, instead of installing it all from scratch.

Here’s a small part of what that node/install-packages command does:

- restore_cache:
    keys:
      - >-
        node-deps-{{ arch
        }}-<<parameters.cache-version>>-<<#parameters.include-branch-in-cache-key>>{{
        .Branch
        }}-<</parameters.include-branch-in-cache-key>><<^parameters.cache-only-lockfile>>{{
        checksum "/tmp/node-project-package.json"
        }}-<</parameters.cache-only-lockfile>>{{ checksum
        "/tmp/node-project-lockfile" }}
      - >-
        node-deps-{{ arch
        }}-<<parameters.cache-version>>-<<#parameters.include-branch-in-cache-key>>{{
        .Branch }}-<</parameters.include-branch-in-cache-key>>{{
        checksum "/tmp/node-project-package.json" }}
      - >-
        node-deps-{{ arch
        }}-<<parameters.cache-version>>-<<#parameters.include-branch-in-cache-key>>{{
        .Branch }}-<</parameters.include-branch-in-cache-key>>Code language: YAML (yaml)

But you don’t have to know about that.

It takes care of caching dependencies in CircleCI for you.

PHP Example

Again, we’ll use an install-packages command to do caching.

But we’ll prefix it with php.

The full command is php/install-packages, on line 18:

diff --git a/.circleci/config.yml b/.circleci/config.yml
index 20bc520..07bc337 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -2,6 +2,7 @@ version: 2.1
 
 orbs:
   node: circleci/node@5.0
+  php: circleci/php@1.1
 
 jobs:
   lint-js:
@@ -26,7 +27,7 @@ jobs:
     steps:
       - checkout
       - node/install-packages
-      - run: composer install
+      - php/install-packages
       - run: npm run build
       - run: composer test
Code language: Diff (diff)

Here’s a small part of the source of that command:

- restore_cache:
    keys:
      - >-
        composer-deps-<<parameters.cache-version>>-{{ checksum
        "<<parameters.app-dir>>/<<parameters.cache-key>>" }}Code language: YAML (yaml)

You might be worried about whether you’ll have to flush this cache when CircleCI runs.

Like if tests fail.

I haven’t had to do that, its caching has been reliable.

Gradle Example

This is different than the install-packages commands earlier.

This wraps steps in a caching command: gradle/with_cache

-- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -2,6 +2,7 @@ version: 2.1
 
 orbs:
   android: circleci/android@2.1
+  gradle: circleci/gradle@3.0
 
 executors:
   android:
@@ -13,9 +14,11 @@ jobs:
     executor: android
     steps:
       - checkout
-      - run:
-          name: Building the APK
-          command: ./gradlew -s assembleDebug
+      - gradle/with_cache:
+          steps:
+            - run:
+                name: Building the APK
+                command: ./gradlew -s assembleDebug
       - store_artifacts:
           path: app/build/outputs/apk/debug/app-debug.apk
Code language: Diff (diff)

Now, this caches that command in CircleCI: