Caching can have a dramatic impact in build durations since it provides a way to preserve dependencies and packages between builds so you don't spend the time installing them for each build. If caching is turned on, it is updated for every build and is available to subsequent builds.
We create a tarball of whatever needs to be cached and upload it to S3 at the end of the build. The roundtrip to and from S3 means that caching is very useful in specific scenarios and not so much in others:
- Files that take a long time to install benefit from caching. So anything related to bundler, npm, composer, pip, etc are great candidates for caching.
- Files that take a long time to download but installed quickly do not benefit from caching since it takes as much time to download from S3 as from the original source. Examples are compiled binaries, JDK packages, etc.
You can turn on caching for your builds by including
cache: true in the
build section of your shippable.yml. This will cache contents of the build directory $SHIPPABLE_BUILD_DIR.
build: cache: true
You can also choose to cache specific folders instead of the entire build directory by using the
The cache_dir_list is an array of absolute paths of the folders that needs to be cached. Please note that you still need the
cache: true in your yml:
build: cache: true cache_dir_list: - absolute path of dir1 - absolute path of dir2 - absolute path of dir3
1. Caching node modules
To cache just node modules, use the snippet below in your
build: cache: true cache_dir_list: - $SHIPPABLE_BUILD_DIR/node_modules
2. Caching ruby gems
bundle install command can take a while depending on what needs to be installed. To avoid paying this installation tax for each build, you can cache your gems using the
yml snippet below as a guide:
build: ci: - bundle package --all - bundle install --gemfile="Gemfile" cache: true cache_dir_list: # replace 2.2.2 with the ruby version you need to cache - /usr/local/rvm/gems/$(rvm strings 2.2.2)/gems - /usr/local/rvm/gems/$(rvm strings 2.2.2)/specifications - $SHIPPABLE_BUILD_DIR
If the above directories don't work for you, you should check where the gems are installed in your container. To do this, cd /usr/local/rvm/gems and see which directories are in there and need to be cached.
3. Per-branch caching
Caching per branch is currently unsupported, so your cache will be updated for each build regardless of branch. Please open a support issue if this is important for your workflow.
4. Caching for matrix builds
If you're running multiple builds per trigger, aka matrix builds, the cache will be updated for each build in the matrix and available for subsequent builds.
You can clear cache in one of two ways:
[cache_reset]in your commit message.
- Clicking on the Clear cache in your Project Settings UI.
In both cases, your cached content will be deleted. This is a one time action, so if cache is still set to true in your yml, the build will generate a new cache which will be used for subsequent builds.
This method is the best way to update your cache, if required.
Removing unwanted files between builds
When caching is enabled, the entire build directory including artifacts (when
cache: true is used) or the entire folder contents (if configured for specific folders) is cached.
This may include unwanted files.
Unless these unwanted files are removed, your tests could fail during the CI build, as Shippable updates the cached directory with the new commits.
Remove the unwanted files in one of the two ways:
git cleancommand: In your
git cleanin the
cisection using the following format:
git clean -fremoves all untracked files not ignored by
git clean -f -Xremoves all untracked files included in
git clean -f -xremoves all untracked files.
Add commands such as
rmto remove the unwanted files in your