Jobs are the executable units of your pipelines that can execute any DevOps activity. A simple way to think of it is, if something can execute in the shell of your laptop, it can execute as a Job.
Jobs take Inputs in the form of Resources, execute tasks that perform the operations necessary and then produce a result i.e. Output(s). Now these Outputs can become Inputs to other jobs and so on forming a dependency-based, event-driven DevOps Assembly Line.
Typical examples of jobs are:
- Execute Continuous Integration (CI) on a repository
- Build, Test and Push Docker images to registries like Docker Hub. Quay, JFrog etc.
- Build deployable machine images with tools like Packer
- Provisioning infrastructure using various automation tools like Ansible, Terraform, Chef, Puppet, etc.
- Deploy applications to Container Orchestration Services like Kubernetes, Amazon ECS, Google Container Engine, Azure Container Service,
- Deploy an application version to a VM cluster
- Execute a functional/integration test suite
- Create immutable service definitions so that applications can roll-forward and roll-back
- Deploy applications with Continuous Deployment(CD) tools like Capistrano, Distelli, Ansible etc.
- Implementing deployment strategies like Blue/Green, Upgrades and Replace
- Perform timely backups of DB, key-value stores etc.
- Create or update an application release version
Jobs are defined in the config file
shippable.resources.yml with the syntax below. Please note that this is the complete syntax and an actual job definition is much smaller than shown here.
jobs: - name: <string> type: <job type name> on_start: - NOTIFY: <notification resource name> steps: - IN: <resource> switch: off - IN: <job> - IN: <resource> versionName: <name of the version you want to pin> - IN: <resource> versionNumber: <number of the version you want to pin> - IN: <params/dockerOption/replicas resource> applyTo: - <image/manifest/release> - <image/manifest/release> - IN: <loadBalancer resource> applyTo: - manifest: <manifest> image: <image> port: <number> - IN: <gitRepoResource with buildOnPullRequest: true> showBuildStatus: true - IN: <manifest/release> force: true - TASK: - script: <any shell command> - script: <any shell command> - OUT: <resource> - OUT: <resource> replicate: <IN resource> - OUT: <resource> overwrite: true on_success: - script: echo "SUCCESS" on_failure: - script: echo "FAILED" - NOTIFY: <notification resource name> on_cancel: - script: echo "CANCEL" always: - script: pwd
For more information, read Job Anatomy
When does a Job execute?
Jobs can be triggered in multiple ways, including both automated and manual triggers. For example, consider the configuration below for my-deploy:
In this configuration, Job-4 will be triggered in one of 4 ways:
- A previous job in the Assembly Line changes an
INresource, i.e. Job-4 will be triggered every time Job-3 updates resource-2.
INjob successfully completed execution, i.e. Job-4 will be triggered every time Job-2 succeeds. A couple of exceptions to this rule are:
- If any Jobs defined as
INs are currently processing, have queued builds or are in a failed state. So in the example above, if Job-1 is in failed state or is queued, Job-4 will not trigger even when Job-2 completes successfully.
- If the Job is in inconsistent state due to dependency failures
- If any Jobs defined as
- You commit to a trigger that is an
INfor the job, i.e. trigger-1 in this example
- Manual trigger: You right click on Job-4 in the SPOG UI and click on Run or Select Run for Job-4 in the Jobs list in the Grid view.
Job-1 will not trigger Job-4 automatically since the dotted line between them indicates that automatic trigger has been switched off.
Please note that changing some resources like cluster, replicas, or dockerOptions manually through a yml commit will not automatically trigger a job. This behavior is meant to prevent unexpected pipeline behavior, since a single commit can contains changes to several resources and cause several trigger points in the pipeline. If you want your job to be triggered when resources are manually edited in the yml, you can add a trigger input for the job and include a change to the trigger resource in the commit every time you want to automatically run your job.
What happens when a Job executes?
When triggered, a Job does the following:
- Get the latest versions of
INresources or latest state information from an
INjob. If an input resource version is pinned, then use that version instead of latest.
- Execute all commands to perform the activity. The set of commands depends on job type.
- Update all
Every time a job executes, a new immutable version is created. This makes it easy to 'replay' older job versions, though you should do this only after verifying that the old input values will not create problems in your Assembly Line.
Supported job types
Shippable supports the following jobs types:
|deploy||Deploy apps/services to Container Platforms or VM clusters|
|manifest||Create App/Service definition (configuration) that is immutable|
|provision||Provision specific resources needed by Container Orchestration Platforms|
|release||Release management for Apps/Services|
|jenkinsJob||Execute a Jenkins Job from Assembly Lines|
|runCI||Execute Shippable CI Job|
|runSh||Execute any Shell command or scripts or supported Command Line Interface commands|
If you need a job that is not listed above, send us an email at firstname.lastname@example.org