Deploy single container to multiple AWS EB environments.

The deploy job helps make your deployments very easy and quick to configure. However, you might want to write your deployment scripts yourself for added control and customization or simply to bring your existing proven CLI based deployment scripts over to Shippable. This page walks through an example of using the Elastic Beanstalk (EB) CLI to deploy a single container application to multiple EB environments.

One common concept for multiple environments is having a beta environment that receives automatic deployments, and a production environment that is only deployed manually. Each environment might need its own unique parameters and settings, and both environments are likely deploying to completely separate EB environments.

Topics Covered

  • Deploying a single container application to multiple EB environments using EB CLI.

Devops Assembly pipeline

This is a pictorial representation of the workflow required to deploy your application. The larger boxes are jobs and the smaller boxes are the input resources for the jobs. Both jobs and inputs are specified in Shippable configuration files.

Final Pipeline

We will be defining the jobs and resources in a step by step manner below.

Configuration

They are three configuration files that are needed to achieve this usecase -

  • shippable.yml: Resources and jobs are defined in this file.

  • Dockerrun.aws.json: This file specifies the image and environment configuration. Placeholders are defined in this file for the image and environment configuration. These placeholders give us flexibility to use the image and environment configuration that you will define in Shippable configuration files.

Content of Dockerrun.aws.json

{
  "AWSEBDockerrunVersion": "1",
  "Image": {
    "Name": "${IMAGE}",
    "Update": "true"
  },
  "Ports": [
    {
      "ContainerPort": "${PORT}"
    }
  ],
  "environment": [
    {
      "name": "ENVIRONMENT",
      "value": "${ENVIRONMENT}"
    },
    {
      "name": "PORT",
      "value": "${PORT}"
    }
  ]
}
  • config.yml:

In your source code, along side the Dockerrun.aws.json file, create a .elasticbeanstalk directory and create a config.yml file inside it. In our sample, we have a single_container folder that contains our Dockerrun.aws.json for this example. The file tree looks like this:

single_container/
├── Dockerrun.aws.json
└── .elasticbeanstalk
    └── config.yml

1 directory, 2 files

Content of config.yml

branch-defaults:
  default:
    environment: ${DEPLOYEBENVPARAMS_PARAMS_AWS_EB_ENVIRONMENT_SINGLE}
environment-defaults:
  ${DEPLOYEBENVPARAMS_PARAMS_AWS_EB_ENVIRONMENT_SINGLE}:
    branch: null
    repository: null
global:
  application_name: ${DEPLOYEBENVPARAMS_PARAMS_AWS_EB_APPLICATION}
  default_ec2_keyname: null
  default_platform: null
  default_region: ${DEPLOYEBBASICCONFIG_POINTER_REGION}
  instance_profile: null
  platform_name: null
  platform_version: null
  profile: null
  sc: null
  workspace_type: Application

Inside config.yml, you will see placeholders defined for application, environment and region. These placeholders will be replaced dynamically with your Shippable configuration giving you tremendous flexibility and reuse.

Prerequisites

Create a Beanstalk application and environment

If you've already got an environment ready to go, you can skip this section. Select "sample application" while creating the environment, because we'll be updating it with our own image pretty soon anyway. It'll take a few minutes to start.

create a new beanstalk app waiting to create application

Once the creation is done, you should see this: application created

Steps

1. Define deploy-eb-basic-image.

  • Description: deploy-eb-basic-image represents your Docker image in your pipeline. In our example, we're using an image hosted on Amazon ECR.
  • Required: Yes.

  • Add the following yml block to your shippable.yml file.

resources:

  - name: deploy-eb-basic-image
    type: image
    pointer:
      sourceName: "679404489841.dkr.ecr.us-east-1.amazonaws.com/nodeapp"
    seed:
      versionName: "latest"

2. Define deploy-eb-basic-config.

  • Description: deploy-eb-basic-config is a cliConfig resource that references credentials needed to setup a CLI for EB.

  • Required: Yes.

  • Integrations needed: AWS Keys.

    The integration defines the AWS key/secret pair that Shippable platform will use to communicate with EB on our behalf. Make sure that the key has appropriate permissions for the different actions required for EB deployments.

Steps

  1. Create an integration for AWS Keys. Instructions to create an integration can be found here. Copy the friendly name of the integration, in our case we named it aws_keys_int.

  2. Add the following yml block to the existing resources section of your shippable.yml file.

resources:
  - name: deploy-eb-basic-config
    type: cliConfig
    integration: aws_keys_int     # The integration created above
    pointer:
      region: us-east-1           # region where you want to deploy

3. Define deploy-eb-basic-params and prod-params

  • Description: deploy-eb-basic-params and prod-params are params resources that define variables we want to make easily configurable. These variables definitions replace the placeholders in the Dockerrun.aws.json and config.yml files. Furthermore, these params are environment specific allowing each environment to be customized.

  • Required: Yes.

Steps

Add the following yml block in the resources section of your shippable.yml file.

resources:

- name: deploy-eb-basic-params
  type: params
  version:
    params:
      ENVIRONMENT: "sample" # used inside our sample image
      PORT: 80  # tells app which port to listen to
      AWS_EB_ENVIRONMENT_SINGLE: "Sample-env" # for the single-container example
      AWS_EB_APPLICATION: "deploy-eb-basic" # the name you gave your eb application

  - name: prod-params
    type: params
    version:
      params:
        ENVIRONMENT: "prod"
        AWS_EB_ENVIRONMENT_SINGLE: "prod"
  • Description: deploy-eb-basic-params is a params resource that defines variables we want to make easily configurable. These variables definitions replace the placeholders in the Dockerrun.aws.json and config.yml files.

  • Required: Yes.

Steps

Add the following yml block to the resources section of your shippable.yml file.

  - name: deploy-eb-basic-params
    type: params
    version:
      params:
        ENVIRONMENT: "sample" # used inside our sample image
        PORT: 80  # tells app which port to listen to
        AWS_EB_ENVIRONMENT_SINGLE: "Sample-env" # for the single-container example
        AWS_EB_APPLICATION: "deploy-eb-basic" # the name you gave your eb application

4. Define deploy-eb-basic-repo

  • Description: deploy-eb-basic-repo is a gitRepo resource which represents our application repository that has all the source and the configuration files we created earlier. We need this resource to access and replace content dynamically in the Dockerrun.aws.json and config.yml files.

  • Required: Yes.

Steps

Add the following yml block to the resources section of your shippable.yml file.

resources:
  - name: deploy-eb-basic-repo
    type: gitRepo
    integration: dr-github
    pointer:
      sourceName: devops-recipes/deploy-beanstalk-basic
      branch: master

5. Define deploy-beta and deploy-prod

  • Description: deploy-beta and deploy-prod are runSh jobs, which let you run any shell script as part of your DevOps Assembly Line. It is one of the most versatile jobs in the arsenal and can be used to pretty much execute any DevOps activity that can be scripted.

    A couple of points to note:

    • We utilize the built-in shippable_replace utility on the Dockerrun.aws.json file as well as the config.yml file to replace placeholders with the actual configuration.
    • We export the IMAGE env variable using the image resource environment variable in the beta job.
    • To ensure that the same image is deployed to production, we store the image tag in the job state of the deploy-beta job and use it in the deploy-prod job.
    • All the inputs of the deploy-prod are switched off, since we want to manually trigger prod deployment.
  • Required: Yes.

Steps

Add the following yml block to your shippable.yml file.

jobs:
  - name: deploy-beta
    type: runSh
    steps:
      - IN: deploy-eb-basic-image
      - IN: deploy-eb-basic-config
        switch: off
      - IN: deploy-eb-basic-params
        switch: off
      - IN: deploy-eb-basic-repo
        switch: off
      - TASK:
        - script: pushd $DEPLOYEBBASICREPO_STATE/single_container && ls -al
        - script: export IMAGE="${DEPLOYEBBASICIMAGE_SOURCENAME}:${DEPLOYEBBASICIMAGE_VERSIONNAME}"
        - script: shippable_replace Dockerrun.aws.json .elasticbeanstalk/config.yml
        - script: eb deploy
        - script: echo "image=${DEPLOYEBBASICIMAGE_POINTER_SOURCENAME}:${DEPLOYEBBASICIMAGE_VERSIONNAME}" >> $JOB_STATE/$JOB_NAME.env

  - name: deploy-prod
    type: runSh
    steps:
      - IN: deploy-beta
        switch: off
      - IN: deploy-eb-basic-config
        switch: off
      - IN: deploy-eb-basic-params
        switch: off
      - IN: prod-params
        switch: off
      - IN: deploy-eb-basic-repo
        switch: off
      - TASK:
        - script: pushd $DEPLOYEBBASICREPO_STATE/single_container && ls -al
        - script: export IMAGE="${DEPLOYBETA_IMAGE}"
        - script: shippable_replace Dockerrun.aws.json .elasticbeanstalk/config.yml
        - script: eb deploy

We use the ebcli to perform the deployment, since it comes pre-installed on the build image, and it takes care of a lot of the work for us. Since we've manually added the config.yml, and our aws cli is already configured with our credentials, all we have to do is execute eb deploy (-v for verbose mode). This will package and deploy our code automatically based on the settings in our config.yml.

6. Import configuration into your Shippable account.

Once you have the shippable.yml file as described above, commit it to your repository. This repository is called a sync repository. You can then follow instructions to add your assembly line to Shippable.

7. Trigger your pipeline

When you're ready for deployment, right-click on the deploy-eb-basic-deploy job in the SPOG View, and select Run Job.

After you run this job, and you should see your modified files in the console output like this: templated files

Finished Deployment

Sample project

Here are some links to a working sample of this scenario. This is a simple Node.js application that runs some tests and then pushes the image to Amazon ECR. It also contains all of the pipelines configuration files for deploying to Elastic Beanstalk.

Source code: devops-recipes/deploy-beanstalk-basic.