Deploying to Amazon ECS

Components

The deployment of Docker containers to the Amazon Elastic Container Service (ECS) involves several major components as follows:

Cluster – a logical grouping of container instances that you can place tasks on (see the next three), see Scheduling Amazon ECS Tasks
Task Definition – a description of an application that contains one or more container definitions, see Amazon ECS Task Definitions
Task – an instantiation of a task definition that is running on a container instance, see Amazon ECS Tasks
Service – a scheduler that ensures that the specified number of tasks are constantly running and rescheduled on failure, and optionally registered against an Elastic Load Balancing load balancer, see Amazon ECS Services

On this page:

Related Pages:

Workflow

The typical deployment workflow comprises two optional and one mandatory phases:

(Optional) Create an ECS Cluster and ECS Container Instances

Run containers serverless via AWS Fargate

Rather than provisioning a cluster of dedicated EC2 instances, you can meanwhile facilitate AWS Fargate to Run containers without managing servers or clusters:

AWS Fargate is a technology for Amazon ECS and EKS^* that allows you to run containers without having to manage servers or clusters. With AWS Fargate, you no longer have to provision, configure, and scale clusters of virtual machines to run containers. This removes the need to choose server types, decide when to scale your clusters, or optimize cluster packing. AWS Fargate removes the need for you to interact with or think about servers or clusters. Fargate lets you focus on designing and building your applications instead of managing the infrastructure that runs them.
[...]

An ECS cluster is the target environment to deploy Docker containers to. If you have explored ECS already, you might have a Default cluster available already in your AWS account. If you do not have one ore more clusters provisioned already, or if you want to isolate the additional containers in a separate cluster, you might need to create one first, which can be achieved in various ways, for example:

You can Launch one or more Amazon ECS Container Instances via the AWS Management Console
- By default, your container instance launches into your default cluster. If you want to launch into your own cluster instead of the default, choose the Advanced Details list and paste the following script into the User data field, replacing your_cluster_name with the name of your cluster.
You can create the ECS Cluster and ECS Container Instances with Tasks for AWS
1. Create an ECS Cluster with the Amazon ECS Cluster task.
2. Create the ECS Container Instances with an appropriate AWS CloudFormation stack via the AWS CloudFormation Stack task - AWS provides two reference architectures as a starting point:
  - Deploying Microservices with Amazon ECS, AWS CloudFormation, and an Application Load Balancer
  - Powering your Amazon ECS Cluster with a mix of Amazon EC2 Spot Instances and Amazon EC2 OnDemand Instances with independent AutoScaling

(Optional) Build and push a Docker image to a Docker registry

Your are going to deploy a Docker container based on a Docker image. This image might exist already, or you might want to create it during the build. The latter can be achieved with the Docker task in Bamboo (available as of Bamboo 5.8), which allows you to:

Build a Docker image
Run a Docker container
- While useful to facilitate Docker within the build itself, running a Docker on the Bamboo agent by means of this action is usually not reasonable for production scenarios - you most likely want to deploy either a single Docker container or an entire service comprised of Docker containers by means of e.g. Tasks for AWS instead, see below.
Push a Docker image to a Docker registry
- You can also use the Amazon ECR Credentials task to push a Docker image to the Amazon Elastic Container Registry (Amazon ECR) with the native Bamboo Docker task - refer to How to push a Docker image to a repository in your Amazon ECR registry with the Bamboo Docker task for details.

Deploy a Docker image to ECS

This is the main and mandatory activity for using Amazon ECS. The three main options for deploying Docker containers to ECS are described in Scheduling Amazon ECS Tasks and available within Tasks for AWS accordingly:

Register a task definition - all three of the following task scheduling options require an available Amazon ECS Task Definition, which can be registered with the Amazon ECS Task Definition task

Schedule a service - this is provided by the Create Service action of the Amazon ECS Service task

Ideally suited for long running stateless services and applications

The service scheduler is ideally suited for long running stateless services and applications. The service scheduler ensures that the specified number of tasks are constantly running and reschedules tasks when a task fails (for example, if the underlying container instance fails for some reason). The service scheduler optionally also makes sure that tasks are registered against an Elastic Load Balancing load balancer. You can update your services that are maintained by the service scheduler, such as deploying a new task definition, or changing the running number of desired tasks. For more information, see Services.

Run a task - this is provided by the Run Task action of the Amazon ECS Task task

Ideally suited for batch jobs

The RunTask action is ideally suited for processes such as batch jobs that perform work and then stop. RunTask randomly distributes tasks across your cluster and tries to minimize the chances that a single instance on your cluster will get a disproportionate number of tasks. For example, you could have a process that calls RunTask when work comes into a queue. The task pulls work from the queue, performs the work such as a data transformation, and then exits. For more information, see Running Tasks

Start a task - this is provided by the Start Task action of the Amazon ECS Task task

Advanced option for leveraging custom schedulers

In addition to providing a set of default schedulers, Amazon ECS also allows you to write your own schedulers that meet the needs of your business, or leverage third party schedulers. The ECSSchedulerDriver is an open source proof of concept that shows you how can integrate Amazon ECS with third-party schedulers; in this case, with the open source Apache Mesos framework. To write your own scheduler, you can use the Amazon ECS List and Describe actions to get the state of your cluster and then use the StartTask action to place your tasks on the appropriate container instance based on your business and application requirements. The StartTask action is available in the AWS CLI, the AWS SDKs, or the Amazon ECS API. For more information, see StartTask in the Amazon Elastic Container Service API Reference

Alternative workflow via CloudFormation

As an alternative to using the Amazon ECS tasks built into Tasks for AWS, it is also possible to provision most ECS components directly via the AWS CloudFormation Stack task and the corresponding CloudFormation resource types:

A typical workflow based on CloudFormation might look as follows:

(optional) create/update the CloudFormation stack that provisions the ECS cluster and ECS container instances – see the Create Stack action of the AWS CloudFormation Stack task
1. You can use the Update stack, if it already exists option to reuse the same Bamboo job for both actions
create/update the CloudFormation stack that provisions the ECS task definition and service – see the Create Stack action of the AWS CloudFormation Stack task
1. You can use the Update stack, if it already exists option to reuse the same Bamboo job for both actions

Frequently Asked Questions (FAQ)

Atlassian account required

Due to a regression within Questions for Confluence Cloud, you need to log in with your Atlassian ID (or sign up for a new account) to access these automatically curated FAQs (refer to UAA-312 for the background).