Day 20: HashiCorp Packer for AMI Automation

Introduction to Packer

What is Packer?
Packer is an open-source tool designed by HashiCorp for creating identical machine images for multiple platforms from a single source configuration. It automates the creation of any type of machine image, including AWS AMI, Docker containers, Azure VHDs, VMware VMX, and more, ensuring that software configurations are consistent across all deployment environments.

Why Use Packer?
The primary advantage of using Packer is its ability to ensure that your application runs identically in different environments by avoiding discrepancies between development, testing, and production. This consistency is crucial in a DevOps practice as it reduces the “works on my machine” problems and simplifies cloud migrations.

Key benefits include:
- Automation: Automates the process of image creation, reducing manual effort and potential errors.
- Immutability: Encourages the use of immutable infrastructure, where changes are made by replacing entire images rather than altering existing ones.
- Version Control: Each image can be versioned and stored, providing a clear audit trail for deployments.
- Integration: Easily integrates with modern CI/CD pipelines, enhancing continuous integration and delivery practices.

Core Concepts of Packer
To effectively use Packer, you need to understand its core components:
1. Templates: These are JSON files that configure the various components of Packer, specifying how the machine images should be built. Templates include definitions for builders, provisioners, and post-processors.

2. Builders: These are responsible for creating machines and generating images from them for various platforms. For example, you can have builders for Amazon EC2, Google Cloud, DigitalOcean, VMware, Docker, etc.

3. Provisioners: Once a machine is up, provisioners install and configure software within the machine, based on scripts or configuration management systems like Puppet, Chef, or Ansible.

4. Post-Processors: These are optional and used to process images after they are built by builders. Examples include compressing files, uploading images to cloud storage, or converting them to different formats.

Setting Up a Basic Packer Template
Here’s a simple example of a Packer template that defines a basic builder and a shell script provisioner to install Nginx on an Ubuntu server:

{
  "builders": [
    {
      "type": "amazon-ebs",
      "access_key": "{{user `aws_access_key`}}",
      "secret_key": "{{user `aws_secret_key`}}",
      "region": "us-east-1",
      "source_ami_filter": {
        "filters": {
          "virtualization-type": "hvm",
          "name": "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*",
          "root-device-type": "ebs"
        },
        "owners": ["099720109477"],
        "most_recent": true
      },
      "instance_type": "t2.micro",
      "ssh_username": "ubuntu",
      "ami_name": "packer-example {{timestamp}}"
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "inline": [
        "sleep 30",
        "sudo apt-get update",
        "sudo apt-get install -y nginx"
      ]
    }
  ]
}

In this template:
- Builders section defines the configuration to build an AWS EC2 instance based on Ubuntu Xenial.
- Provisioners section includes a simple shell script to install Nginx on the instance.

This chapter sets the foundation for understanding how Packer functions and prepares you to delve into more detailed configurations and optimizations in the following chapters.

Setting Up Your Environment

Before you can start creating machine images with Packer, you need to set up your development environment. This chapter guides you through the installation of Packer and setting up the necessary configurations to prepare for your first image build.

Installing Packer
Packer is available for Windows, macOS, and Linux. Follow these steps to download and install Packer on your system:

1. Download Packer:
— Go to the Packer Downloads page on the HashiCorp website.
— Select the appropriate package for your operating system and architecture.

2. Install Packer:
— For Windows: Unzip the package and place the `packer.exe` in a directory included in your system’s PATH.
— For macOS and Linux:
— Unzip the package.
— Move the `packer` binary to a directory included in your PATH, such as `/usr/local/bin`.

3. Verify Installation:
— Open a terminal or command prompt.
— Type `packer` and hit Enter. If Packer is installed correctly, you should see a list of Packer commands and options.

Initial Configuration
After installing Packer, you’ll need to perform some initial setup to ensure that Packer can communicate with the services where you want to build your images. For instance, if you are building AWS images, you need to configure access to your AWS account.

1. Setting Up Environment Variables:
— For AWS, configure your access key, secret key, and default region by setting up environment variables:

export AWS_ACCESS_KEY_ID="your_access_key_here"
export AWS_SECRET_ACCESS_KEY="your_secret_key_here"
export AWS_DEFAULT_REGION="us-east-1"

— Similar environment variables can be set for other cloud providers like Azure or Google Cloud.

2. Creating a Basic Packer Template:
— Create a new file named `simple-template.json`.
— Insert a basic configuration that defines a builder. For this example, we’ll continue with the AWS builder used in Chapter 1.

First Packer Build Command
With Packer installed and your environment configured, let’s run a simple Packer build to ensure everything is set up correctly.

1. Navigate to the Directory:
— Open a terminal and navigate to the directory containing your `simple-template.json`.

2. Build the Image:
— Run the following command:

packer build simple-template.json

— Packer will start the image creation process based on the specifications in your template. It might take a few minutes depending on the complexity of the build.

3.Verify the Output:
— If the build completes successfully, Packer will output the details of the created image, such as its ID and location.

This completes the setup of your Packer environment and confirms that you can build a basic image.

Building Your First Image
In this chapter, we’ll delve into creating your first machine image using Packer, focusing on understanding templates, builders, and provisioners. We will also create a practical example that builds upon the setup from Chapter 2.

Understanding Packer Templates
A Packer template is a JSON file that tells Packer what builders, provisioners, and post-processors to use when creating a machine image. Here’s a breakdown of a basic Packer template structure:

- Builders: Define where and how the machine images are built.
- Provisioners: Define scripts or automation tools used to set up the operating system or install software.
- Post-Processors: Define actions to execute after the machine image is built, such as uploading the image.

Example: Creating an AWS EC2 Instance Image
We’ll build on the initial AWS example, expanding it to include more detailed provisioning.

Template File: vars.json`

{
   "region": "us-east-1",
   "source_ami": "ami-0866a3c8686eaeeba",
   "instance_type": "t2.micro",
   "vpc_id": "vpc-050c410583bc359ab",
   "subnet_id": "subnet-0972cb2e040f53da4"
}

Template File: packer.json`

{
    "_comment" : "Create a AWS AMI with ubuntu",
    "variables": {
      "region": "",
      "source_ami":"",
      "instance_type":"",
      "vpc_id": "",
      "subnet_id": ""
    },
    "_comment1" : "packer build -var \"aws_secret_key=foo\" template.json",
    "_comment2" : "packer build -var-file packer-vars.json template.json",
    "builders": [
  {
    "type": "amazon-ebs",
    "region": "{{user `region`}}",
    "source_ami": "{{user `source_ami`}}",
    "instance_type": "{{user `instance_type`}}",
    "ssh_username": "ubuntu",
    "ami_name": "Surya-Prasad-Build-{{isotime | clean_resource_name}}",
    "vpc_id": "{{user `vpc_id`}}",
    "subnet_id": "{{user `subnet_id`}}",
    "associate_public_ip_address": true,
    "tags": {
      "Name": "Surya-Prasad-Build-{{isotime | clean_resource_name}}"
    }
  }
],
    "provisioners": [{
      "type": "shell",
      "inline": [
        "sudo useradd -m ansibleadmin --shell /bin/bash",
        "sudo mkdir -p /home/ansibleadmin/.ssh  ",
        "sudo chown -R ansibleadmin /home/ansibleadmin/.ssh",
        "sudo touch /home/ansibleadmin/.ssh/authorized_keys",
        "sudo usermod -aG sudo ansibleadmin",
        "echo 'ansibleadmin ALL=(ALL) NOPASSWD: ALL' | sudo tee -a /etc/sudoers",
        "echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIE/vpi6tYRRN2pDylHGl5RlH5IP6VkxthmjCNBAPt413' | sudo tee -a /home/ansibleadmin/.ssh/authorized_keys"
        ]
    },
    {
       "type": "shell",
       "inline": [
        "sudo apt update -y",
        "curl https://get.docker.com | bash"
       ]
    },
    {
      "type": "file",
      "source": "docker.service",
      "destination": "/tmp/docker.service"
    },
    {
        "type": "shell",
        "inline": [
            "sudo cp /tmp/docker.service /lib/systemd/system/docker.service",
            "sudo usermod -a -G docker ubuntu",
            "sudo systemctl daemon-reload",
            "sudo service docker restart"
        ]

    },
    {
      "type": "shell",
      "inline": [
        "sudo apt install -y unzip stress net-tools jq"
      ]
   }
    
]
  }

Running Your Packer Build
To create an AWS AMI with Docker installed:
1. Save the Template:
— Store the above JSON configuration in a file named packer.json`.
2. Execute the Build:
— Open a terminal, navigate to the directory containing your template, and run:

packer validate --var-file vars.json packer.json
packer inspect --var-file vars.json packer.json
packer build --var-file vars.json packer.json

— Packer will initialize the build process based on the AWS EBS builder, provision the instance with docker, and create an AMI.

3. Verify the Image:
— Once the build is complete, Packer will provide the ID of the newly created AMI.
— Log into your AWS console and verify that the AMI exists and is configured with docker.