For the past few years, I’ve been using Ansible to create reproducible builds on my AWS EC2 infrastructure.

To keep my deployments flexible, I’ve been making use of Ansible’s dynamic inventory. For AWS EC2, this takes the form of an external python script and ini file that can be called via the --inventory (or --i) flag:

ansible -i ec2.py -m ping

By default, the ec2.py script will create a large number of groups that can be used directly by Ansible’s hosts field, including groupings by region, AMI, and tags.

The returned hosts can also be filtered with a variety of options. In the example lines below, I configured the ec2.ini file to filter by the us-west-2 region and a specific tag:

regions = us-west-2
instance_filters = tag:Name=redacted

I can then use the dynamically created tag group name to target specific hosts:

ansible tag_Name_redacted -i ec2.py -m ping

The dynamic inventory script has worked well for years, but when upgrading to Ansible 2.8, I noticed the following deprecation warning:

[DEPRECATION WARNING]: The TRANSFORM_INVALID_GROUP_CHARS settings is set to allow bad
characters in group names by default, this will change, but still be user configurable on
deprecation. This feature will be removed in version 2.10. Deprecation warnings can be
disabled by setting deprecation_warnings=False in ansible.cfg.

While attempting to fix this deprecation, I discovered that the Ansible developers have been hard at work on an official inventory plugin ecosystem, including an EC2 inventory source named aws_ec2.

This plugin replaces the ec2.py and ec2.ini files with a YAML file, such as:

plugin: aws_ec2
regions:
  - us-west-2
filters:
  tag:Name: redacted

It can be called in a manner similar to the external python script:

ansible -i aws_ec2.yml -m ping

There are, however, a few changes in defaults between the external script and the new plugin. These can be easily seen with the ansible-inventory command:

ansible-inventory -i aws_ec2.yml --list

For instance, using the external script, the above filters will create the following host group (among many others), while also returning the public IP of the hosts if available:

"tag_Name_redacted": {
    "hosts": [
        "###.###.###.###"
    ]
}

But the new plugin with the above filters will only return:

"aws_ec2": {
    "hosts": [
        "ip-xxx-xxx-xxx-xxx.us-west-2.compute.internal"
    ]
}

While it might be possible to adapt your deployment script to these changes, the aws_ec2 plugin is highly configurable, and with a few options we can make a true drop-in replacement for the external script:

plugin: aws_ec2
regions:
  - us-west-2
filters:
  tag:Name: redacted
keyed_groups:
  - key: tags.Name
    prefix: tag_Name
hostnames:
  - ip-address

The only issue I had during this process was determining the correct hostnames value to use. By default the aws_ec2 plugin will return dns-name, falling back to private-dns-name. These defaults are not listed on the plugin’s documentation page, and instead I had to find them in the plugin code. Since all my hosts have public IP addresses, I wanted to reproduce the behavior of the external script, and thankfully ip-address is one of many options available for hostnames.

I have already replaced the external script with the new plugin throughout my deployment infrastructure. With a small YML file, I was able to remove several hundred lines of external scripts and configuration that I had to maintain myself.