Cloud & Engineering

Everett Toews

AWS Account Cleaner

Posted by Everett Toews on 27 September 2022

aws, AWS Account Cleaner, AWSweeper, ReadOnlyAccess

be-a-tidy-kiwi-modernWhether you’re new to AWS and just learning or you’re a seasoned veteran, working with Amazon Web Services (AWS) usually involves extensive experimentation. That means constantly creating, reading, updating, and deleting resources as you experiment. However, sometimes you may forget to delete something or AWS automatically creates associated resources that you might not even be aware of that don’t get automatically deleted. This can all amount to considerable cruft in your account, which can be quite costly in terms of dollars or cognitive load as you have to sift through resources and question whether or not they are actually necessary. Be a tidy Kiwi and keep your AWS account clean!

Choosing an AWS Account Cleaner

be-a-tidy-kiwiThere are a number of tools that can do the job to keep an AWS account clean. I won’t go into an exhaustive comparison but I was primarily looking for a tool that met the following requirements.

  • Works well with AWS resource tags (important for accounts following the best practices for Tagging AWS resources)
  • Supports a majority of AWS resources
  • Reasonably well maintained

In the end, I chose AWSweeper. It is flexible in how it works with resource tags, supports many AWS resources by leveraging the AWS Terraform provider, and is reasonably well maintained with many thanks to its maintainer and contributors!

Using AWSweeper

AWSweeper has good documentation so I’ll just provide my experience with using it.

Installation

By far, the fastest way to start using AWSweeper is via CloudShell. In a CloudShell session, follow the installation instructions. Double-check that you’re using the most recent release.

[cloudshell-user@ip-10-0-189-11 ~]$ curl -sSfL https://raw.githubusercontent.com/jckuester/awsweeper/master/install.sh | sh -s v0.12.0
jckuester/awsweeper info checking GitHub for tag 'v0.12.0'
jckuester/awsweeper info found version: 0.12.0 for v0.12.0/linux/amd64
jckuester/awsweeper info installed ./bin/awsweeper

[cloudshell-user@ip-10-0-189-11 ~]$ awsweeper --version
version: 0.12.0
commit: 09952ce
built at: 2022-02-08T22:59:30Z
using: go1.17.3

Configuration

AWSweeper requires a YAML file to configure it. Keep it simple to start and begin with exactly one resource type.

Create a file called awsweeper.yaml and add this one line.

aws_instance:

Dry Run

Always do a dry-run of AWSweeper first before you do anything else. That gives you a baseline of what resources are already in your account.

[cloudshell-user@ip-10-0-182-207 ~]$ awsweeper --dry-run awsweeper.yaml

• SHOWING RESOURCES THAT WOULD BE DELETED (DRY RUN)

---
Type: aws_instance
Found: 3

Id: i-0250d2cd1c2640b16
Tags: [Name: Important] [Purpose: Prod]
Created: 2022-09-02 04:56:30 +0000 UTC

Id: i-0221ae44df26eba10
Tags: [Name: Temporary] [Purpose: Experiment]
Created: 2022-09-02 04:56:30 +0000 UTC

Id: i-0958016c7117fe5df
Tags: [Name: No Purpose]
Created: 2022-09-02 23:08:49 +0000 UTC

---

• TOTAL NUMBER OF RESOURCES THAT WOULD BE DELETED: 3

Here we see that there are a couple of EC2 instances in this account. They would be deleted if we weren’t doing a dry-run but even when you’re doing a real-run, there’s a confirmation step before anything is actually deleted.

Tag Filters

Using tag filters lets us control what AWSweeper deletes. The tag filters are pretty good but they take some getting used to. When a tag matches, that means the resource(s) are to be deleted. I found I had to do an example table like the one below for a resource to get a feel for how the “NOT” operator works for tag filters.

The “Purpose” table headers below mean whether the EC2 instance has that tag or not.

Filter Purpose: Experiment Purpose: Prod No Purpose tag

- tags:

    Purpose: Experiment

Delete Keep Keep

- tags:

    Purpose: NOT(Experiment

Keep Delete Keep

- tags:

    NOT(Purpose): Experiment

Keep Delete Delete

- tags:

    NOT(Purpose): NOT(Experiment)

Delete Keep  

Tag Filters Example

To use a tag filter, include it below the resource type you want to filter in awsweeper.yaml.

For example.

aws_instance:
- tags:
Purpose: Experiment

That tag filter would delete any EC2 instance tagged with Purpose: Experiment.

[cloudshell-user@ip-10-0-182-207 ~]$ awsweeper --dry-run awsweeper.yaml

• SHOWING RESOURCES THAT WOULD BE DELETED (DRY RUN)

---
Type: aws_instance
Found: 1

Id: i-0221ae44df26eba10
Tags: [Name: Temporary] [Purpose: Experiment]
Created: 2022-09-02 04:56:30 +0000 UTC

---

• TOTAL NUMBER OF RESOURCES THAT WOULD BE DELETED: 1

Including More Resources

Add more resources to your awsweeper.yaml file one by one or in small batches between dry-runs. This enables you to build up your understanding of how AWSweeper will affect the resources. As you go, add tags to your resources and your awsweeper.yaml file as necessary to ensure nothing important gets deleted accidentally.

For example.

aws_instance:
- tags:
Purpose: Experiment
aws_key_pair:
- tags:
Purpose: Experiment
aws_security_group:
- tags:
Purpose: Experiment

 

These tag filters would delete any of those resources with tag Purpose: Experiment.

[cloudshell-user@ip-10-0-182-207 ~]$ awsweeper --dry-run awsweeper.yaml

• SHOWING RESOURCES THAT WOULD BE DELETED (DRY RUN)

---
Type: aws_instance
Found: 1

Id: i-0221ae44df26eba10
Tags: [Name: Temporary] [Purpose: Experiment]
Created: 2022-09-02 04:56:30 +0000 UTC

---

---
Type: aws_key_pair
Found: 1

Id: EC2 Tutorial
Tags: [Purpose: Experiment]

---

• TOTAL NUMBER OF RESOURCES THAT WOULD BE DELETED: 2

 

After many rounds of adding resources from the list of Supported resources, you’ll wind up with a file that looks like awsweeper.yaml.

Conclusion

AWSweeper is a very useful tool but can be quite dangerous too. Take care to always do a dry-run before executing it for a real-run. For maximum protection, create an IAM User with programmatic ReadOnlyAccess to use with AWSweeper.

Personally, I find running AWSweeper fascinating. It gives you some real insight into what AWS is doing and how its services are stitched together.

Of course, it also does an excellent job of keeping your AWS Account free of cruft and helps keep costs down to boot!

 

 

 

If you like what you read, join our team as we seek to solve wicked problems within Complex Programs, Process Engineering, Integration, Cloud Platforms, DevOps & more!

 

Have a look at our opening positions in Deloitte. You can search and see which ones we have in Cloud & Engineering.

 

Have more enquiries? Reach out to our Talent Team directly and they will be able to support you best.

Leave a comment on this blog: