AWS Tagging for fun & profit

Hey there,

It’s Robin from CFD Engine & two weeks after saying I’d be writing less about AWS, I’m doing it again 🤷‍♂️

If my number one piece of AWS advice is to: “turn it off when you’re done” then my number two is to: “tag your stuff.”

Adding tags to your resources is a super-easy way to get organised, save a bit of time & give yourself the option to see how much you’re spending per-project, per-customer or per-simulation.

Here’s the lowdown, in about 3 mins…

What’s tagging?

Tagging is just like it sounds, adding little bits of metadata to your AWS resources to identify &/or categorise them.

You can tag anything that costs money (which is pretty much everything on AWS) & easily includes the tiny subset that we use for CFD – files, storage & instances.

You can add whatever tags you’d like (each tag is just a name & a value) to create something that suits your specific needs.

Once you’ve added some tags, you can use them in a variety of ways, from simple filtering & cost reporting, through to the basis for complex automation rules.

Why bother?

The number one reason (for me) to tag something, is to be able to tell it apart from another, outwardly-identical version. Which instance is running which job? Which disc has the files that I’m looking for? The cost reporting is just a bonus.

You could keep track of this stuff in a spreadsheet (or in a database) but sometimes just giving things a name tag will do.

It gets more important as your usage (&/or your team) grows. Asking if someone could “check on run 5”, and them knowing which machine it’s on, without having to log into 10 other machines, is a big time saver.

How I use tags

Even in my tiny team-of-one I still find tags handy for organising stuff, especially when I’ve got a few jobs on the go. That said, I don’t go overboard.

I tag instances with the project & the ID of the run they’re working on.

I’ll occasionally tag storage volumes to indicate what’s on them (especially if I have to swap them between machines).

And more recently I’ve started tagging what stage a simulation is at (mesh - solve - post) to give me an idea of how things are going.

I don’t tag individual files in storage though. You can, but I just rely on a simple directory structure to find what I’m looking for (plus I can’t face going back & retrospectively tagging them all 🤫)

I used to tag everything manually, which was pretty hit-&-miss. Since then I’ve added a couple of lines to my Allrun scripts so that jobs can tag themselves (via the AWS’ command-line tools). Now it’s more hit than miss 🎯

Once tagged, this is what I do with the data…

Cost Reports

AWS has a decent cost reporting tool which lets you filter by tags. For example, I can easily pull up a chart of the compute costs associated with a particular project, across the whole of last year…

I also use it to compare projects, to find out how much individual simulations cost, or to spot the outliers that have cost me a packet.

You could use it to re-bill compute costs to clients, but I don’t do that.

A few basic tags can help you keep track of how much you’re spending per project & give you an idea whether you might need to raise your rates (or reduce your costs).

“What’s running?”

Far & away my most frequent use of tags is to show a summary of what’s running, like this 👇

It’s the output of the Describe Instances command, shown in my terminal, as a table.

The addresses are fake, but you get the idea – what’s up? what’s running? & what stage is it at? All without having to log into any machines or webpages. I could add more detail with more tags, but this is a handy little summary & can be useful for spotting possible problems 🔍

Give it a go

TL;DR If you’re running CFD on AWS, you really should be tagging your stuff 👍

Even just clicking on your instances in the web console & giving them a name is worth doing. When you get bored of that, you can add a couple of lines to your run script & automate your tagging using the command-line tools.

Let me know how you get on & what you find when you go digging in your cost reports 🙈

Until next week, stay safe,