All posts
Observability

What Is Actually Running in Your AWS Account?

DevLift Engineering6 min read

Ask an engineer what is running in their AWS account and watch what happens. They will open the console. Then they will pick a region. Then a service. Then they will say "well, in this region, for EC2, there are eleven."

None of that is an answer, and it is not their fault. The console is organised the way AWS is organised: by service, then by region. The question is organised the other way round.

This matters beyond tidiness. You cannot secure, price or decommission what you cannot enumerate, and every one of those problems eventually lands on the same missing list.

Why does nobody know?

Four structural reasons, none of them about competence.

Everything is regional. A resource in ap-south-1 is invisible from a console session in us-east-1, and there is no "all regions" view for most services. Teams find things in regions nobody remembers choosing, usually created during an evaluation two years ago.

Creation paths outnumber deletion paths. Terraform, the console, a CDK stack, a Lambda that provisions things, a support engineer during an incident, a CI job. Every one of those creates. Almost none of them delete.

Tags are aspirational. Every organisation has a tagging policy. Every organisation has resources that predate it, resources created during incidents, and resources created by services on your behalf, which get whatever tags that service felt like applying. An inventory built on tags describes the tagged subset and quietly omits the rest, which is exactly the part you needed.

Terraform state is a partial map. It is authoritative for what it manages and completely blind to everything else. The gap between "in state" and "in the account" is where most surprises live, and nothing tells you the size of that gap by default.

What does the console not tell you?

Three things specifically, and they compound.

What you are not looking at. You have to know a service exists to check it. Nobody audits Kinesis if nobody remembers using Kinesis, and that is exactly where an orphaned stream sits billing quietly for three years.

What is attached to what. A security group's page tells you its rules. It does not lead with which resources use it, so "is this safe to delete" turns into an investigation. The same is true of subnets, roles and volumes.

What is idle. Running and useful look identical in a list. An RDS instance with zero connections for four months renders exactly like your primary database.

Where do you actually start?

Four sources, in order of how much they cost you to use.

The bill, first. More on this below, because it is underrated.

The Resource Groups Tagging API. GetResources returns tagged resources across services in a region. It is the fastest broad sweep available, and its limitation is in the name: untagged resources are absent, and not every resource type supports tagging. Use it for breadth, not for completeness, and iterate regions explicitly.

AWS Config. The closest thing to a real inventory: it records resource configurations and their history, including relationships between resources. list-discovered-resources gives you the enumeration; the relationship data is what answers "what is attached to this". Two caveats: it has to be enabled before it records anything, so it cannot tell you about last year, and it carries a per-item cost that surprises people who enable it account-wide without checking.

Per-service API calls. The fallback for whatever the above misses. Tedious, complete, and worth scripting once for the services you actually use rather than all of them.

Why is the bill the best inventory you have?

Because it is the only source with no opinion about how a resource was created.

Cost and Usage Reports list what you are being charged for, broken down by service, region, usage type and resource ID where available. A resource created by hand, in an unusual region, with no tags, by someone who left the company, appears in the bill exactly like everything else. It has to: AWS wants to be paid for it.

Start there and work backwards. Group by service and region, sort by cost, and go down the list asking "what is this and who owns it". The expensive unknowns surface immediately, which is also the correct order to investigate them in.

Two limitations to know. Free resources are invisible, and some of those matter for security even at zero cost, such as security groups, IAM roles and unattached elastic IPs in certain configurations. And resource level detail requires the detailed CUR rather than the summarised console view.

This is the same data that answers why the bill grows every quarter, read for a different purpose. There, you are looking for waste. Here, you are looking for existence.

What about things Terraform does not manage?

Measure the gap rather than assuming it.

Take the resource list from discovery, take the resource list from state, and subtract. What remains is unmanaged infrastructure, and it sorts into three piles:

Should be managed and is not. Created by hand, still needed. Import it into state, or accept that it is undocumented and will be deleted by someone eventually.

Should not exist. Leftovers from evaluations, incidents and migrations. This pile is always bigger than expected and is where the easy cost savings are.

Managed by something else. Created by EKS, by a Lambda, by another team's stack. Not a problem, but worth labelling so the next person does not investigate it again.

The size of that first pile is the honest measure of how much your infrastructure as code actually covers, and it is usually lower than the number people quote.

How do you keep the inventory from rotting?

The one-off audit is the trap. It is genuinely useful, it takes a week, it produces a spreadsheet, and it is stale within a fortnight because provisioning did not stop while you were counting.

Three habits make it durable:

Run it on a schedule and diff it. The list is mildly interesting. The change between last week's list and this week's is what you actually want: what appeared, what disappeared, what changed. A weekly diff to a channel gets read; a monthly spreadsheet does not.

Alert on the diff, not the state. "Four new resources in eu-west-2, none tagged" is actionable. "There are 1,847 resources" is wallpaper.

Make new resources cheap to attribute. Enforce tags at creation through policy rather than auditing them afterwards. Every untagged resource is a future investigation, and the investigation costs more than the tag would have.

What should you do with what you find?

In this order, because the ordering is what makes it finish.

  1. Anything unknown and expensive. Highest value, smallest list. Start here for the same reason you start a cleanup with the largest item.
  2. Anything unknown and internet facing. A security question rather than a cost one, and worth checking before the cheap-and-idle work.
  3. Anything idle. Zero connections, zero invocations, zero traffic for a defined period. Stop before deleting, wait, then delete: an idle resource that turns out to matter announces itself within a week.
  4. Anything untagged but legitimate. Tag it now while you know what it is. This is the step everybody skips, which is why the next audit is as hard as this one.

Resist deleting anything on the first pass purely because nobody claimed it. Silence means the owner is on holiday at least as often as it means the resource is abandoned.

The short version

The console cannot answer the question because it is organised by service. The bill can, because it is organised by what exists.

Start with cost, widen with the tagging API and Config, subtract Terraform state to find the unmanaged remainder, then run the whole thing weekly and read the diff rather than the list.

The value is not in the first inventory. It is in the second one, which tells you what changed.


DevLift discovers what already exists in an account, including the resources no Terraform state knows about, and renders the topology rather than a flat list. Book a walkthrough, or read why the bill grows every quarter.

See what this looks like on your own cloud account

DevLift's agents run continuous cost, drift and compliance detection across AWS, Azure and GCP, and propose fixes as reviewable changes, not dashboards. A walkthrough takes 30 minutes.

Schedule a demo

Keep reading