I really like OpenTofu/Terraform, but if you have dealt with IaC you know that a big part of the job is importing infrastructure that was previously clickopsed or created by other means into tfstate. If you have done it, you also know that each resource is imported in a different way, using different IDs, and it is a slow, painful process that gets worse and worse the more resources you manage.

To speed things up a bit, usually I bolt down some shell one-liners, but it is still a very tedious and error prone process. Sure, the most recent versions now have the import {} blocks and they support for_each, but that still doesn’t solve the core problem of each resource having a different way of being imported. You still have to go through the documentation and find the correct import ID for every single thing. Over the years I wished there was an easier way to do this, but I haven’t gotten around to how to actually solve the problem, especially how to compute the IDs necessary for importing each resource. At least not until now!

The Import Headache

Recently I was in the middle of a big project refactor where I had to import a huge amount of resources through different terragrunt units in different accounts and environments. The bash script to assist with the task was getting more and more complex when it occurred to me that most of the information needed to compute the import IDs is often retrievable right from the plan itself. The information that is not there can often be computed from other data in the plan or with some kind of lookup! My brain starts racing, it was that itch that doesn’t want to go away… and I immediately pause what I am doing and start sketching a solution.

Alright… So I can get stuff from the plan, but how do I parse it? Terraform has a tfjson package that contains all the structs to be able to parse the plan, great! But there is actually another problem. How do I know what information I need to build the import IDs? I couldn’t find a way to piggyback on the providers’ code to get such information, so I would need to find a way to parse the providers’ documentation to figure it out… dammit, this is exactly why I never tried to do anything about this in the first place.

What if there was some kind of markdown crunching machine that could crunch all that data from the providers’ documentation and extract what information is needed to build the import IDs? Only if… oooh wait… Don’t they call that Artificial Intelligence these days? Hell yeah! I can just throw an AI agent at it and it will get me all I need. I just need to make sure the code is decoupled and extensible enough so AI agents can safely work and add different providers without stepping on each other’s toes. Cool, I have a plan!

The “Clankers”

After a few hours of bashing my keyboard, I got the core structure of the CLI tool down and working. It was time to get the clankers to do their part! I explained to the agent about the tool I was building, explained how I was planning the tool architecture, and before I even let it go brrr, I asked it to generate an AGENTS.md file. Once that was done, I asked it to crunch the AWS Provider documentation, find a way to extract the information needed to build the import IDs, and code it!

I built a great script! But I deleted it!

I let the agent free to do its thing. I accidentally looked at its reasoning and the damned clanker was building a document cruncher script to parse the markdown files and extract the information. When it finished it happily told me “Hey I built a doc cruncher script to do this, I’ve added the AWS Provider with the necessary logic to build the import IDs, and I’ve cleaned up all the temporary code and the doc cruncher script”.

Wait, you did what? Hell no!!! Bring that thing back! And by the way, build it in a way that is reusable to facilitate not only adding new providers but also updating existing ones when new versions are released with resources added or removed! The clanker happily agreed, brought back the script, and updated the AGENTS.md with the information and the pattern focused on adding new providers.

And the payoff was immediate! With the documentation cruncher and each cloud provider isolated with its own logic, expanding the tool became trivial. I literally just threw the agent at it again and said “Do Google Cloud next! Now do Azure, and now Scaleway.” Within minutes, the agent cloned their docs and generated the code for the thousands of resource ID mappings.

Polishing the Experience

I then focused on the user experience of the tool while actually using it myself in the real world and… Damn! Importing resources into tfstate is now so easy! The import IDs are all extracted from the plan or can be computed through existing information in it. However, certain resources require opaque IDs (like an AWS VPC ID vpc-xxxxxx or an EC2 Instance ID) that simply do not exist in the plan before the resource is imported. For those, tfimport gracefully falls back to the cloud provider SDK in the background (lazy loading), looks up your infrastructure using the tags or names provided in the composition, grabs the real physical ID, and feeds it back into the import logic.

It was then when I started hitting some weird networking issues. Later I found out the true reason for them, but since tfimport was running terragrunt import directly, my mind immediately assumed that I was probably hitting the AWS API rate limits. That brought me to the idea that maybe it would be better to generate a .tf file with import {} blocks. The experience actually became much better, so much so that it became the default behavior of the application.

And you know the best part of building the tooling we actually use? A bit later I realized that I needed a way to ignore certain resources and decided to add a new flag, --ignore. This is because a lot of times we are importing some resources and creating new ones, and we don’t want tfimport to try to import resources that don’t actually exist!

Try tfimport

If importing pre-existing resources in the cloud is a task you do frequently, next time give tfimport a try!

You can find the source code and installation instructions on the official GitHub repository.