Init

At this point everyone can understand: when things break, it can make even a bad day a hell of a lot worse. That’s doubly true with technology. Deploying Infrastructure as Code (IaC) can reduce the risk and simplify future deployments.

The general concept of IaC is to automate the provisioning of servers/services/infrastructure. It also can serve as documentation if you are diligent about sticking to building your infrastructure through code and not individualizing configurations using the command line interface (CLI). In general, IaC streamlines the software development lifecycle through consistency and reproducibility of deploying environments and enables Continuous Integration/Continuous Deployment (CI/CD) for developers to deploy their code, testers to test the code and just make life a lot easier, even on those bad days.

As long as you have access to your code (in github/gitlab/etc.), the work required to build/rebuild/scale can be significantly more efficient.

I’ve been using Proxmox now for a couple years, and have worried about what would happen if my server stopped… serving (see what I did there?). I’ve had this project in my head for a while. Essentially, I want to automate my server deployments, whether it’s my ebook server (calibre), media server (JellyFin), DNS server(pi hole), dedicated game server (steam), FreeIPA, or whatever server that I rely on, or don’t. Maybe I just want a quick developer/test environment for me to test something out on occasion? It’s wasteful to keep those Virtual Machines around, and if I can deploy a whole test environment in minutes, what does it matter?

It would also be nice to automate the various workstations I have set up in my proxmox environment. Bonus points for dot file management.

My current Proxmox environment has all been configured manually and would take me days to re-deploy the entire environment if the server shits the bed.

I’ve decided to use Terraform to deploy my infrastructure. Partially because there’s a provider which connects directly to Proxmox, partially because it’s new-ish, and works with a lot of different systems. One quick note, Terraform licensing has changed in the past year. So, to go with my typical focus on Open Source solutions I’ll be using OpenTofu. I’m not entirely familiar with the differences between Terraform and OpenTofu, but I’m assuming the code itself is compatible, even if the command is slightly different (“terraform init” vs “tofu init”).

To completely automate your Infrastructure as Code how I want to also requires configuration management. I hope to cover that at a later date.

This project is to get me comfortable and familiarize myself with Terraform/OpenTofu in my own private cloud, with the expectation to move towards projects within AWS. Hopefully this helps reduce some of your own ‘creative cursings’.

I hope as I journal my adventure, you get an idea of my sense of humor and maybe even pick up on some inside jokes.

So let’s get started turning our messy cloud into a ‘happy little cloud’.

Plan

Set up the Role, User, and API token needed in Proxmox.
Find the Terraform provider needed to connect to our Proxmox environment.
Full in the required configuration information for our provider
Test our connection
Build our first virtual machine “automagically”.
Destroy our VM –for science.
Refine our work
Redeploy
Refine again
Deploy again

Apply-ing Proxmox

We first need to set up our user account in proxmox. Open a shell on the proxmox server, either as root, or as a user with ‘sudo’ permissions. You can follow along and look at how these commands populate within the web interface as well. These steps will only need to be done once, as we will keep using this in the future. I’m using Proxmox 8.2, if you’re using an older or newer version, your mileage may vary.
These steps are done from the Proxmox Shell on a specific node/web interface, or using SSH to a specific node

First, we need to create the role for the Terraform provider and give it the proper privileges by using the following command:

pveum role add TerraformProv -privs "Datastore.AllocateSpace Datastore.Audit Pool.Allocate Sys.Audit Sys.Console Sys.Modify VM.Allocate VM.Audit VM.Clone VM.Config.CDROM VM.Config.Cloudinit VM.Config.CPU VM.Config.Disk VM.Config.HWType VM.Config.Memory VM.Config.Network VM.Config.Options VM.Migrate VM.Monitor VM.PowerMgmt SDN.Use"

2. The next step is to create the user that we will use for Terraform, you can optionally provide a password using “–password <password>”, but we aren’t going to use this account to login. Instead we are going to create an API key for Terraform to use. Notice we are creating the user in the ‘pve’ (proxmox virtual environment) realm, this will not be a pam user –able to log into the server itself. Also be aware, when talking about ‘user-id’ in Proxmox, this will be ‘thebeets@pve’, NOT just ‘thebeets’.

pveum user add thebeets@pve

3. The final step in configuring our Proxmox server for using OpenTofu is going to be creating the API token. ‘killertofu’ is what I named my token. You can name it whatever you want, just be mindful of spaces and special characters. “–privesep 0”, is turning off privilege separation. If you get an error such as ‘user thebeets@pve has valid credentials but cannot retrieve user list, check privilege separation of api token’, this is the reason for it. If you forget this, you can modify the value in the web interface, under ‘DataCenter’ -> ‘Permissions’ -> ‘API Tokens’ -> <token Name> and uncheck the box, or lookup the command to modify the token in the Proxmox shell using ‘man pveum’.

pveum user token add thebeets@pve killertofu --privsep 0

If all goes well, you’ll get an output on your screen, that displays your full token ID and your token value. You need to capture this information –ideally in a secure location. We will use this in our later steps to connect to our Proxmox environment and create resources using OpenTofu

Apply-ing OpenTofu

These steps are done from your local workstation or a Virtual Machine. NOT a Proxmox Node/Shell
So now that we have our Proxmox server configured to work with OpenTofu, lets get started on creating our first project. You will need to install OpenTofu by following the available tutorials/instructions for your Operating System.

Once you have OpenTofu installed lets go ahead and create a directory for our project. You’ll want to either do this in a VM on your existing Proxmox server, or your laptop/desktop. We will not be using the Proxmox Shell for this work. Open up a local shell/terminal (I’m using zsh on a Mac).

# make a directory called 'opentofu' within the current user's home directory to group our opentofu projects together, then create a directory within 'opentofu' for our code to create a proxmox vm. Lastly, using touch, create 2 empty files: main.tf and vars.tf 
mkdir -p ~/opentofu/proxmox-vm
cd ~/opentofu/proxmox-vm
touch main.tf vars.tf

Now lets get started on what we want to do. First step to OpenTofu is to list our required providers. Providers can be found on TerraForm’s Registry. Specifically, we are going to get a Proxmox Provider, if you search for ‘Proxmox’ on the registry site, you’ll find 2. One by BPG, and one by Telmate. The Telmate one seems more current and widely used, we’ll use that one. Clicking on ‘Use this provider’ in the top right should open a dialog window with the code that needs to be copied to use the provider.

terraform {
  required_providers {
    proxmox = {
      source = "Telmate/proxmox"
      version = "3.0.1-rc3"
    }
  }
}

provider "proxmox" {
  # Configuration options
}

Perfect. We now have a starting point. Notice that we have a placeholder for configuration options, these will need to be updated based on your Proxmox configuration. Let’s go ahead and talk about what all we need for this.

Going into the documentation section on the provider page, we’ll see that the options are documented! You’ll also notice that the steps we followed earlier to setup and configure Proxmox are located on here (I’m not a genius, I just RTFM). Albeit, my names are a bit more rockin’, it is typically best to use more descriptive names, like the examples do. Scrolling down the page, you’ll see what information the provider requires: URL, Username, and Password. But we are going to use the API token we used before to connect –so it is a little different. Open up the main.tf file with your favorite text editor and copy the code from the provider information and paste it into your main.tf file. Terraform (and thus OpenTofu) uses Hashi Corp Language (HCL), making anything after a ‘#’ a comment, and not part of the actual code. Now, we are going to tailor the proxmox provider section for our proxmox environment so we can connect to it.

provider "proxmox" {
# This is the url to the proxmox environment, 8006 is the default port that the proxmox web interface runs on, the rest (/api2/json) is the service path
  pm_api_url = "http://<dns-entry-or-IP-of-your-server>:8006/api2/json"

# This turns on debugging
  pm_debug = true

# Use this if your sever does not have https encryption
  pm_tls_insecure = true

# This is the API Token information we got before when we configured Proxmox. pm_api_token_id is the full token name from Proxmox. pm_api_token_secret is the token value from Proxmox.
  pm_api_token_id="thebeets@pve!killertofu"
  pm_api_token_secret="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx"
}

That should cover configuring the connection to our server. Now lets initialize OpenTofu to pull our provider and check our connection is configured correctly.

Terraform follows a 4 step deployment process:

Init
Plan
Apply
Destroy
Section titles make sense now?

From your shell:

# change to our opentofu project (make no assumptions)
cd ~/opentofu/proxmox-vm
# initialize providers
tofu init
# see what OpenTofu plans to do with our infrastructure, in this case, not much. 
tofu plan

With any luck, you’ll get green text notifying you of the successful connection. One error I kept running into was mentioned above with the API token having privilege separation, see above for how to fix that.

Congratulations! You should now have a working connection to your Proxmox environment! We’re one step closer to automating our infrastructure in Proxmox!

So let’s go back to our handy documentation on the Terraform Registry page for our proxmox provider, “Telmate/proxmox”. Under ‘Resources’, on the left side, you’ll see a few options, our goal is to create a Virtual Machine using OpenTofu, so we want to look into the documentation ‘proxmox_vm_qemu’. So let’s look at the information on that page a bit more in detail.

As it calls out, to create a QEMU VM, you need to create a resource block, and it even gives you an example of the code you need for that. Let’s copy that code and build off of that.

Destroy

So we now have an empty container of a VM, there is no disk, there is nothing more than a skeleton. So we have some work ahead of us. But for now, we have a VM that is taking up space in our environment. Let’s destroy what we have:

tofu destroy

Once you execute the command, you’ll see a similar screen to the ‘plan’ and ‘apply’ commands, which tell you what will be built. This time however, it tells you what will be deleted/destroyed. Go ahead and indicate ‘yes’ you want to destroy. Once the command finishes you’ll notice all of the resources you created are no longer there in the WebUI. This is the beauty of IaC, build and destroy at will. Theoretically, only use what you need in the moment.

Wrapping Up

So far we have created a skeleton of a Virtual Machine in Proxmox using OpenTofu/Terraform. In the next post, we will begin covering configuring our Virtual Machine to include disks and network devices.

Deploying Proxmox with OpenTofu: Part I