Automating Proxmox VM Hardware Discovery for cloud-init Migration

Introduction

As we prepare to migrate our virtual machine infrastructure to a fully automated Cloud-Init deployment, we need a clear understanding of the current VM configurations. This includes hardware specs, network interfaces, storage configurations, serial ports, and mounted ISO images. Rather than manually gathering this data, we automated the process using Proxmox's API and jq to filter and format the output.

This script ensures that only the relevant project-related VMs are scanned, specifically those with names containing -core- or -isp-[0-9]-, so we can focus on mission-critical systems.

How the Script Works

The script runs through a few key steps:

1. Query the Proxmox API for a list of all VMs in the cluster.
2. Loop through each VM ID and retrieve its full hardware configuration.
3. Extract important details, such as:
   • VM name
   • CPU and memory allocation
   • Attached disks
   • Network interfaces and MAC addresses
   • Serial ports
   • Mounted ISO images (if any)
4. Filter the results, ensuring only VMs relevant to our project are included.
5. Format the output for readability, making it easy to analyze.

The filtering step ensures we only capture VMs that match specific naming patterns:

• -core-: This identifies essential infrastructure VMs.
• -isp-[0-9]-: This captures service provider instances, where [0-9] is any number.

By focusing on these naming patterns, we exclude irrelevant test or development VMs, streamlining the data collection process.

The Script

#!/bin/bash

for vmid in $(curl -k -s -X GET "https://<PROXMOX_HOST>:8006/api2/json/cluster/resources?type=vm" \
               -H 'Authorization: PVEAPIToken=root@pam!<TOKEN_ID>=<TOKEN_SECRET>' | jq -r '.data[].vmid'); do
    
    VM_DATA=$(curl -k -s -X GET "https://<PROXMOX_HOST>:8006/api2/json/nodes/<NODE_NAME>/qemu/$vmid/config" \
              -H 'Authorization: PVEAPIToken=root@pam!<TOKEN_ID>=<TOKEN_SECRET>')

    VM_NAME=$(echo "$VM_DATA" | jq -r '.data.name')

    # Apply filter: Only include project VMs with '-core-' or '-isp-' in the name
    if echo "$VM_NAME" | grep -qE '(-core-|-isp-[0-9]-)'; then
        echo "----------------------------------"
        echo "VM ID: $vmid"
        echo "$VM_DATA" | jq -r '
        {
          "VM Name": .data.name,
          "CPU Cores": .data.cores,
          "Memory MB": .data.memory,
          "OS Type": .data.ostype,
          "Disks": [(.data | to_entries[] | select(.key | test("^(scsi|ide|virtio|sata)[0-9]+$")) | "\(.key): \(.value)")],
          "MAC Addresses": [(.data | to_entries[] | select(.key | test("^net[0-9]+$")) | "\(.key): \(.value | sub(".*?="; "") | sub(",.*"; "") )")],
          "Serial Ports": [(.data | to_entries[] | select(.key | test("^serial[0-9]+$")) | "\(.key): \(.value)")],
          "CD/DVD Drive": [(.data | to_entries[] | select(.key | test("^ide[0-9]+$")) | "\(.key): \(.value)")]
        }'
    fi
done

Why This Matters for Cloud-Init

Cloud-Init is a powerful tool for automated VM provisioning, ensuring that instances are consistently deployed with the correct configurations. Before migrating, we need to understand the current hardware specs and configurations of our Proxmox VMs so we can replicate them in our Cloud-Init templates.

This script provides a clear inventory of the infrastructure, allowing us to:

• Define accurate Cloud-Init YAML configurations.
• Identify networking needs (MAC addresses, bridges, etc.).
• Ensure that serial consoles and ISO-based installs are properly accounted for.
• Spot inconsistencies in VM setups that need to be standardized.

By automating this process, we save time and reduce human error, ensuring a smooth transition to Cloud-Init-based provisioning.

Next Steps

• Convert the gathered data into Cloud-Init configuration templates.
• Automate Cloud-Init deployments to fully replace manual VM creation.
• Test and validate Cloud-Init automation before rolling it out to production.

This script is the first step in our migration process, ensuring we have an accurate snapshot of our infrastructure before making the leap to fully automated deployments.

Conclusion

Automating Proxmox VM hardware discovery is crucial for migrating to Cloud-Init. This script provides a reliable way to gather necessary details while filtering out irrelevant VMs. By leveraging the Proxmox API, jq, and Linux scripting, we have a scalable and repeatable solution for infrastructure analysis.

As we move toward full automation, this inventory will be the foundation of our Cloud-Init deployment strategy, ensuring every VM is correctly configured from the start.