Most guides on the internet show you how to upgrade Proxmox 5.4 to 6.0 via the built-in mechanism. However, for major version upgrades, I prefer to do an installation from scratch, NOT applying the update via apt-get upgrade. This method of upgrading allows me to clean up any crud that has accumulated over the years. Lastly, this guide will cover how to backup your VMS before upgrading to Proxmox 6.0.

System hardware

• CPU: Intel Xeon CPU E5-1620 v3 @ 3.50GHz
• RAM: Crucial 128GB (8 x 16GB) 288-Pin DDR4 SDRAM ECC DDR4 2133 (PC4 17000) Server Memory Model CT4K16G4RFD4213
• Network card: Intel PRO/1000 PT – 4 ports
• Storage
  • Proxmox OS disk
    • Kingston SV300 120GB SSD
  • ssd-tank ZFS pool
    • Samsung 860 EVO 4TB SSD
  • hdd-tank ZFS pool
    • 2 xWestern Digital WD4000 4TB
    • Crucial MX500 240GB SSD as cache drive

Optional: Backing up VMs

Create a backup directory

1. Login into Proxmox
2. Select “Datacenter” on the left
3. Select “Storage”
4. Select “Add” then “Directory”
   1. Enter “VM-backups” into ID
   2. Enter “<ZFS disk drive folder path – I am going to use my hdd-tank ZFS pool>” into Directory
   3. Select “VZDump backup file” for Content
   4. Select “Add”

Backup VM

1. Select a VM
2. Select “Backup”
3. Select “Backup now”
   1. Select “VM-backups” for Storage
   2. Select “Stop” for Mode
   3. Select “LZO (fast)” for compression
   4. Leave e-mail blank
   5. Select “Backup”

Write Proxmox ISO to USB on macOS

1. Open a web browser and browse to “https://www.proxmox.com/en/downloads”
2. Download Proxmox VE 6.0
3. Plugin a USB to your macOS machine
4. Open Disk utility
5. Select the USB drive on the left
6. Select “Erase” at the top
   1. Enter “proxmox-usb” for
   2. Select “MS-DOS (FAT)” for format
   3. Select “Erase”
7. Open a terminal
8. cd ~/Downloads
9. hdiutil convert -format UDRW -o proxmox-ve.dmg proxmox-ve_*.iso
10. diskutil list
    1. Get file path of USB drive
11. diskutil unmountDisk /dev/disk<X>
12. sudo dd if=~/Downloads/proxmox-ve.dmg of=/dev/disk<X> bs=1m
    1. This process will take roughly ~10mins
13. Eject USB drive

Installing Proxmox

Boot from USB

1. Insert USB into server
2. Power up server and boot from USB
3. Select “Install Proxmox VE” from the menu
4. Accept license agreement
5. Install
   1. Select “<target disk>” to install Proxmox
      1. In my case, I will be installing on my Kingston SSD
6. Location and time zone
   1. Enter “<country> for Country
   2. Select “<timezone>” for timezone
   3. Select “<keyboard layout>” for keyboard layout
7. Administrator password
   1. Enter admin password
   2. Enter admin password again
   3. Enter an e-mail for the admin user
8. Network configuration
   1. Select “<network interface>” for management interface
   2. Enter “<FQDN of proxmox>” for hostname
   3. Enter “<Static IP addr>” for IP address
   4. Enter “<Netmask>” for netmask
   5. Enter “<gateway>” for gateway
   6. Enter “<DNS server> ” for DNS
9. Summary
   1. Select “Install”
   2. Reboot

Setup Proxmox

Setup ZFS pool storage for SSD

Destroy original pool

1. Open a terminal and SSH into the Proxmox machine
2. zpool import
   1. Get pool name of SSD
3. zpool import -f <ZFS pool name for ssd>
4. zpool destroy <ZFS pool name for ssd>
5. zpool import
   1. Confirm pool was deleted
6. cfdisk <SSD disk /dev/sd<X> >
   1. Delete all partitions
   2. Write the changes
7. reboot

Create ZFS pool

1. Login into Proxmox via the webgui
2. Select “<Proxmox node>” under Datacenter
3. Select “Disks” then “ZFS”
4. Select “Create ZFS” at the top
   1. Enter “ssd-tank” into Name
   2. Check “Add storage”
   3. Select “Single Disk” for RAID level
   4. Select “On” for compression
   5. Enter “12” for ashift
      1. How to correctly configure ZFS ashift
   6. Select “<SSD disk /dev/sd<X> >” in the device section
   7. Select “Create”

Create ZFS directory for ISOs

1. Select “Datacenter” on the left
2. Select “Storage”
3. Select “Add” then “Directory”
   1. Enter “ISO-storage” for ID
   2. Enter “/ssd-tank” for directory
   3. Select “ISO images” for content
   4. Check “Enabled”
   5. Select “Add”
4. Select “ISO-storage” on the left
5. Select “Content”
6. Select “Upload”
   1. Select ISO file from local machine
   2. Select “Upload”

Create ZFS directory for VMs

1. Select “Datacenter” on the left
2. Select “Storage”
3. Select “Add” then “ZFS”
   1. Enter “ssd-tank-thick” for ID
   2. Select “ssd-tank” for ZFS pool
   3. Select “Disk image” for Content
   4. Check “Enabled”
   5. DEselect “thin provision”
   6. Enter “4k” for block size
   7. Select “add”
4. Repeat step 3 above but instead enter “ssd-tank-thin” for ID and CHECK “thin provision”

Optional: Mount pre-existing ZFS share

1. Open a terminal and SSH into the Proxmox machine
2. zpool import
   1. This should show the old ZFS pools
3. zpool import -f <ZFS pool name>
   1. Mine is hdd-tank
   2. The ZFS pool should be mounted to /hdd-tank
4. If you browse to Proxmox via a web browser
5. Select “<Proxmox node>” on the left
6. Expand “Disks” and select “ZFS”
   1. Your pools should pop-up

Optional: Mount VM backups

1. Select “Datacenter” on the left
2. Select “Storage”
3. Select “Add” then “Directory”
   1. Enter “VM-backups” into ID, like above
   2. Enter “<ZFS disk drive folder path>” into Directory
   3. Select “VZDump backup file” for Content
   4. Select “Add”

Restore VMs from backups

1. On the left select “VM-backups”
2. Select a VM backup and then select “Restore” at the top
   1. Select “<ZFS pool>” for Storage
   2. Leave the VM ID as the default
   3. Select “Restore”
      
   4. Delete backup

Delete old HDD ZFS pool

Delete old HDD ZFS pool

1. Select Datacenter
2. Select “Storage”
3. Remove “VM-backups”
4. Open terminal
5. SSH into proxmox
6. zpool destroy <ZFS HDD pool name>
7. cfdisk <HDD disk /dev/sd<X> >
   1. Delete all partitions
   2. Write the changes

Create HDD ZFS pool with SSD cache

1. Open terminal
2. SSH into Proxmox
3. lsblk -l
   1. List all drives
4. zpool create tank mirror <first mirror drive> <second mirror drive> cache <cache drive>
   

Create HDD VM directory for VMs

1. Select Datacenter on the left
2. Select “Storage”
3. Select “Add” then “ZFS”
   1. Enter “hdd-tank-thick” for ID
   2. Select “hdd-tank” for ZFS pool
   3. Select “Disk Image” for content
   4. Check “Enable”
   5. DEcheck “Thin provision”
   6. Enter “4k” for block size
   7. Select “Add”
4. Repeat step 3 but set ID to “hdd-tank-thin” and CHECK “thin provision”

Setup APT repo

1. Open terminal
2. SSH into Proxmox
3. vim /etc/apt/sources.list
4. Replace deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise with deb http://download.proxmox.com/debian/pve buster pve-no-subscription
5. apt-get update -y && apt-get upgrade -y && reboot

References

• Download Proxmox software, datasheets and documentation
• How to backup and transfer a Proxmox VM to another Proxmox Node
• Upgrade from 5.x to 6.0
• Creating and Destroying ZFS Storage Pools
• Creating a ZFS Storage Pool With Cache Devices
• How to correctly configure ZFS ashift
• cannot connecto to (https) enterprise.proxmox.com/
• Proxmox – Package Repositories