Skip to content

Undo: remove CertiStack from your systems

This page takes back what the quickstart set up, in the order that keeps the tools working while you still need them: stop anything that would start a run, check nothing is half done, take back the access, then remove the files. Nothing here touches your backups or your source VMs, and a finished run has already removed its sandbox (the VM, overlays, loop devices, SDN zone and VNet). Each heading names the host to run its commands on. The examples use the quickstart's names; use yours.

1. On the controller: stop scheduled runs

If you scheduled runs with automation, remove the schedule and the wrapper now, before any credential goes. For the systemd timer and the cron entry of that page:

sudo systemctl disable --now [email protected]
sudo rm -f /etc/systemd/system/[email protected] /etc/systemd/system/[email protected]
sudo systemctl daemon-reload
sudo rm -f /etc/cron.d/certistack /usr/local/bin/certistack-validate

Then check that no run is going:

pgrep -af 'certistack run'

It prints nothing when no run is going. Wait for a running one to finish, or press Ctrl-C once in its terminal and wait for its cleanup.

2. On the node: check that no run left anything

An interrupted run can leave a protected workspace and a sandbox behind. As root on the node:

ls /var/lib/certistack/workers/
qm list
pvesh get /cluster/sdn/zones --output-format json

workers/ is empty (or missing) when nothing is retained, qm list shows no sandbox VM (IDs from 9100 up), and the zone list shows no zone of your plan (csZone1 in the quickstart). If a workspace is listed, finish its recovery first with the command in Issue 26:

/var/lib/certistack/workers/<run>/certistack-node recover \
  --state-dir /var/lib/certistack \
  --env-file /var/lib/certistack/workers/<run>/runtime.env

If the workspace holds a file credentials-removed.txt, that command cannot run: the node removed the credentials when it could not recover. Start one more certistack run from the controller (it recovers the journal first), or remove the leftover sandbox VM and zone yourself (Issue 27). Do not go on while qm list or the zone list still shows one.

3. On the Proxmox VE node (as root): take back the access

Remove the API token, the user and the role, then look for permissions that stayed:

pveum user token remove certistack-svc@pve automation
pveum user delete certistack-svc@pve
pveum role delete CertiStackRole
pveum acl list | grep -i certistack

The last command prints nothing when every permission went with the user and the role. Deleting the user may print user config - ignore invalid acl token 'certistack-svc@pve!automation' a few times; that is Proxmox noting the permissions of the token you just removed, and it is harmless.

Remove the overlay storage and the worker's own files. On the node only: the controller keeps its reports under /var/lib/certistack/reports, and the same prefix would delete them there. Read the storage's directory first, because pvesm remove forgets it (it is /var/lib/vz/certistack-scratch in the quickstart unless you put the storage on another disk):

grep -A6 '^dir: certistack-scratch' /etc/pve/storage.cfg
pvesm remove certistack-scratch
rm -rf <the path printed above> /var/lib/certistack

Remove the controller's key from root's authorized_keys. Its comment is certistack-controller. On a cluster this file is shared by every node: keep a second root session open while you edit it, and write the file back in place (a plain sed -i can replace the shared link):

grep -c ' certistack-controller$' /root/.ssh/authorized_keys
grep -v ' certistack-controller$' /root/.ssh/authorized_keys > /root/authorized_keys.new
cat /root/authorized_keys.new > /root/.ssh/authorized_keys && rm /root/authorized_keys.new
grep -c ' certistack-controller$' /root/.ssh/authorized_keys

The count is 1 before and 0 after. The packages from quickstart step 2, libguestfs-tools and nftables, stay; remove them with apt remove only if nothing else on the node uses them.

Then, from the controller, confirm the key no longer works (do this before step 5 deletes it):

ssh -i /secure/certistack/id_ed25519 -o IdentitiesOnly=yes -o BatchMode=yes \
  -o StrictHostKeyChecking=yes -o UserKnownHostsFile=/secure/certistack/known_hosts \
  [email protected] true

It must end in Permission denied (publickey...). Host key verification failed means the test did not run.

4. On the Proxmox Backup Server (as root): take back the read-only user

proxmox-backup-manager acl list | grep -i certistack
proxmox-backup-manager user delete-token certistack-ro@pbs ro
proxmox-backup-manager acl update /datastore/datastore1 DatastoreReader --auth-id certistack-ro@pbs --delete true
proxmox-backup-manager user remove certistack-ro@pbs
proxmox-backup-manager acl list | grep -i certistack

The first command shows the datastore path of the permission (add the namespace if you gave setup rbac one); use that path in the third. Deleting the token takes the token's permission with it, but the user's own stays until the third command removes it. The last command prints nothing when everything is gone.

5. On the controller: remove the files

Keep the reports if you need them: they are your record of what each run found.

certistack telemetry reset
rm -f /secure/certistack/controller.env /secure/certistack/id_ed25519 \
  /secure/certistack/id_ed25519.pub /secure/certistack/known_hosts \
  /secure/certistack/pve-root-ca.pem
sudo rm -f /usr/local/bin/certistack

certistack telemetry reset comes first because it needs the program: it deletes telemetry.json (your answer to the first-run question, and the install ID if you had opted in) from your configuration directory.

What stays

  • The reports in /var/lib/certistack/reports and the plans you wrote: yours to keep or delete.
  • The release key ring /secure/certistack/release-signing-key.gpg, the download directory ~/certistack-download (with the certistack/ clone), pve-access.sh and pbs-access.sh, and the /secure/certistack and report directories themselves: plain files and folders; delete them when you like. telemetry reset may leave an empty telemetry.json.lock beside where the state was.
  • The PBS host key in ~/.ssh/known_hosts (from step 4 of the quickstart), and the token secrets, which may be in your shell history and terminal scrollback.
  • The node's packages libguestfs-tools and nftables.
  • If you set up scheduled runs: the certistack system user and the directory /var/lib/certistack/plans that automation created.

If you also followed getting started and made a certistack account on the node, remove /etc/sudoers.d/certistack and the account (userdel -r certistack). If you used its release install, remove /opt/certistack/releases/<version> and the /usr/local/bin/certistack link on the controller.