Goals + Recap

  • Got Ansible Pull working, so now I can update a playbook and the VM fetches it and does what I tell to! Who’s a good VM?
  • Struggled with getting LEGO to generate a first time cert. And then like a ignorant fool, I made too many requests of Letsencrypt and got locked out. Shoulda done dry runs! I forgot! Will simply copy over my certs from ye olde plaice and then figure out a way to do the renew script. Even for later, methinks I will leave initial certificate generation to be a manual thing. For the big AWS project, I will figure something else out.
  • Will focus on getting Lego and certs working and getting an app or two at most deployed on the new production cluster.
  • Note to Self: Quit at 3p. Get back to life.

The Big Plan

  • The plan is to redo the cluster again and do my own instance of
    • K3s
    • Sealed Secrets
    • Certmanager (Not using it)
    • Letsencrypt (using pre existing Letsencrypt certs)
    • Get Traefik Ingress to work
    • Figure out a way to get certs automatically into the cluster
  • And once that is done, figure out an app to move (Miniflux or Hedgedoc?); 2025-12-03: Kanboard it is!
  • Begin by moving (lifting and shifting in popular parlance) Kanboard to the cluster
    • Cert will probably be needed (Wildcard cert works now, just like it does without the cluster)
    • Convert a docker-compose to kubernetes manifests
    • Learn how to configure an app with code
    • Learn how to store data and back it up
    • Figure out secrets, if there are any (for now sealed secrets ok, figure out vault and vault injection later)
    • Learn how to tunnel through and reverse proxy
    • Make Kubernetes manifests work with flux
    • Figure out how to automate deployment of manual manifests
    • Figure out how to migrate there if there is any in an old app
    • Figue out how to automate updation of images in manual manifests
    • Get another app (Miniflux) deployed
    • Figure out what needs to happen as part of the lifecycle. What you want in the cluster, what stays out, do they intersect, how do updates of cluster happen? VM (node) updates as well?
    • Then begin to think along the lines of Live Deploys. Prototype locally and once it works, migrate to production immediately
    • Convert Kubernetes manifests to Helm Charts (optional, based on energy)
  • Go live! Git is source of truth. Two repos.
    • One for the Main node and its update
      • Terraform will provision node and install package, setup firewall
      • Figure out how to get Terraform to get the node talking to the git forge
      • Structure repo, copy every thing node related there, and make sure stuff gets updated periodically and idempotantly, via ansible pull and a systemd timer
    • The other one for k3s and flux
      • Convert everything I have done locally to run on prod. Add more steps as you do them below

2025-12-10 09:50

  • Paused a systemd service via Ansible-Pull. Feels very cool!
  • 2025-12-10 10:21: I keep getting distracted with the bigger picture and what ifs rather than focussing on what is right in front of me.
  • 2025-12-10 13:00: Lots of ansible work. Good progress. Lunch break. I used to think that Ansible roles were too granular and now I seem to be doing the same with my tasks 😂

2025-12-10 14:15

  • Will just copy the certs for now
  • 2025-12-10 17:23: Got the plays structured nad worknig just the way i want
    Got syncthing installed, service installed, and working through the firewall.
  • 2025-12-10 18:14: Copied files and got sealed secrets up and runnig within seconds!
  • 2025-12-10 18:35: Got Kubeseal installed via ansible, without a single hitch!
  • 2025-12-10 19:24: Manually put in the secret to the cluster. Will look at automation after a few days when Letsencrypt decides to talk to me.
  • 2025-12-10 19:25: And just like that Traefik is up!
  • 2025-12-10 20:06: Couldn’t get Miniflux working. Will work on it tomorrow. Calling it a night.
  • 2025-12-10 20:30: Couldn’t quite leave it alone. I think I forgot to encrypt the cert with the new cluster’s sealed certificate. Will give that a try. If it works good. If not, we try again tomorrow! :)
  • 2025-12-10 21:00: It was the damned cert! Ok, now I know I was not going batty :)
  • Figure out http to https tomorrow. Haproxy was doing it here. Need to figure out how to do it with Traefix directly. Time for bed.

Feedback on this post?
Mail me at feedback at this domain.