My Homelab Automation Workflow: Setting Up Semaphore, GitHub Actions, and Jenkins

In my last post, I discussed the deployment phases and focused on deploying monitoring and logging services. However, after a bit of reflection, my priorities shifted, and I decided to focus on automation instead! As I worked through the services, it quickly became clear that establishing the foundation for managing future deployments should take precedence.

I decided to use Semaphore for Ansible and GitHub Actions. I plan to use Ansible for server configuration across environments and GitHub Actions to deploy the latest Docker Compose stacks. I also think Docker Swarm could be a viable option in the future. At some point, I may shift my focus to Swarm or perhaps explore other tools like HashiCorp Nomad.

Why these tools, though? There are certainly alternatives. Semaphore is just one of many popular Ansible derivatives, but it fit my needs perfectly for a homelab—easy to use, minimal fuss, and simple deployment. Plus, its integration with GitHub was a feature I really appreciated. GitHub Actions is gaining popularity, and since I’d previously had success using it to deploy my old Kubernetes services, I thought it would be a great fit for Docker Compose deployments. Unfortunately, there was an incident with GitHub Actions this weekend that prevented me from deploying much—more on that in a future post.

I also set up a new Jenkins controller and Jenkins agents. While I don’t have a clear plan for this yet, Jenkins is a technology I’m familiar with and want to continue using. Perhaps it’ll serve as a CLI runner for services or jobs. For now, it's ready to handle whatever I throw at it.

I ended up hosting Semaphore in a Docker Compose container with a Postgres backend that I set up on a VM in Proxmox. The process was very user-friendly—creating variable groups, repositories, and inventory files was quick and easy. I also chose to self-host the GitHub Actions runners on Proxmox VMs, as the workload for each runner could eventually overwhelm a Raspberry Pi. The GitHub documentation was thorough and easy to follow, but I did encounter a few challenges setting up the correct SSH keys (more on that later). Once I got past that hurdle, things went smoothly—each new commit to the dev runner automatically triggered a deployment. This is one of the most powerful features for me—no extra work is needed when iterating on deployments, and I can see the outcomes in near real-time.

Setting up Jenkins (controller and agents) was also relatively painless, thanks to Semaphore. I was able to set up the controller, agent, and container agent with the automation tools I prioritized. What a relief!

As with any project, there were a few lessons learned. For me, the biggest challenge was setting up the right SSH keys for GitHub Actions. It wasn’t immediately clear to me that GitHub Actions required the keys to be placed in ~/.ssh/deploy_key, in addition to using ssh-keyscan and adding that output to ~/.ssh/known_hosts. I spent more time than I’d like on this, but the effort paid off. Another lesson learned was while setting up agents for Jenkins, I realized that the Jenkins user I created needed to be included in the adm and systemd-journal groups. Now, I can use a template file in Ansible to configure a service file with environment-specific variables, such as {{ jenkins_url }}, {{ jenkins_agent_secret }}, and {{ jenkins_agent_name }}.

Setting up Semaphore, GitHub Actions, and Jenkins has laid a solid foundation for automating my homelab deployments. Despite some challenges with SSH keys and Jenkins agents, the process has been rewarding and will streamline future deployments. With automation now in place, I’m looking forward to exploring tools like Docker Swarm or HashiCorp Nomad and continuing to deploy the remaining services while working on new ones.