My favorite deployment method for applications is Kubernetes. Kubernetes is a cluster orchestration tool, it is an open source project released in 2015 by google. Basically with Kubernetes you can deploy your workloads as containers and it will manage them across multiple "nodes" (computers that become agents for your cluster workloads). My first experience professionally with it was in 2017, at the time I didn't do much research about the tool but just needed to manage my docker containers. I must admit that the project failed because I didn't understand how to scale containers or properly allocate memory, so our team made the decision to migrate back to running our software on VMs and blade servers. Fast forward 4 years or so I finally decided to give Kubernetes another try with a web application, I was on a team of 3 developers and had been running containers in GCP app engine using CircleCI to build and deploy. I went all in with Kubernetes, we had 3 clusters actually, one that was managed by Jenkins for CI/CD which deploys workloads in a second for a staging environment and a third for the production environment. It worked very, very well, and is still going on 4 years of operation (at the time of this writing).
I never went back and building and maintaining Kubernetes clusters has actually become my profession in my current role as a DevOps engineer. I run them locally with minikube and I have one in DigitalOcean, where this website is running. One of the things I like about it is the flexibility as well as the way it makes my applications highly available without much legwork. Up until the time of this writing I haven't built a workflow to deploy this website (which is running in my cluster, I manually applied the deployment) to Kubernetes, so that's what I want to do today. This is a step-by-step pipeline while documenting it here for my reference as much as yours! Let's get to it!
Pre-requisites
There are a few things you will need to know and have to follow along with this article:
- Knowledge of Kubernetes
- Basic understanding of Github Actions
- A Kubernetes cluster (Create one in DigitalOcean, use my referral code, get $200 to follow along)
- A Github repository with an application the builds a docker container.
- Optionally, you might want a workflow that builds your docker image and pushes it to Github packages, as that is a dependency, you can see how to do that in my previous article. If you have your own way of doing that, great! Just make sure you use that docker image when deploying via the workflow below.
Preface
I will be using the github repository that hosts the code for this website, feel free to take a look. Note that the contents of the repository may change and I will be referencing the snippets separately so they don't change.
Step 1 - Create Your Deployment yaml File
Once you have a Github repository, you will need to create your deployment yaml file at the root of your repository, in my repo the name of this file is production-k8s.yaml. You can name this file whatever you want and I generally name it by cluster, you'll see that in my repo I have minikube-k8s.yaml and production-k8s.yaml (at least at the time of this writing), just make sure yours is at the root of the repository.
For this guide the contents don't really matter, it just has to be a valid Kubernetes config. I usually put everything necessary for the application to run it, just using general separation of concerns to decide.
You'll see in my deployment I didn't explicitly define the app name or the image name, they are variables instead called $APP_NAME and $IMAGE_NAME. These values will be replaced using envsubst in one of our following workflow steps. I am doing this because I have completely automated building the container and I also prefer to make a standard name for my app resources that are created so everything related can be easily recognized, you'll see how I define those in following steps.
👉 Using $APP_NAME and $IMAGE_NAME is optional feel free to change this, if you want to define these as constants. It's up to you.
Step 2 - Create a DigitalOcean Access Token
Before we can start deploying into the cluster we need to create an access token, also you must have a Kubernetes cluster available for your deployments, if you sign up to DigitalOcean with this link you get $200 free to follow along.
👉 How to Create Personal Access Tokens
The only caveat with the second one is maybe it's a good idea to give this token the least privilege it needs in order to work with the Kubernetes cluster. In my case I just used Custom Scopes and checked everything for Kubernetes. I also gave mine a memorable name for github actions. Make sure to save the token, in the next step we'll add it to your repository.
Step 3 - Access Your Cluster
You'll need to add two secrets to your repository. Adding a secret to your repository is very easy, I'll go over it quickly below but here's documentation on how to from Github.
For the first, go to your github repository, click the "Settings" tab. On the left you will see a "Secrets and variables" menu option then "Actions". In this case I just added a repository secret, so click the "New repository secret" button.
In the form that shows up for "New repository secret", name your DIGITALOCEAN_ACCESS_TOKEN and paste your token for the "Secret" input. Click "Add secret".
You'll also want to add the cluster name, which you made when you created the cluster. I named the secret PRODUCTION_CLUSTER and added my cluster name for the secret value.
Now your repository has the required access details for managing Kubernetes resources.
Step 4 - The Deployment Workflow
Now that you've got your Kubernetes deployment yaml and your cluster access sorted out we can build the deployment workflow in Github Actions. Prior to this you would also (optionally) need to figure out how to build the container. The deployment I'm gong to go over here assumes you have built the container in a separate build, specifically it is actually triggered by the one written about in my previous post.
To start you should create a file at the path ".github/workflows/deploy.yaml". Below is the finished file, but I will go through it section by section:
The first block declares the name of the workflow, "Deploy" in this case and a specific trigger. In my case, the trigger is when the workflow, "Build and Push Docker Image", from my "Docker Builds with Github Actions" article completes. What this means is that when a commit happens on the branch main, a docker image is built and pushed to Github Packages, once that is done the new image is used to update the deployment in Kubernetes. This could be anything for you, you can reference the Github documentation for how to set up your specific trigger if you want to do something different from what I'm doing here:
The second block below defines the jobs for the workflow, in this case we have only one named "deploy-to-k8s", and it sets the workflow to run on an ubuntu runtime. Then the the workflow checks out the repository code and sets up the access we created in the last step:
The third sets a few variables, both of these are optional as long as you hard code the attributes in the Kubernetes file:
The COMMIT_SHA variable is the tag to differentiate the image from others and the second, APP_NAME, is the name of the resources inside the Kubernetes cluster. These are replaced in the following and last section:
Above, in the fourth block of the workflow file, you see a few more interesting commands. We login to Github Packages, the workflow should have read access to its own Github Packages (where we deployed the image to in my previous post) without any set up using the GITHUB_TOKEN secret which is automatically provided. Then we actually do some work in the "Apply deployment config" step, here you can see the optional substitution I'm making for $IMAGE_NAME and $APP_NAME and piping into the deployment into our cluster (begins here with "kubectl apply -f -").
If you hardcoded everything in your version of "production-k8s.yaml", block 4 would look like this instead:
Finally, at the end in "Verify deployment" we wait for the deployment to finish before allowing the workflow to end. FYI, this is very helpful when you are working in a team that isn't as familiar with Kubernetes, if all you do is run the "Apply deployment config" then the your workflow ends "successfully" but the deployment is still rolling out pods. (😩 This is particularly slow when using EKS with Fargate in AWS, Fargate can take up to a minute to provision nodes.)
Conclusion
😮💨 That's all there is to it! You've taken a Kubernetes deployment configuration file, dropped it into your Github repository, and built a workflow with Github Actions that creates the resources in your DigitalOcean cluster. What you saw today was me in real time creating the deployment workflow for this website, what I mean is I wrote the article as I implemented it here 😜. Thanks for reading!