Minikube and OAuth2

Adding authentication to your local Minikube Kubernetes Cluster

Photo by kyle larivee on Unsplash

In a previous article, we looked at how to expose Minikube to the public Internet and generate a Let’s Encrypt certificate to support HTTPS. Here is the next step: with HTTPS in place, you can use OAuth2 to authenticate API access.

TL;DR

OAuth2 can be used to protect access to APIs, and there is plenty of material around to learn how OAuth2 works. This article shows how to implement OAuth2 with Kubernetes/Minikube by covering three topics:

• Configure an OAuth2 provider, in this case, we are using Github.
• Deploy the OAuth2-Proxy.
• Reconfigure an existing Kubernetes application running on Minikube to enable OAuth2.

The process is not complicated — but some research was needed to bring everything together for the Minikube environment, so this article may save you some of that time by showing a working example.

What you need to bring

• Everything from the previous article (Domain name, Linux PC, Minikube installation, Let’s Encrypt production certificate).
• A Github account to utilize Github as an OAuth2 provider.

How does this work?

There are a lot of good OAuth2 explanations around, so this section just focuses on what it means for your application without going into OAuth2 details.

If you have a working API served via HTTPS running on Kubernetes/Minikube, using the Nginx Ingress that comes with Minikube as an addon: there is no too much to do to implement OAuth2. You need to set up the OAuth2 provider, and you need to tell the application Ingress to check the authentication of an incoming API call with the OAuth2 provider before accessing the API.

This can be done by adding two lines of annotations to the Ingress that you have in place already. The annotations point to an OAuth2 Proxy that you will need to configure and deploy locally. The OAuth2 Proxy will know about the external OAuth2 provider that performs the actual authentication work and relieves you from dealing with user management and other complications.

The example discussed in this article uses very specific components to get things going. But there are choices: different Ingress Controllers, OAuth2 providers, or OAuth2 proxies can be used. But here we use specifically the Minikube Ingress addon, Github as OAuth2 provider, and OAuth2 Proxy. Let’s look at these three elements in the next sections.

There are many variations of how to authenticate, e.g. by email, groups, domains, etc. Here we do authentication of existing Github users, but you can expand from there.

Configure OAuth2 at Github

With a Github account comes the ability to create an OAuth2 application. Go to https://github.com/settings/developers, and click on OAuth App in the left-side menu, and then New OAuth App on the right, or click on the large green “Register a new application” button in the middle if this is your first registration:

After that, there is a screen where the basics need to be configured. It is straightforward. You will need to use your domain name as the Home Page, and the same name with added /oauth2/callback in the Authorization Callback field.

You can then see your generated client ID… you need this later, so make a copy of this code:

Then you need to generate a new client secret. There is a button for that — you can generate many and manage them individually. But more important right now: Use “Generate a new client secret” and really take note of it, because you can’t see this again. If you missed it, just generate a new client secret. The screen looks like this:

You will use the client ID and the client secret when configuring the OAuth2-Proxy. If you already cloned the Github repository for this article: a good place to store these values is in the config.sh file, more details about this are coming up.

Start Minikube and deploy your application

We need to make Minikube accessible to the public Internet and have HTTPS configured with a production certificate… you should check out the previous article about how to do that. You can use the deployment configuration or the running cluster from there as-is: you should be able to use a browser to access the hello and welcome services through your domain name with HTTPS.

With a running cluster in place, we go on to enable OAuth2 for the services. The YAML files in the earlier Let’s Encrypt article and this OAuth2 article are identical, except for the provisions needed for OAuth2. We will patch these changes into the live service.

Moving forward in the article we assume that Minikube is running with the app and that KUBECONFIG is set appropriately. You can also start from scratch using a supplied start.sh script, but you need a production certificate for your domain.

Deploy OAuth2-Proxy

The Github repository for this OAuth2 article includes a YAML file for the deployment of the OAuth2 Proxy. You should get this repository now, either through git clone or some other method.

git clone git@github.com:klaushofrichter/Minikube-and-OAuth2.git
cd Minikube-and-OAuth2

The YAML file oauth2-proxy.yaml.template for the OAuth2 Proxy needs to be customized before deployment by using envsubst which inserts configuration parameters. Here is a list of environment variables to configure, you need to change the values for your situation and copy these into the terminal:

export NAMESPACE="my-app"
export PUBLICURL="your.domain.com"
export OAUTH2_PROXY_CLIENT_ID="<value from github above>"
export OAUTH2_PROXY_CLIENT_SECRET="<value from github above>"
export OAUTH2_PROXY_COOKIE_SECRET="<value generated below>"

The NAMESPACE value should be the namespace of your application in this example, if you did not customize it differently, it would be my-app. PUBLICURL is the domain you are using — the production certificate you need to have is for that domain. The OAUTH2_PROXY_CLIENT_ID and OAUTH2_PROXY_CLIENT_SECRET need to be the same values you saw during the Github OAuth2 Application setup previously. Finally, the OAUTH_PROXY_COOKIE_SECRET should be generated by you… a recommended method is to use the value produced through this python call:

export OAUTH2_PROXY_COOKIE_SECRET="`python -c 'import os,base64; print(base64.urlsafe_b64encode(os.urandom(16)).decode())'`"
echo ${OAUTH2_PROXY_COOKIE_SECRET}

This produces an odd string of characters with a few = at the end. Use the full output for the cookie secret as shown above.

With the environment variables set, you can generate the YAML file and deploy it into the live cluster.

cat oauth2-proxy.yaml.template | envsubst > oauth2-proxy.yaml
kubectl apply -f oauth2-proxy.yaml

You can see the OAuth2 Proxy deployment, the associated service, and the other deployments and services in the namespace with this:

kubectl get all -n ${NAMESPACE}

The OAuth2 Proxy YAML template also includes an Ingress for the OAuth2-Proxy, which you can see with this command:

kubectl get ing -n ${NAMESPACE} 

You can see two ingresses: one called oauth2-ingress and another one called my-ingress. The second ingress belongs to the application — we are going to replace this one in the next section so that the OAuth2 process gets inserted into the API access. Both ingresses should show your domain name and ports 80 and 443.

(Re)configure the Kubernetes Application

We need to change just one thing in the existing application: add an annotation that instructs the Ingress to perform the authentication before accessing the target API. The place to do that is the application’s Ingress annotation section. Here are the top few lines from the file secure-ingress.yaml.template, with the additions in bold:

apiVersion: networking.k8s.io/v1 
kind: Ingress 
metadata:   
  name: my-ingress   
  namespace: ${NAMESPACE}   
  annotations:     
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    nginx.ingress.kubernetes.io/auth-url: \
      "https://$host/oauth2/auth"
    nginx.ingress.kubernetes.io/auth-signin: \
      "https://$host/oauth2/start?rd=$escaped_request_uri"

The YAML file goes on with many more lines, but the only change we need to do to the application ingress is the addition of the last two lines shown above in bold: These lines instruct the ingress to redirect requests to the OAuth2 Proxy.

There are two variables in the new annotation that should not be touched by the envsubst customization method. To keep these intact, we simply define them as the original value. See below:

export host="\$host"
export escaped_request_uri="\$escaped_request_uri"

With PUBLICURL and NAMESPACE defined earlier, we can now generate the YAML file for the updated ingress and apply it. We delete the existing Ingress my-ingress before applying the new one with the OAuth2 support.

cat secure-ingress.yaml.template | envsubst > secure-ingress.yaml
kubectl delete ing my-ingress -n ${NAMESPACE}
kubectl apply -f secure-ingress.yaml

You are all set. You can see the new annotations in the Ingress with this command:

kubectl describe ing my-ingress -n ${NAMESPACE}

Don’t worry about the error: endpoints “default-http-backend” not found message. That’s because we did not define a default HTTP backend.

You can now use your browser and visit your endpoint. If things went well, you should see a login screen from Github, where you can type in your credentials, and only then your actual API is accessed.

If you did a few experiments already, you may need to remove existing cookies that allow bypassing the authentication because you authenticated already, so you need to remove cookies related to your domain. Firefox offers a convenient button to do that when clicking the Lock icon to the left of the URL address bar once you are authenticated and on your site. Chrome shows a few more details at the same place, and there is a cookie button that lets you see and delete cookies related to the site. After the removal of the site cookies, you need to authenticate again.

Using a bash scripts to set it up

The instructions above rely on the setup of the earlier Medium article. However, the Github repository for this OAuth2 article comes with its own start.sh script, slightly simplified: for example, no cert-manager is installed as we already have a production certificate.

There is a config file config.sh to separate secrets from other files that are OK to be shared — you need to enter your secrets in config.sh before using start.sh. Note that the config.sh file is listed in .gitignore so that you don’t accidentally expose your secrets to others via git. It is also assumed that the certificate file follows the naming convention used in the often-cited earlier article. Certificate files following this naming convention are also in .gitignore. The naming convention is below, you need to replace the YOUR.DOMAIN.COM section with the value of ${PUBLICURL}:

prod-YOUR.DOMAIN.COM-cert.yaml

In any case, before running the script on your computer, you should examine it. It’s simple enough, so you can do your customizations and make sure that the script does not do harm to your data or environment by reading through it, before running it on your PC.

Regarding the cert-manager: it is perfectly reasonable to install it with this setup — if the certificate expires, it would update it via Let’s Encrypt. But note that this does not automatically create a backup. Without a backup, you may run into renewal trouble due to rate limitations for production certificates.

Where to go from here?

The example shown here is just a proof of concept, and further detailing is needed to reach a more useful place:

• Customizing the login experience, e.g. adding a logo to the login screen or use the device authentication flow.
• Processing the credential detail like the user name within the application.
• Handling of the OAuth2 process via curl, e.g. to integrate with CICD.

Thanks

Again, here is a big thanks to everyone who shared questions, ideas, findings, and methods with the community. This article is an attempt to contribute something back.