Deploying Joomla, MySQL, Prometheus & Grafana on Amazon EKS

This article is based on the container service for Kubernetes provided by AWS. The clients of AWS are provided with a service named as EKS(Elastic Kubernetes Services) which help the users to create a kubernetes cluster in just one go. This is a task given by Mr. Vimal Daga Sir under LinuxWorld India AWS EKS Specialized Training. In this article you will come to know How to deploy Joomla, MySQL, Prometheus and Grafana on AWS EKS. I will discussion each and every thing in details. So, let’s go…..

What is AWS?

AWS stands for Amazon Web Services. It is a public cloud. It is the world’s most comprehensive and broadly adopted cloud platform, offering over 175 fully featured services from data centers globally. It is a subsidiary of Amazon that provides on-demand cloud computing platforms and APIs to individuals, companies, and governments, on a metered pay-as-you-go basis. There are 24 globally launched Regions with 76 Availability Zones and 216 Points of Presence (205 Edge Locations and 11 Regional Edge Caches).

What is Kubernetes(K8s)?

Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available.

Containers are a good way to bundle and run your applications. In a production environment, you need to manage the containers that run the applications and ensure that there is no downtime. For example, if a container goes down, another container needs to start. Wouldn’t it be easier if this behavior was handled by a system?

That’s how Kubernetes comes to the rescue! Kubernetes provides you with a framework to run distributed systems resiliently. It takes care of scaling and failover for your application, provides deployment patterns, and more. For example, Kubernetes can easily manage a canary deployment for your system.

Kubernetes provides you with:

• Service discovery and load balancing
  Kubernetes can expose a container using the DNS name or using their own IP address. If traffic to a container is high, Kubernetes is able to load balance and distribute the network traffic so that the deployment is stable.
• Storage orchestration
  Kubernetes allows you to automatically mount a storage system of your choice, such as local storages, public cloud providers, and more.
• Automated rollouts and rollbacks
  You can describe the desired state for your deployed containers using Kubernetes, and it can change the actual state to the desired state at a controlled rate. For example, you can automate Kubernetes to create new containers for your deployment, remove existing containers and adopt all their resources to the new container.
• Automatic bin packing
  You provide Kubernetes with a cluster of nodes that it can use to run containerized tasks. You tell Kubernetes how much CPU and memory (RAM) each container needs. Kubernetes can fit containers onto your nodes to make the best use of your resources.
• Self-healing
  Kubernetes restarts containers that fail, replaces containers, kills containers that don’t respond to your user-defined health check, and doesn’t advertise them to clients until they are ready to serve.
• Secret and configuration management
  Kubernetes lets you store and manage sensitive information, such as passwords, OAuth tokens, and SSH keys. You can deploy and update secrets and application configuration without rebuilding your container images, and without exposing secrets in your stack configuration.

What is Amazon EKS?

Amazon Elastic Kubernetes Service (Amazon EKS) is a managed service that makes it easy for you to run Kubernetes on AWS without needing to stand up or maintain your own Kubernetes control plane. Kubernetes is an open-source system for automating the deployment, scaling, and management of containerized applications.

Amazon EKS runs Kubernetes control plane instances across multiple Availability Zones to ensure high availability. Amazon EKS automatically detects and replaces unhealthy control plane instances, and it provides automated version upgrades and patching for them.

Amazon EKS is also integrated with many AWS services to provide scalability and security for your applications, including the following:

• Amazon ECR for container images
• Elastic Load Balancing for load distribution
• IAM for authentication
• Amazon VPC for isolation

Amazon EKS runs up-to-date versions of the open-source Kubernetes software, so you can use all the existing plugins and tooling from the Kubernetes community. Applications running on Amazon EKS are fully compatible with applications running on any standard Kubernetes environment, whether running in on-premises data centers or public clouds. This means that you can easily migrate any standard Kubernetes application to Amazon EKS without any code modification required.

Amazon EKS control plane architecture

Amazon EKS runs a single tenant Kubernetes control plane for each cluster, and control plane infrastructure is not shared across clusters or AWS accounts. This control plane consists of at least two API server nodes and three etcd nodes that run across three Availability Zones within a Region. Amazon EKS automatically detects and replaces unhealthy control plane instances, restarting them across the Availability Zones within the Region as needed. Amazon EKS leverages the architecture of AWS Regions in order to maintain high availability. Because of this, Amazon EKS is able to offer an SLA for API server endpoint availability.

Amazon EKS uses Amazon VPC network policies to restrict traffic between control plane components to within a single cluster. Control plane components for a cluster cannot view or receive communication from other clusters or other AWS accounts, except as authorized with Kubernetes RBAC policies.

This secure and highly-available configuration makes Amazon EKS reliable and recommended for production workloads.

How does Amazon EKS work?

Getting started with Amazon EKS is easy:

1. First, create an Amazon EKS cluster in the AWS Management Console or with the AWS CLI or one of the AWS SDKs.
2. Then, launch worker nodes that register with the Amazon EKS cluster. We provide you with an AWS CloudFormation template that automatically configures your nodes.
3. When your cluster is ready, you can configure your favorite Kubernetes tools (such as kubectl) to communicate with your cluster.
4. Deploy and manage applications on your Amazon EKS cluster the same way that you would with any other Kubernetes environment.

What is Joomla ???

Joomla is an open-source content management system used for creating Web content. It is written in PHP and makes use of a MySQL database for storing data and uses object-oriented programming techniques. We are going to use pre created Docker Images of Joomla and MySQL.

What is Prometheus ???

Prometheus is a free software application used for event monitoring and alerting. It records real-time metrics in a time series database built using a HTTP pull model, with flexible queries and real-time alerting. We are going to use it to monitor our Multi Node Cluster.

What is Grafana ???

Grafana is open source visualization and analytics software. It allows you to query, visualize, alert on, and explore your metrics no matter where they are stored. In plain English, it provides you with tools to turn your time-series database (TSDB) data into beautiful graphs and visualizations.

Pre-requisites for this Practical

1. First, we need to have an AWS account and then we have to create an IAM user with power user privileges in our AWS root account.
2. Download and install AWS CLI from https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html
3. Download and install the eksctl from https://eksctl.io/
4. Download Helm and Tiller from https://helm.sh/
5. Make the AWS CLI, eksctl, helm and tiller executable as an environment variable in the Operating System(Windows, macOS, or Linux).
6. Make a profile of AWS IAM user in O.S. using AWS CLI.
7. Download an editor like Visual Studio Code.

Let’s Get Started with our Task

In this practical we will perform the following tasks:

1. We are going to create a Kubernetes Cluster in AWS EKS
2. We will configure Kubectl in our system in order to communicate with our cluster.
3. Then we are going to deploy Joomla web-app with MySQL database in cluster with Application Load balance(ALB) using Kustomization YAML file.
4. We will setup Joomla with MySQL Database.
5. We will setup helm and tiller
6. Then we will deploy Prometheus and Grafana in the Cluster
7. Then we will connect the Prometheus(TSDB) with Grafana and export dashboard from Grafana Labs.

Step — 1: Configure AWS CLI

AWS CLI in Powershell 7(Win10)

Step — 2: Write a YAML file to create EKS cluster

To create cluster using eksctl we need to create one YAML file. As you can see below we have created a file named “ddcluster.yaml”. In this YAML file we have passed on some information on the basis of which AWS EKS will create the K8s Cluster in the cloud.

In this cluster config file, we are creating three nodeGroups’(ng1, ng2 and ng-mixed). Every nodeGroup are having different desired capacity of nodes or EC2 instances, but ng-mixed is on-demand nodeGroup which will horizontally scale if the load increases or client/User needs it.

Step — 3: Create the K8s Cluster in EKS using eksctl command

In PowerShell, just run the following command

eksctl create cluster -f ddcluster.yaml

dd-eks-cluster is created

Next if I go to AWS and go to EKS, we can see that the Cluster has been created.

If I go to Cloud Formation in AWS, we can see that cloud formation created the cluster, Node Groups, VPC and everything that is needed.

Also if I go to EC2 Dashboard, we can see total 5 Instances have been deployed successfully. So, finally our cluster is ready to use.

Step — 4: Configure the kubectl Command

We are going to run the below mentioned CMD which will setup the kubeconfig file in ~/.kube directory.

aws eks update-kubeconfig --name dd-eks

🎉Congrats we are done with whole setup of Kubernetes.🎉

⚡ Now we can easily deploy our applications.

Step — 5: Deploy Joomla and MySQL Database in the K8s cluster

We will create and execute 3 YAML file to deploy Joomla and MySQL sequentially.

1. MySQL Deployment File

Using this above mentioned file we going to create MySQL pod which will work as database for Joomla. In this file, we have created an Cluster IP service for the database as we don’t want the outside world to access our database. Then we have created one PVC, attached to “/var/lib/mysql”, which will make my MySQL data permanent. Means due to any reason the pod gets terminated we won’t loose any information. We have also created one deployment file to deploy MySQL pod. Notice one thing that the password of MySQL is coming from another file. Usually we create one secret file and inside that we securely save our passwords and give the location of that file. Now our deployment automatically retrieve that password from that file.

2. Joomla Deployment File

Using this above mentioned file we going to create Joomla pod. In this file, we have created an Load Balancer service for the Web-App as we want the outside world to access our site. Then we have created one PVC, attached to “/var/www/html”, which will make my Joomla data permanent. Means due to any reason the pod gets terminated we won’t loose any information. We have also created one deployment file to deploy Joomla pod.

3. Kustomization File

This file will automatically find out other files and will run them. You can think it as a small automation. Rather than we manually run each file we simply mention the files names in it. Remember one thing the name of this file is fixed.

Step — 6: Execute the files using kubectl

At first it’s always good practice to create the namespace for each purpose. This makes things organized. We have to create one namespace called “joomla” first.

kubectl create namespace joomla

Then change the default namespace to “joomla”

kubectl config set-context --current --namespace=joomla

After that run the Kustomization file which execute all the files sequentially.

kubectl create -k .

After running the Kustomization file, run

kubectl get all

🎉So, finally our Joomla Web-App and MySQL Database pod have been deployed properly.🎉

Step — 7: Setup Joomla and connect to MySQL

In the first page just fill up the basic information like site name, email, username and password.

In next page we have to provide the database details. In Host name type “mysql” because this is the same name of our MySQL deployment. Next in username type “root”. You can also create one user in Database but for that you have to write some more extra lines on MySQL deployment file. In password give that password that you stored in that Secret file. In Database name give “Joomla” and it will create one Database called “Joomla”.

Then we have to run the installer and within few seconds our Web App will be ready to use. Next you will get one login screen, simply login and you will see this kind of page given below.

Then we will create a new article.🌟

Step — 8: Setup Helm and Tiller

We have to setup Helm first. In order to setup Helm we only have to run 3 command line instructions in Windows PowerShell.

helm init
helm repo add stable https://kubernetes-charts.storage.googleapis.com/
helm repo update

After the Helm setup, now we will setup Tiller. Here we going to run few command line instructions. This will create one service account for tiller so that tiller can deploy our required resources like pods, PVC etc.

kubectl -n kube-system create serviceaccount tiller
kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
helm init --service-account tiller
helm init --service-account tiller --upgrade

Step — 9: Setup Prometheus and Grafana

Setup Prometheus

Now we can easily deploy Prometheus using helm. So, here we are going to run 4 command line instructions to deploy Prometheus. First one is to create one namespace for Prometheus and second one to deploy it using the official Prometheus official helm chart, the next one to get the IP-address of the Prometheus server so that we can integrate with Grafana and the last one is to port-forward Prometheus Server to access the Prometheus Dashboard.

kubectl create namespace prometheus
helm install stable/prometheus --namespace prometheus --set alertmanager.persistentVolume.storageClass="gp2" --set server.persistentVolume.storageClass="gp2"
kubectl get svc -n prometheus
kubectl -n prometheus port-forward svc/<provided_name>-prometheus-server 8888:80

🌟So, our Prometheus Server is up and running in EKS cluster.🌟

Setup Grafana

Just like Prometheus we will run 3 command-line instructions to setup Grafana.

kubectl create namespace grafana
helm install stable/grafana --namespace grafana --set persistence.storageClassName="gp2" --set adminPassword=grafana --set service.type=LoadBalancer
kubectl get svc -n grafana

Login to Grafana -> “admin:grafana” and we will find a dashboard that looks like this 👇

Step — 10: Connect Prometheus to Grafana

We have to add the data source in Grafana as Prometheus by providing the endpoint of Prometheus which will be https://<private_ip_of_node>:9100. Then we have to save the changes.

After that we create our own dashboard accordingly or we can export from the Grafana Labs.

To export from Grafana Labs, first we have to go to https://grafana.com/grafana/dashboards and find a suitable dashboard which can work pretty well with our K8s Cluster Setup. We have used Cluster Monitoring for Kubernetes by Pivotal Observability (id-10000).

Click on the plus icon (+) on left side and click on Import. Next give the value “10000”. Actually this is pre created Dashboard for Kubernetes Multi Node Cluster. Next select the Prometheus and next click on Import. Then you will be able to see complete Monitoring of Kubernetes all the Cluster and everything.

So, we have successfully completed task.

Now I will show you how to delete the K8s Cluster easily

To delete the K8s Cluster we have to run a simple command line instruction that is to delete the cluster using the same cluster YAML file.

eksctl delete cluster -f ddcluster.yaml

Alright, thanks everyone for reading this article. Thank you so much Mr. Vimal Daga Sir for motivating and teaching us the latest technology to make India Future Ready. I really enjoyed while doing this task.

Happy Coding Everyone!! Peace!!

GitHub Repository

DipadityaDas/AWSEKSTask

Connect with me on linkedin

Dipaditya Das - University of Engineering &amp; Management (UEM) - Howrah, West Bengal, India |…