BUILDING CUSTOM DOCKER IMAGES - PART3
KUBERNETES FUNDAMENTALS
Introduction
Welcome to Part 3 of our comprehensive 5-part tutorial series. In this segment, we will focus on deploying the customized “remote-host” container that we created in Part 2. Our deployment target will be a Kubernetes cluster, introducing you to more advanced concepts in container orchestration.
Prerequisites
1. A functional Kubernetes cluster
2. WordPress and MySQL pods deployed on the cluster
- For guidance on deploying these, please refer to our previous tutorials on WordPress and Jenkins.
- While WordPress is used in this example, any deployment with a running MySQL pod will suffice
Note: The “remote-host” pod we configured in Part 2 includes the MySQL client, making it suitable for MySQL database backups. If you intend to back up other RDS systems, ensure you install the appropriate client in the “remote-host” container as outlined in Part 2.
Concepts
Introducing Kubernetes Namespaces
In this tutorial, we’ll explore the concept of Kubernetes Namespaces, a powerful feature for organizing and isolating cluster resources.
By default, Kubernetes creates the following namespaces:
1. default
2. kube-node-lease
3. kube-public
4. kube-system
When no namespace is specified, Kubernetes deploys pods to the “default” namespace. The other three namespaces are reserved for Kubernetes system pods and cluster maintenance.
Custom Namespaces
For better resource organization, we can create custom namespaces. In this tutorial, we’ll:
1. Create a new namespace called “remote-host”
2. Deploy our “remote-host” pod within this custom namespace
This approach demonstrates best practices in resource isolation and management within a Kubernetes environment.
Note: While we use “remote-host” as our namespace name, you can choose any appropriate name. Just remember to adjust your Kubernetes YAML files accordingly.
By the end of this part, you’ll understand how to:
- Create custom namespaces in Kubernetes
- Deploy pods to specific namespaces
- Organize and isolate resources effectively in a Kubernetes cluster
Let’s begin with creating our custom namespace and deploying our “remote-host” pod.
Part 3
Deploying the Customized “remote-host” Container
Step 1: Configuring and Deploying the Remote Host Pod
In this deployment, we’ll continue using AWS Elastic File System (EFS). We’ll mount one EFS storage on the master node to store the “remote-host” configuration files, and another EFS storage on the worker node for storing the “remote-host” data and backup files.
On the Master Node
1. Log in to the Master Node:
- Log in to the master node as the user “jenkins.”
2. Configure the “remote-host” YAML Files:
- Clone the “remote-host” YAML files from the GitHub repository.
- Ensure that the correct path is set for the
hostPath
in the “remote-host-pv.yaml” file.- The “remote-host” pod will run in its own namespace called “remote-host.” You can use the same pod to take backups of multiple databases running in different namespaces.
3. Static Cluster IP for “remote-host”:
- To ensure that the “remote-host” always runs on a static cluster IP, configure the service for “remote-host” as a ClusterIP.
- In the service YAML file (e.g., “remote-host-service.yaml”), hard-code the ClusterIP (e.g.,
10.97.129.185
). This IP should be within the range “10.244.0.0/16.”- Example of service YAML file
apiVersion: v1 kind: Service metadata: name: remote-host namespace: remote-host spec: type: ClusterIP ports: - protocol: TCP name: remote-host port: 22 targetPort: 22 selector: app: remote-host clusterIP: 10.97.129.185
Mounting EFS for Configuration Files
- If you want to use the same EFS mount point for all Kubernetes configuration files (including WordPress and Jenkins), skip the following step:
$ cd ~ $ sudo yum install -y git $ mkdir simple-remote-host
- If you prefer separate mount points for each configuration file, perform the following steps:
- Mount the EFS created for the config files to the directory you’ve created (e.g., “/home/jenkins/simple-remote-host”):
$ sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport fs-011a40c8f22e5b34d.efs.ap-south-1.amazonaws.com:/ /home/jenkins/simple-remote-host
- Verify that the mount is successful:
$ df -k
4. Download the “remote-host” YAML Files:
- Change to the “simple-remote-host” directory:
$ cd simple-remote-host
- Download the required “remote-host” YAML files from the GitHub repository:
$ git clone https://github.com/AvangelsTech/simple-wordpress-files.git .
Now let’s proceed to set up the Worker node for the “remote-host” data files.
Step 2: Setting Up “remote-host” Data Files on the Worker Node
1. Log in to the Worker Node:
- Log in to the worker node as the user “jenkins.”
2. Create the Data Files Directory:
- Navigate to the home directory:
$ cd ~
- Create a directory for your “remote-host” data files (you can choose any name you like):
$ mkdir remote-host-data-files
- Move into the newly created directory:
$ cd remote-host-data-files
- Create a temporary directory (we’ll use it later in the tutorial):
$ mkdir tmp
3. Mount EFS for Data Files:
- If you want to use the same EFS mount point for all Kubernetes data files (including WordPress and Jenkins), skip the following step.
- Otherwise, mount the EFS created for the WordPress data files to this directory. The mount command can be found in the AWS EFS console and has been explained in the associated article:
$ sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport fs-09b70357fe334b9f6.efs.ap-south-1.amazonaws.com:/ /home/jenkins/remote-host-data-files
- Verify that the mount is successful:
$ df -k
Congratulations! You’ve completed the directory setup on both the master node and the worker node.
Step 3: Creating the “remote-host” Deployment on the Master Node
1. Create the “remote-host” Namespace:
- Navigate to your home directory:
$ cd ~
- Create a YAML file for the “remote-host” namespace (e.g., “remote-host-namespace.yaml”):
apiVersion: v1 kind: Namespace metadata: name: remote-host
- Apply the namespace configuration:
$ kubectl create -f remote-host-namespace.yaml
2. Deploy the “remote-host” Pod:
- Deploy the “remote-host” using the following command:
$ kubectl create -f simple-remote-host/
- The “remote-host-deploy.yaml” file specifies pulling the “remote-host” container from the AWS ECR repository.
3. Verify Successful Deployment:
- Check if the “remote-host” is successfully deployed:
$ kubectl get all -n remote-host
- This command displays relevant resources within the “remote-host” namespace.
4. Test SSH Connection:
- Log into the “remote-host” pod from the Jenkins pod:
$ kubectl exec -ti <remote-host-pod-name> -n remote-host /bin/bash
Replace
<remote-host-pod-name>
with the actual name of the deployed pod.
- Once inside the pod, SSH to the “remote-host” using the cluster IP (since pods are running in different namespaces):
# ssh remote_user@<cluster-ip-remote-host> -p 22
Provide the
remote_user
password when prompted.
- If the connection is successful, you should be logged into the remote host.
Step 4: Taking a Manual Backup and Storing it in an S3 Bucket
Now that our setup is complete, let’s test if the remote host is functioning end-to-end. We’ll verify whether we can connect to the WordPress MySQL database and successfully take a backup of it to an AWS S3 bucket.
Follow these steps:
1. Create an S3 Bucket:
- Using the AWS console or AWS CLI, create an S3 bucket. Ensure that the bucket name is unique across AWS. For example:
mywp-backup-10apr23
.- If you need assistance, search online for instructions on creating an S3 bucket in AWS.
2. Log in to the Remote-Host Pod on the Master Node:
- Execute the following command to log in to the “remote-host” pod:
$ kubectl exec -ti remote-host -n remote-host /bin/bash
3. Take a Backup of the MySQL Database:
- Inside the “remote-host” pod, run the following command to back up the WordPress MySQL database:
# mysqldump -u root -h 10.96.0.101 -p wordpress > /tmp/wordpress_db.sql
- Replace
10.96.0.101
with the ClusterIP of the WordPress MySQL service.- The database name is “wordpress.”
- Provide the database password when prompted.
- This command creates a backup file in the
/tmp
directory within the “remote-host” pod.
4. Move the Backup to the S3 Bucket:
- We previously configured the “awscli” with the necessary inputs in Part 2 of this tutorial.
- Use the following command to copy the SQL file from the container to your S3 bucket:
# aws s3 cp /tmp/wordpress_db.sql s3://mywp-backup-10apr23/wordpress_db.sql
If everything is successful, you’ll find the SQL file in your S3 bucket.
Congratulations! We have successfully completed the manual integration of the “remote-host” pod with the WordPress MySQL database. This end-to-end system demonstrates effective communication between the “remote-host” pod and the WordPress MySQL database.
In our next tutorial, Part 4, we will take the automation a step further by integrating the “remote-host” pod with Jenkins to streamline the entire process of MySQL database backups.
Part 4