In the DevOps world, monitoring is a very important part. Here by monitoring I mean monitoring of application server, database server, system resource usage etc.
Let’s discuss the below question.
What is Monitoring?
Generally IT Engineer check logs when there is some issue or something not working as expected. Below are some built-in tool or command which we use to check the logs
But we all know checking the logs using the above commands is a lengthy process and takes huge time. And sometimes it becomes so difficult to find the exact cause of an issue by reading the logs.
For example any business critical website which handles a lot of traffic, if anything happen to that site in the working hours or customer visiting hours, and you start looking at the log files, metrics to find the issue by using the above commands then probably you are not doing the right thing. You might be able to find the issue within the downtime, but there can be cases which are really difficult to find by reading the logs files, and doing this within the downtime may be a huge headache. That is why we need monitoring of the logs, metrics.
To solve the above issue we need some kind of tool which will help us in –
- Fetching the logs from different sources to a centralized place
- Correlating the logs and cross-reference them to give a holistic view
- Log parsing, searching, filtering, highlighting etc. facility
- Log retention as per our requirement
- It needs to help us to detect anomaly detection and give us the alert in the right time.
- Most importantly, we should be able to see some visualization using those logs, metrics data.
Together, all the above feature is called monitoring in DevOps, which is a very important part. We call the tool as ‘monitoring tool’ in general which will provide us all the above facility.
When we select any tool for this purpose, we need to think about the below things –
- The tool should have an intuitive user interface so that various team member can use the dashboards.
- It should be able to collect logs from various sources.
- It should be able to fetch metric sets.
- Furthermore, it should have some alerting facility etc.
In market there are many tools available, among them some are free, some are paid and some of them are SaaS (Software As A Service) offerings. So depending on your requirement and budget you can choose tools. It is always better to start with some free tools and do some POC and then take the decision. Doing some POC will surely help you to broaden your thinking.
Why do we need monitoring?
Generally, in the background of any business critical web application there are a lot of things running like
- Many servers (application server, database server etc.)
- Network Component (Routers, firewalls etc.)
- Lots of logs (syslogs, access logs, service logs, audit logs etc.)
- Metrics (performance metrics, transaction metrics, resource utilization metrics etc.)
If all the above looks good and the web application works fine, then nobody cares until something go wrong. That is why we need some sort of monitoring of the services, servers, database etc.
If you are interested in learning DevOps, please have a look at the below articles, which will help you greatly.
- Kubernetes Series: Part 1 – Introduction to Kubernetes | Background of Kubernetes
- Kubernetes Series: Part 2 – Components of Kubernetes cluster | Kubernetes cluster in detail
- Kubernetes Series: Part 3 – What is Minikube and How to create a Kubernetes cluster (on Linux) using Minikube?
- Introduction to Ansible | High Level Understanding of Ansible
- What is End-to-End Monitoring of any web application and Why do we need it?
- Network Connectivity Troubleshooting Guide for Beginners