Docker logs are important tools for monitoring and managing the behavior of containerized applications. They provide a way to troubleshoot container runtime issues, as well as any problems that may arise. However, it’s important to note that there are two types of logs generated when using Docker: Docker logs and container logs.
Docker logs provide information about the Docker daemon and its behavior. Container logs provide information about the applications running inside the container. Both can be used to audit the activity of your containerized applications and the Docker environment. They can provide valuable information for understanding how an application interacts with its environment or dependencies. And they also help in tracking changes and ensuring that your systems are being used as intended.
If you’re not using Docker logs and container logs, you’re missing out on invaluable tools for understanding your containerized applications. By understanding what’s going on under the hood, you can debug issues and optimize your deployments. Read on to learn more about the differences between Docker logs and container logs and where they’re stored.
What Are Docker Logs?
Docker logs refer to logs generated by the Docker daemon and other Docker system components. These logs contain information about Docker daemon activities, such as starting and stopping containers, creating images, and connecting to networks.
It’s important to note that information in the Docker daemon logs is not typically relevant to application concerns unless the container itself is not starting.
Here are some reasons why Docker daemon logs are important:
Debugging
Docker logs can help you understand the behavior of the Docker daemon. This means they can help you identify issues that could be causing problems with starting or stopping containers.
Monitoring
Docker logs can be used to monitor the Docker daemon’s activity and identify potential problems or performance issues. By analyzing logs in real-time, you can identify trends or patterns that indicate a problem. Then you can take corrective action if necessary.
Compliance
Depending on your industry and regulatory requirements, you may need to keep logs for a certain period to be compliant. Docker logs can be stored and managed in a centralized log management system, such as SolarWinds® Loggly®, to make it easier to keep and access logs when needed.
Performance Optimization
Docker logs can provide insight into the performance and behavior of the Docker daemon. This can help you optimize performance and identify opportunities for improvement.
Security
Docker logs can be used to monitor security events and identify potential vulnerabilities or threats. By analyzing logs to identify patterns or indicators of compromise, you can take proactive steps to protect your systems and data.
What Are Docker Container Logs?
Unlike Docker daemon logs, Docker container logs are the logs generated by processes running inside a container. These logs can contain information about container runtime activity and any output or error messages generated by processes running in the container.
In the case of a Node.js server application, the container logs would contain any output or error messages generated by the Node.js server process and any other processes running inside the container.
Here’s an example of how Docker container logs can be used in the context of a Node.js server application:
- You create a Docker image and deploy it to a Docker host.
- You start a container from the image and run the Node.js server inside the container.
- The server starts processing requests and generating output and error messages. These messages are written to the container logs.
- You use the docker logs command to view the container logs. The logs contain any output or error messages produced by the Node.js server and any other processes running in the container.
- If you experience problems with the server, such as errors or performance issues, you can use the logs to troubleshoot the problem. For example, if the server crashes, the logs may contain information about what caused the crash.
- You can also use tools like Loggly to collect and analyze the container logs in a centralized location. This can be useful for monitoring and troubleshooting your Node.js server and the Docker environment.
Where Are Docker Logs Stored?
Docker logs each container’s standard output (and standard errors) and saves them as JSON files. Each line in JSON format has a timestamp and a specified source (stderr or stdout). Each log file contains information about a container.
Docker logs are stored as JSON files on the host machine’s file system in /var/lib/docker/containers by default. There’s a separate directory for each container in this directory. And the logs for this container are stored in a file named <container_id>-json.log in the container directory.
For example, on Linux, logs for a container with the ID some_container_123 would be stored in the following directory:
/var/lib/docker/containers/some_container_123/some_container_123-json.log
On Windows, the default location for Docker logs is in the following directory if Docker Desktop is running on Hyper-V:
%ProgramData%\Docker\containers\some_container_123\some_container_123-json.log
If you’re running Docker Desktop on WSL2, then the logs can be found in the following directory:
\\wsl$\docker-desktop-data\version-pack-data\community\docker\containers\some_container_123\some_container_123-json.log
On macOS, the default location for Docker logs is in the following directory:
/usr/local/var/lib/docker/containers/some_container_123/some_container_123-json.log
Although the above is accurate for the latest OS versions, it’s recommended to check the Docker documentation for the specific version of Windows or macOS you’re using for more detailed information on where the logs are stored. Additionally, you can use the following command to inspect a running container and retrieve the path to the log file for that container:
$ docker inspect -f ‘{{.LogPath}}’ some_container_123
Viewing Docker Logs
You can view both types of logs with the docker logs command. To view logs for a specific container, you can use the docker logs command and specify the container name or ID.
For example:
$ docker logs some_container_123
To view the logs for the Docker daemon and other Docker system components, you can use the docker logs command without specifying a container ID or name. This will print the logs from the Docker daemon and other system components to the console.
$ docker logs
Also, keep in mind that the logs only contain information written to the container’s standard output and standard error. If the application running inside the container is not configured to produce logs, the docker logs command will not show any.
It’s also worth noting that you can use tools and services to collect and analyze container logs, as well as Docker daemon logs, from multiple sources in one centralized location. This can be useful for troubleshooting and monitoring the behavior of your containerized applications and the Docker environment.
What About Managing and Maintaining Docker Logs?
Managing and maintaining Docker logs can be a tedious task, but it’s essential for ensuring the smooth operation of your containerized applications. One important practice is to regularly clean out old logs to prevent disk space from being consumed. This can be achieved by using the docker logs prune command or by setting up a log rotation schedule.
Another best practice is to use separate storage for Docker logs, rather than relying on the default storage location in the host machine’s file system. This can be achieved by using a centralized log management system like Loggly, which allows you to store and analyze logs from multiple sources in one central location.
Finally, it’s important to consider the possibility of customizing the log configurations to suit your needs. This can include setting log levels, specifying custom log locations, and configuring log rotation. By following these best practices, you can effectively manage and maintain your Docker logs and avoid the tedious task of managing logs. With a centralized log management system like Loggly, this task becomes much more efficient and manageable.
Why Loggly Over the Conventional Docker Logs Command?
There are several reasons why you might choose Loggly for your Docker logs rather than the traditional docker logs command:
Centralized Log Management
Loggly provides a central location to store and analyze your Docker logs, making monitoring and troubleshooting systems and applications easier. Loggly lets you view and search your Docker logs from one place instead of having to log in to them individually on each Docker host.
Real-Time Analytics
Loggly provides real-time analytics and visualization tools for analyzing Docker logs so you can quickly identify and fix problems as they arise. You can set up alerts to be notified when specific log events occur. You can also create custom dashboards and reports to understand the behavior of your Docker environment and containerized applications.
Scalability
Loggly can process large volumes of log data. It’s also designed to scale to the needs of your organization. This can be especially useful if you have a lot of Docker hosts or containers that generate logs.
Integration With Other Tools
Loggly integrates with a wide variety of tools and services, such as monitoring and alerting systems. This makes integrating Docker logs into your overall logging and monitoring strategy easy.
Summary
Docker logs and Loggly log management are valuable tools for understanding and managing the behavior of containerized applications. Docker logs provide a way to monitor and troubleshoot the runtime activity of containers, and Loggly is a cloud-based log management and analytics platform that helps organizations collect, analyze, and visualize log data from various sources.
Docker logs are stored in the host machine’s file system by default, but the location can be customized using the –log-driver and –log-opt options when starting a container. Loggly can be used to collect and store Docker logs in a centralized location, providing real-time analytics and visualization tools for gaining insights into the behavior of your Docker environment and containerized applications. Overall, Docker logs and Loggly are essential for debugging, monitoring, compliance, performance tuning, security, and auditing in the context of containerized applications.
Check out Loggly today!