Platform log files

DCImanager 6 saves information about the operation of services in log files. The data from log files can be used to diagnose the platform operation. Log files are stored in the /var/log directories of docker containers.

Log files list

Main log files on platform server

Container	File path	Contents
dci_auth_back_1	/var/log/licupdate.log	logs of the license validation
dci_back	/var/log/dci_1_writer.log /var/log/dci_1_reader.log	logs of the main DCImanager 6 service
	/var/log/dci_location_check_update.log	logs of the update service for containers at the location. The service starts when the container is restarted and updates the containers at the location, if necessary
	/var/log/dci_locationctl.log /var/log/dci_ansiblectl_location_<location id>.log /var/log/ansible.log	logs of creating and configuring a location
	/var/log/dci_operationctl.log	log of operations performed at the location
dci_consumer_1	/var/log/worker_daemon.log	logs of periodic equipment polls and obtaining statistical data
dci_dns_proxy_1	/var/log/dns_proxy_integration.log /var/log/dns_proxy_service_1_reader.log /var/log/dns_proxy_service_1_writer.log	logs of the modules: Integration with PowerDNS Integration with DNSmanager 6
dci_ipmgr_1	/var/log/ip_1_reader.log /var/log/ip_1_writer.log	logs of address space management service
dci_ipmi_proxy_v2_client_1	/var/log/ipmi_proxy_goproxy.log	goproxy service logs. The service transmits data to the VNC client and proxies Websocket requests to the VNC server
dci_ipmi_proxy_v2_client_1	/var/log/ipmi_proxy_update_checker.log	logs of the update service for BMC proxy module at the location. The service starts each time the container is restarted and updates the module if necessary
vm_ldap_1	/var/log/isp_ldap.log	logs of synchronization with the LDAP directory
migrator	/var/log/migrate.log /var/log/bill_migrate.log	migration of objects from DCImanager 5

The platform installation log is saved in the file /opt/ispsystem/dci/install.log.

The log files of the diagnostics operation are listed in the Diagnostics check article.

The authorization service logs are written to the standard output of the dci_auth_back4_1 container.

Main log files on location server

Container	File path	Contents
eservice_handler	/var/log/dci_switch_<id>_handler.log	logs of switch handlers
	/var/log/dci_pdu_<id>_handler.log	logs of PDU handlers
	/var/log/dci_ipmi_<id>_handler.log	logs of BMC handlers
	/var/log/dci_intel_amt_handler.log	logs of Intel AMT handlers

Comments

Working with logs

Copy log file from a container

To copy a log file from the container to the current directory, run the command:

docker cp <container_name>:<path_to_log> ./

Comments to the command

Example command

docker cp dci_auth_back_1:/var/log/licupdate.log ./

View log files in a container

Enter the required container:
```
docker exec -it <container_name> sh
```
Comments to the command
<container_name> — container name
Example command
```
docker exec -it dci_back sh
```
View the log file using standard Linux utilities. For example, cat, tail, less, grep, etc.
Example command: Find out when the switch was polled
```
grep 'Switch status start' /var/log/dci_switch_1_handler.log
```
Exit the container:
```
exit
```

Viewing log files via stdout

Separate log files are not created for some services. Logs of these services are written to stdout. To view such logs, run the command:

docker logs <container_name> --tail <lines> --since <period>

Comments to the command

Example command: Display the last 100 log lines of the authorization service created in the last 60 minutes

docker logs dci_consul_1 --tail 100 --since 60m

Collecting platform logs

To collect all the log files of the platform in one directory:

Connect to the server with the platform via SSH.

Create the bash script getlog.sh with the following content:

#!/bin/bash

BEGIN_DATE=$(date +'%Y-%m-%d %H:%M:%S' --date '-1 day')
SINCE=${1:-${BEGIN_DATE}}

/bin/echo "Remove old files"
/bin/rm -rf /home/logs && /bin/mkdir -p /home/logs/container && /bin/mkdir -p /home/logs/service && /bin/cd /home/logs

/bin/echo "---===Step 1/3===---"
DOCKER_CONTAINER_NAMES=`journalctl --field CONTAINER_NAME`
CONTAINERS=($DOCKER_CONTAINER_NAMES)

/bin/echo "Get logs of the containers"
for container in ${CONTAINERS[@]}
do
    /bin/echo "Get log of the container: ${container}"
    /bin/journalctl --no-pager CONTAINER_NAME=${container} --since "${SINCE}" > /home/logs/container/${container}.log
done

/bin/echo "---===Step 2/3===---"
DOCKER_SERVICES_NAMES=`journalctl --field SYSLOG_IDENTIFIER`
SERVICES=($DOCKER_SERVICES_NAMES)

/bin/echo "Get logs of the services"
for service in ${SERVICES[@]}
do
    /bin/echo "Get log of the service: ${service}"
    /bin/journalctl --no-pager SYSLOG_IDENTIFIER=${service} --since "${SINCE}" > /home/logs/service/${service}.log
done

/bin/echo "---===Step 3/3===---"
ARCHIVE_NAME="logs_$(date +'%Y_%m_%d_%H_%M_%S').tar.gz"
tar -cvzf /home/$ARCHIVE_NAME -C /home/logs .

/bin/echo "Done"

Run the script using one of the commands:
Before collecting logs, the script deletes all files from the /home/logs/ directory.
Collect logs for the last day
```
bash getlog.sh
```
Collect logs since a certain date
```
bash getlog.sh "YYYY-MM-DD"
```
Collect logs since a certain date and time
```
bash getlog.sh "YYYY-MM-DD HH:MM:SS"
```
Comments to the commands

YYYYY-MM-DD — date from which logs should be collected
HH:MM:SS — time from which logs should be collected

The script:
1. Saves container logs in the /home/logs/container/ directory.
2. Saves service logs in the /home/logs/service/ directory.
3. Creates an archive logs_<current_date>_.tar.gz in the /home/ directory.

Logging in journald

Logging settings

The platform uses the journald system service to collect logs. journald collects logs for all containers except dci_consul_1. You can change the journald logging level; see the related page.

The journald service on the platform server and the location server is configured automatically. The service configuration is saved in the /usr/lib/systemd/journald.conf.d/dci.conf file. If the service has already been configured on the server, the configuration set by the platform will have a higher priority.

Configuring the journald service may fail if the syslog and rsyslog service settings on the server have been changed.

Configuration parameters set by the platform:

Storage=persistent — store files on disk;
Compress=yes — compress data in logs;
SystemMaxFileSize=500M — the maximum size of the log file is 500 Mb. When this value is reached, logs are rotated;
SystemMaxUse=20G — the maximum disk size for storing logs is 20 GB. When this value is reached, older files will be deleted.

Operations with logs

journald saves logs in the /var/log/journal/ directory. You can view the logs using the journalctl utility. Examples of how to use the utility:

Get id of containers for which logging is performed

journalctl --field CONTAINER_NAME

View container logs

journalctl -f CONTAINER_NAME=<container_id>

Get id of services for which logging is performed

journalctl --field SYSLOG_IDENTIFIER

View service logs

journalctl -f SYSLOG_IDENTIFIER=<service_id>

Get logs of the authorization service from the certain date

journalctl --no-pager CONTAINER_NAME=dci_auth_back4_1 --since "YYYY-MM-DD"

Comments to the command

Useful tips