Time synchronization

The following terms are used in the article:

You can configure automatic synchronization of the system time on the platform server and cluster nodes with an accurate time server (NTP server). Synchronizing the time on the servers:

simplifies work with logs. For example, when analyzing incidents;
avoids problems with the libvirt service. If the time difference between the platform server and the cluster node is more than 15 minutes, the service cannot check the validity of the libvirt certificate and terminates with an error.

The following time synchronization modes are available:

VMmanager platform server — time on cluster nodes is synchronized with the platform server. Time synchronization on the platform server is not performed automatically. You can configure it manually using chrony or ntp software;
standard time servers for the OS used — time on each server is synchronized by local settings of chronyd or ntpd service. This mode is used by default;
other servers — time of each server is synchronized with NTP servers specified by the administrator.

Work logic

Time synchronization service is started when installing or upgrading the platform to version 2025.09.1 and higher. Service disabling is not supported in the platform. For correct operation of the service, chrony software must be installed on the platform server and cluster nodes beforehand.

Before adding a new node to the cluster, the platform checks NTP operation on it. If the check fails, the node is not added to the cluster. Verification steps:

The chrony software is installed on the server. If chrony could not be installed, then ntp software is installed. If ntp could not be installed, the check fails.
Server time is synchronized in the mode selected in the platform settings.
If synchronization is not performed and the time difference between the platform server and the node is more than 15 minutes, the test terminates with an error.

Time synchronization setting

Setting the synchronization mode

To set the time synchronization mode:

In the right menu, click the icon → Global settings section.
In the Time section, click Edit.
Select the servers with which the time will be synchronized:
- VMmanager platform server;
- Standard time servers for the OS used;
- Other servers → specify the IP addresses or domain names of the servers.
Click Save button.

After the settings are applied, time synchronization will start on the cluster nodes. Synchronization may take about one hour.

If some nodes were disconnected from the cluster at the time the settings were applied (for example, they were in maintenance mode), the synchronization settings will not be applied automatically after those nodes are connected. To apply the settings:

In the right menu, click the icon → Global settings section.
In the Time section, on the Time is not synchronized on X nodes banner, click Apply settings.

Notification settings

To receive notifications about synchronization problems, create a notification with the parameters Time is not synchronized on the node and Errors in node tasks. For more information about creating notifications, see theNotifications article.

Synchronization fine-tuning

You can change the following synchronization settings:

check interval. The default value is every 60 minutes;
allowable deviation of the server time from the accurate time source. The default value is 5 seconds.

These parameters are set via environment variables in the nodewarden container. To change their values:

Connect to the server with the platform via SSH. For more information about connecting via SSH, see Workstation setup.
Create the patch file /opt/ispsystem/vm/patch_time_sync.yaml with the following contents:
```
version: "3.5"
services:
  nodewarden:
    environment:
      TIME_SYNC_INTERVAL: <T_sec>
      TIME_SYNC_MAX_OFFSET: <T_ms>
```
Comments:
- <T_sec> — check interval in seconds;
- <T_ms> — allowable deviation in milliseconds.

Apply the patch:

sudo vm add-patch -p=time_sync -f /opt/ispsystem/vm/patch_time_sync.yaml

Verify that the changes have been applied:
```
sudo docker exec nodewarden env | grep TIME_SYNC
```
If the patch is applied successfully, the output will contain the new values of the variables:
Example output
```
TIME_SYNC_INTERVAL: 30
TIME_SYNC_MAX_OFFSET: 1000
```

Diagnostics

To diagnose problems related to time synchronization, examine the log files:

/opt/ispsystem/vm/install.log on the platform server — time synchronization on the platform server;
/var/log/vm_reader.log* and /var/log/vm_writer.log* in the vm_box container — time synchronization errors;
/var/log/node_alert.log in the vm_box container — notification logs;
/var/log/nodewarden.log in the nodewarden container — monitoring logs.

Useful tips