If the VMs status is “Unavailable”

The article contains the following terms:

Problem

A virtual machine (VM) has the Unavailable status. Typically, the issue exists on the cluster node where the problematic VM is located and is related to:

incorrect operation of the libvirtd service;
certificate problems.

Diagnosis

To confirm the cause of the problem, perform the following checks. The presence of at least one sign confirms the problem:

On the cluster node where the VM with the Unavailable status is located, check the status of the libvirtd service:
1. Connect to the cluster node via SSH. For more information about connecting via SSH, see Workstation setup.
2. Run the command:
```
systemctl status libvirtd
```
  Indicator: The service is stopped or the output contains errors related to socket files.
  Example output
```
● libvirtd.service - Virtualization daemon
 Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
 Active: failed (Result: exit-code) since Wed 2026-02-25 05:06:57 CET; 7h ago
 Docs: man:libvirtd(8)
```
On the platform server, check the vmwatch.log:
1. Connect to the server with the platform via SSH. For more information about connecting via SSH, see Workstation setup.
2. Run the command:
```
docker exec -it vm_back tail -f /var/log/vmwatch.log
```
  Indicator: libvirtd errors or entries about certificate problems are present.
  libvirtd connection error example
```
Connect to libvirt failed: virError(Code=38, Domain=7, Message='unable to connect to server')
Connection to libvirt not active, reconnect 
```
  Example certificate error
```
libvirt: XML-RPC error : authentication failed: Failed to verify peer's certificate
```

Solution

Incorrect operation of `libvirtd`

If diagnosis indicates problems with the libvirtd service, restart the service:

Connect to the cluster node via SSH. For more information about connecting via SSH, see Workstation setup.
Restart the libvirtd service:
```
systemctl restart libvirtd
```
Check the status:
```
systemctl status libvirtd
```

If restarting did not help, restart the socket files:

systemctl stop libvirtd-ro.socket
systemctl stop libvirtd-admin.socket
systemctl stop libvirtd.socket
systemctl stop libvirtd.service
systemctl start libvirtd-tls.socket
systemctl start libvirtd-ro.socket
systemctl start libvirtd-admin.socket
systemctl start libvirtd.socket

Make sure there are no certificate errors in the vmwatch.log:
1. Connect to the server with the platform via SSH. For more information about connecting via SSH, see Workstation setup.
2. Run the command:
```
docker exec -it vm_back tail -f /var/log/vmwatch.log
```
3. If errors persist, re-issue the certificate on the node following the instructions in the Certificate problems section.

Certificate problems

If the vmwatch.log contains an error related to certificates, re-issue the certificate on the cluster node:

Obtain the ID of the cluster node with the unavailable VM:
1. Go to the Nodes section → select the desired node → the id column.
2. Save the obtained value.
Re-issue the certificate using one of the following methods:
- via API request:
  1. Connect to the cluster node via SSH. For more information about connecting via SSH, see Workstation setup.
  2. Run the API request:
```
curl -H 'internal-auth: on' input:1500/vm/v3/node/<node_id>/cert -d '{}'
```
    Comment
    <node_id> — ID of the node with the unavailable VM. Obtained in step 1.
- via Swagger:
  1. In the right menu, click the Services icon → Swagger.
  2. In the Select a definition field, select vm.
  3. Go to the Node section.
  4. Select the following method:
```
POST /node/{node_id}/cert Updating libvirt certificates on a host
```
  5. Click Try it out.
  6. Enter the ID of the node with the unavailable VM. Obtained in step 1.
  7. Click Execute.

For more details, see the article Using Swagger.

Restarting the vmwatch service

If there are no problems with libvirtd and the certificate, but the VM status has not updated, restart the vmwatch service in the vm_back container:

Connect to the server with the platform via SSH. For more information about connecting via SSH, see Workstation setup.

Run the command:

docker exec -it vm_back supervisorctl restart vmwatch

Useful tips

Connected topics: