Incorrect VM status after migration

Problem

A virtual machine (VM) has been successfully migrated to the target cluster node, but the platform still shows it on the source node.

The following symptoms are observed:

one of the following VM states:
- the VM shows as Active, but migration tasks show as Failed, are stuck in a pending state, or are missing entirely;
- the VM is stuck in the Migrating state, while migration tasks are completed and show as Successfuly;
- the VM shows as Stopped, while the migration task shows as Running.
running virsh list --all|grep <VM_name> on the source node does not show the target VM;
running virsh list --all|grep <VM_name> on the target node shows the VM in the running state.

Cause

The platform did not receive confirmation of successful migration completion due to a network failure or an error while processing the request. As a result, the actual VM state diverges from the state in the platform DB.

Diagnostics

To confirm the cause of the problem:

In the web interface, open the card of the problematic VM and record the VM name and the cluster node.
Connect to the server with the platform via SSH. For more information about connecting via SSH, see Workstation setup.

Determine the actual VM location:

docker exec vm_back vssh -a 'virsh list --all|grep <name_VM>' 2>/dev/null

Example output

[root@VMmanager10 ~]# docker exec vm_back vssh -a 'virsh list --all|grep blue-rubellite' 2>/dev/null

Node 8:
 8    191_blue-rubellite   running

Explanation:

Node 8 — cluster node ID;
191_blue-rubellite — internal VM name.

If the VM is running on a cluster node different from the one specified in the web interface, the problem is confirmed.

Solution

Warning

Before following the instructions, create a backup of the platform. For more information, see the article Creating backups manually.

To solve the problem:

Determine the request_id of the migration task:
1. In the right menu, click the icon Tasks list .
2. Select the migration task for the destination VM.
3. Save the value of the request_id field.
Connect to the platform server via SSH. For more information about connecting via SSH, see the article Workstation setup.

Install the jq:

Astra Linux, Ubuntu

apt update && apt install jq -y

AlmaLinux

dnf install jq -y

Obtain an authorization token. To do this, send a POST request to a URL in the following format https://domain.com/auth/v4/public/token. For more information, see the Authorization by token in the article API guide.

Obtain the task data by sending a request to the taskmgr:

curl -s -k -X GET -H "accept: application/json" -H "Content-Type: application/json" "https://<domain>/taskmgr/v4/task?where=request_id%20EQ%20'<request_id>'" -H "x-xsrf-token: <token>" | jq -r '.list[] | select(.status == "failed") | "=========================================================", (.stdin.metadata.name), "", .name, "", .id, "", (.callback_params | tostring), "========================================================="'

Explanation:

<domain> — server address with the platform (IP address or domain name);
<request_id> — ID of the migration task obtained in step 1;
<token> — authorization token obtained in step 3.

Example command

curl -s -k -X GET -H "accept: application/json" -H "Content-Type: application/json" "https://10.5.11.10/taskmgr/v4/task?where=request_id%20EQ%20'c53699f24ce3b06f17801c4859238cf1'" -H "x-xsrf-token: 10006262-01dd7b7e-4284-42d7-a458-8d195bf9af28" | jq -r '.list[] | select(.status == "failed") | "=========================================================", (.stdin.metadata.name), "", .name, "", .id, "", (.callback_params | tostring), "========================================================="'

The response will contain the following data:

Example output

=========================================================
blue-rubellite

host_migrate

10942

{"disks":[{"id":355,"storage":1}],"host_id":191,"node_id":6,"is_dummy":false,"is_master":false,"interfaces":[{"id":198,"dhcp":false,"name":"ens3","model":"virtio","queues":1,"is_vxlan":false,"vxlan_id":0,"mac_address":"52:54:00:57:2D:BC","node_interface":14,"cluster_interface":0,"node_interface_name":"vmbr0"}],"dst_node_id":8,"host_is_active":true,"security_log_action":"[1787719398009][account_id 100000] Migration host id 191","linked_clones_cold_migration":true}
=========================================================

Note

For a group migration, the output contains several blocks separated by

===

. Check each block and follow the instructions for all problematic VMs.

Explanation:

blue-rubellite — VM name. Make sure it matches the target VM;

Important
If the VM was renamed manually in the platform, the name in the API output may not match. In this case, use the VM ID for verification: compare the host_id value inside callback_params with the VM ID in the web interface — they will match.
host_migrate — task name;

Warning
The output must contain exactly the value host_migrate. If the task name differs, the instructions are not applicable. Contact technical support.
10942 — task ID;
{"disks":[...]} — callback_params — parameters used to synchronize data.

Send an API request to complete the migration task:

docker exec vm_back curl -X POST -H 'Instance-ID:1' -H 'internal-auth:on' -o- -D- http://localhost:1500/vm/v3/task/<task_id>/success?name=host_migrate -d '{"output":{"content":"","type":"text"},"params":<callback_params>}'

Explanation:

<task_id> — ID of the migration task obtained in step 5 (in the example — 10942 );
<callback_params> — parameters from step 5 (in the example — {"disks":[...]} ).

Example command

docker exec vm_back curl -X POST -H 'Instance-ID:1' -H 'internal-auth:on' -o- -D- http://localhost:1500/vm/v3/task/10942/success?name=host_migrate -d '{"output":{"content":"","type":"text"},"params":{"disks":[{"id":355,"storage":1}],"host_id":191,"node_id":6,"is_dummy":false,"is_master":false,"interfaces":[{"id":198,"dhcp":false,"name":"ens3","model":"virtio","queues":1,"is_vxlan":false,"vxlan_id":0,"mac_address":"52:54:00:57:2D:BC","node_interface":14,"cluster_interface":0,"node_interface_name":"vmbr0"}],"dst_node_id":8,"host_is_active":true,"security_log_action":"[1787719398009][account_id 100000] Migration host id 191","linked_clones_cold_migration":true}}'

Example of a successful response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Content-Length: 4
Connection: keep-alive

The command passes callback_params for correct synchronization of data about the VM location in the platform DB.

Check the result in the web interface. Make sure that:
1. The migration task has the status Successfuly.
2. The VM is displayed on the destination cluster node.

Useful tips