Unable to start the libvirt pool. "Duplicate hash table key" error

Problem

After a failure in the Ceph storage, operations with virtual machines (VMs) cannot be performed: creation, deletion, and OS reinstall on a VM. At the same time, the following issues are observed on one or more cluster nodes:

• libvirt pool is in the inactive state:
  Pool status example

  #virsh pool-list --all
   Name State Autostart
  -----------------------------------
   cloud_hdd inactive yes
  

• The libvirt pool is started with the virsh pool-start command. An error occurs when trying to start it:
  
  Error example when trying to start the pool

  # virsh pool-start cloud_hdd
  error: Failed to start pool cloud_hdd
  error: internal error: Duplicate hash table key 'cloud_hdd/15139_vm733939'

• In libvirtd logs, there are errors retrieving metadata from the RBD pool and retrieving image data:
  

  Feb 06 12:56:25 node20853.dev.example.com libvirtd[3250509]: 2026-02-06T12:56:25.279+0300 7f8dd2ffd700 -1 librbd::image::RefreshRequest: failed to retrieve pool metadata: (110) Connection timed out
  Feb 06 12:56:25 node20853.dev.example.com libvirtd[3250509]: 2026-02-06T12:56:25.279+0300 7f8dd2ffd700 -1 librbd::ImageState: 0x7f8da00a70a0 failed to open image: (110) Connection timed out

• VMs that use images from the problematic pool do not start. On running VMs with a problematic image, the No bootable device error occurs inside the guest OS.

Cause

The Duplicate hash table key error occurs on the libvirt side. 

When the pool starts, libvirt contacts the Ceph cluster through the librados library, gets the list of RBD images, and builds an internal data table where each image name corresponds to a unique record (key). The normal behavior is when each name appears once. In this case, there are physically two RBD images with the same name in Ceph. When libvirt gets such a list, it tries to add two identical keys to its table. This breaks the logic.

Diagnostics

To confirm the cause of the issue:

1. Connect to the cluster node over SSH. For more information about SSH connection, see Workstation setup.
2. Get the names of all libvirt pools on the cluster node:

   virsh pool-list --all

   Output example

   [root@node20857 ~]# virsh pool-list --all
    Name State Autostart
   -----------------------------------
    cloud_hdd inactive yes
    cloud_ssd active yes

3. Run a start attempt for the inactive pool:
   

   virsh pool-start <pool_name>

   Sign of the issue: the output contains the following error:

   error: Failed to start pool cloud_hdd
   error: internal error: Duplicate hash table key 'cloud_hdd/15139_vm733939'

4. Connect to the Ceph network storage.
5. Make sure the image of the problematic VM is duplicated:

   rbd ls <pool_rbd>|grep <vm_name>

   Output example

   [root@node20833 ~]# rbd ls cloud_hdd|grep vm733939
   15139_vm733939
   15139_vm733939

   The presence of two identical lines in the output confirms the cause of the issue.

Solution

To solve the issue:

1. Connect to Ceph.
2. Delete the duplicate broken image.
3. Start the pool on all cluster nodes.

If both images are broken, contact your system administrator to recover the data.

Preparation

Check whether the VM disk exists on the cluster node:

1. Connect to the server with the platform over SSH. For more information about SSH connection, see Workstation setup.
2. Determine the internal name and cluster node of the target VM:
   

   docker exec vm_back vssh -a 'virsh list --all | grep <VM_name>

   Output example

   [root@VMmanager10 ~]# docker exec vm_back vssh -a 'virsh list --all | grep grayish-indicolite '
   multiple errors:
   node 6:
   Node 7:
    5    28_grayish-indicolite    running

   Explanations:

   • 7 is the cluster node ID;
   • 28_grayish-indicolite is the internal VM name.
3. Connect to the cluster node with the target VM over SSH. For more information about SSH connection, see Workstation setup.
4. Run the command: 
   

   virsh domblklist <VM_internal_ВМ>

   Output example when the disk is present

   virsh domblklist 28_grayish-indicolite
    Target     Source
   ---------------------------------------------------------------
    vda        cloud_hdd/13516_vm715346

   If the disk is present, there is a high probability that one of the images is available.

Perform an exact diagnosis of image status:

virsh shutdown <vm_name>

virsh destroy <vm_name>

1. Connect to the Ceph network storage.
2. Rename one of the images:

   rbd rename <pool_rbd>/<image_name> <rbd_pool>/<image_name>_rename
   

   Explanations:

   • <pool_rbd>, <image_name> — specified in step 5 of the “Diagnostics” section.
   Output example

   rbd rename cloud_hdd/13516_vm715346 cloud_hdd/13516_vm715346_rename

3. Check the images of the target VM:

   rbd ls <pool_rbd>|grep <vm_name>

   Output example

   [root@node20833 ~]# rbd ls cloud_hdd | grep vm715346
   13516_vm715346_rename
   13516_vm715346

   Expected result: the output contains two different names.

4. Check the status of each image:

   rbd info <pool_rbd>/<image_name>

   If the command returns metadata (size, format, etc.), the image is available. If the output contains the error error opening image: (2) No such file or directory, the image is broken.

   Output example when the image is available

   [root@node20833 ~]# rbd info cloud_hdd/13516_vm715346_rename
   rbd image '13516_vm715346_rename':
    size 80 GiB in 20480 objects
    order 22 (4 MiB objects)
    snapshot_count: 0
    id: 9bc1a0963ddfac
    block_name_prefix: rbd_data.9bc1a0963ddfac
    format: 2
    features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
    op_features:
    flags:
    create_timestamp: Sat Feb 7 12:36:46 2026
    access_timestamp: Sat Feb 7 19:35:07 2026
    modify_timestamp: Sat Feb 7 19:34:55 2026

   Output example when the image is broken

   [root@node20833 ~]# rbd info cloud_hdd/13516_vm715346
   rbd: error opening image 13516_vm715346: (2) No such file or directory

Choose the solution scenario:

• go to the One of the images is available section if:
  • the disk is present in the output of virsh domblklist;
  • one of the images in the rbd info output returns metadata, and the other returns an error;
• go to the Both images are broken section if:
  • the disk is absent from the output of virsh domblklist;
  • both images return the error opening image: (2) No such file or directory error in the output of rbd info;
• contact your system administrator if you encounter unusual scenarios:
  • the disk is present, but both images are broken (the disk in the VM configuration points to a non-existent image);
  • the disk is absent, but one of the images is available (the image exists in storage, but the VM does not reference it).

One of the images is available

After completing the steps in the Preparation section, you have confirmed that one of the images is available.

To solve the issue:

1. Delete the broken image:

   rbd rm <pool_rbd>/<image_name>

   Output example

   [root@node20833 ~]# rbd rm cloud_hdd/13516_vm715346
   Removing image: 100% complete...done.

2. Rename the available image back to the original name:

   rbd rename <pool_rbd>/<image_name>_rename <rdb_pool>/<image_name>

   Command example

   rbd rename cloud_hdd/13516_vm715346_rename cloud_hdd/13516_vm715346

3. Make sure there are no duplicates, and that the disk name in the VM configuration matches the image name in Ceph storage. The disk name is obtained in the output of the command virsh domblklist, step 4 of the “Preparation” section.
   Attention!

   If a mistake was made during renaming and the final image name in Ceph does not match the disk name in the VM configuration, the VM will not be able to connect to the disk from Ceph when it starts. Inside the guest OS, this will appear as no bootable device: the No bootable device error.

   Run the command:

   rbd ls <pool_rbd> | grep <name_vm>

   Expected result: the output contains one line.

   Output example

   [root@node20833 ~]# rbd ls cloud_hdd|grep vm733939
   15139_vm733939

4. Connect to the cluster node over SSH. For more information about SSH connection, see Workstation setup.
5. Start the libvirt pool:

   virsh pool-start <pool_name>

   Output example

   [root@node20853 ~]# virsh pool-start cloud_hdd
   Pool cloud_hdd started

6. Make sure all pools are started:

   virsh pool-list --all

   Output example

   [root@node20853 ~]# virsh pool-list --all
    Name State Autostart
   ---------------------------------
    cloud_hdd active yes
    cloud_ssd active yes

   If the pool does not start or the error points to another problematic image, this means there is another duplicate image in the pool. In this case, repeat the procedure from step 1 for the new image mentioned in the error.

7. Start the VM through the web interface or with the command:

   virsh start <internal_vm_name>

8. Connect to the server with the platform over SSH. For more information about SSH connection, see Workstation setup.
9. Start the libvirt pool on all cluster nodes. To do this, run the command on the server with the platform:

   docker exec vm_back vssh -a 'virsh pool-start <rbd_pool>'

   Output example

   [root@VMmanager10 ~]# docker exec vm_back vssh -a 'virsh pool-start cloud_hdd'
   multiple errors:
   node 6: error: Failed to start pool cloud_hdd
   error: Requested operation is not valid: storage pool 'cloud_hdd' is already active
   node 7:
   Pool cloud_hdd started

   If there is no such storage on the cluster node, the command will fail with the following error:

   node <id>: error: failed to get pool '<pool_name>'
   error: Storage pool not found: no storage pool with matching name '<pool_name>'

   This is normal behavior and not a problem.

10. Make sure the following error is not present in the output:

    error: Failed to start pool cloud_hdd
    error: internal error: Duplicate hash table key 'cloud_hdd/15139_vm733939'

    If the error is present, this means that problematic VMs with duplicate images remain in the pool. Repeat the procedure for all VMs listed in the error output.
    Expected result: 
    
    • pools on all nodes of the target cluster are started without the Duplicate hash table key error;
    • a message appears on each cluster node when the pool is started:

      storage pool 'cloud_hdd' is already active

Both images are broken

After completing the steps in the Preparation section, you have confirmed that both images are broken.

If the data on the VM disk is valuable, contact the system administrator to try to recover the broken RBD images. Recovering images in Ceph is a separate task that requires special knowledge and tools, and is beyond the scope of this guide.

If recovery is not possible or not required, follow these steps:

1. Connect to the Ceph network storage.
2. Rename both broken images:
   

   rbd rename <pool_name>/<image_name_1> <pool_name>/<image_name_1>_bad_image
   rbd rename <pool_name>/<image_name_2> <pool_name>/<image_name_2>_bad_image_2

   Command notes:

   • _bad_image, _bad_image_2 are suffixes for the new names of the broken images.
   Output example

   rbd rename cloud_hdd/15139_vm733939 cloud_hdd/15139_vm733939_bad_image
   rbd rename cloud_hdd/15139_vm733939_rename cloud_hdd/15139_vm733939_bad_image_2

   This is needed to:
   • eliminate the duplicate-name conflict when starting the pool;
   • prevent the VM from accidentally attaching to one of the images;
   • keep the images for possible recovery by the system administrator.
3. Check the status of each image:
   

   rbd info <pool_name>/<image_name>_bad_image
   rbd info <pool_name>/<image_name>_bad_image_2

   If both outputs contain error opening image: (2) No such file or directory, the images are unreadable.

4. Delete both broken images:
   
   Caution

   If the broken images are needed by your system administrator for data recovery, do not delete them. Skip this step and go to step 6. At this point, the images have already been renamed and do not block pool start.

   rbd rm <pool_name>/<image_name>_bad_image
   rbd rm <pool_name>/<image_name>_bad_image_2

5. Connect to the cluster node over SSH. For more information about SSH connection, see Workstation setup.
6. Start the libvirt pool on the cluster node:
   

   virsh pool-start <pool_name>

7. Connect to the server with the platform over SSH. For more information about SSH connection, see Workstation setup.
8. Start the pool on all cluster nodes:
   

   docker exec vm_back vssh -a 'virsh pool-start <pool_name>'

   If there is no such storage on the cluster node, the command will fail with the following error:

   node <id>: error: failed to get pool '<pool_name>'
   error: Storage pool not found: no storage pool with matching name '<pool_name>'

   This is normal behavior and not a problem.
   Expected result: pools on all nodes of the target cluster are started without the Duplicate hash table key error.

9. If the images were deleted in step 5, recreate the VM through the platform web interface. See Creating a virtual machine for details.