Connecting a network LVM storage

You can connect a network LVM storage to the cluster. Before that, configure the storage area network (SAN) and connect the storage to the cluster nodes as a block device. Read more in Pre-configuring SAN.

Summary of LVM

Logical Volume Manager (LVM) is a subsystem that allows you to use different areas of the same hard disk and/or areas from different hard disks as one logical volume. The size of file systems of logical volumes is not limited to one disk, as the volume can be located on different disks and partitions.

Main LVM designations:

• PV (Physical Volume) — partitions of disks or entire "unpartitioned" disks;
• VG (Volume Group) — a set of physical PV volumes combined to form a single disk;
• LV (Logical Volume) — partitions created on the physical space of the VG volume group.

LVM storage supports one virtual disk image format — RAW. Read more about LVM in the official documentation.

In VMmanager, LVM storage is used only for storing virtual machine (VM) disks. Each disk is stored in a separate LV with the name in the "<vm_id>_<vm_name>” format.

Configuring the multipath

Multipath is a technology for connecting storage network nodes using multiple routes. If one controller fails, the operating system will use another controller to access the device. This increases system fault tolerance and allows for load balancing. Multipath devices are combined into one using specialized software.

To configure multipath:

1. Install the multipath-tools software:

   Ubuntu

   apt install -y multipath-tools
   

   AlmaLinux

   dnf install device-mapper-multipath

2. In the configuration file /etc/multipath.conf, change the value of user_friendly_names to no:

   user_friendly_names no
   

3. Restart the multipathd service:

   systemctl restart multipathd
   

4. On each storage node, combine all disk connections into a single multichannel connection:

   multipath /dev/sdX
   

   Comments to the command

   /dev/sdX is the device for the first disk. For example, /dev/sda

5. The multipath command will group all paths into a single mapper device. To see the list of grouped devices, run the command:

   multipath -ll
   

   This device will have the same ID on all nodes.

   Example output

   root@node1:~# multipath -ll
   36589cfc0000008af95b3633c714c5c57 dm-0 TrueNAS,iSCSI Disk
   size=550G features='0' hwhandler='1 alua' wp=rw
   `-+- policy='service-time 0' prio=50 status=active
     `- 2:0:0:0 sda 8:0 active ready running

6. On each storage node, add a filter to the devices {...} block in the LVM configuration file /etc/lvm/lvm.conf:

   filter = [ "a|/dev/mapper/.*|", "r|.*|" ]
   

   Comments

   a|/dev/mapper/.*| — enables /dev/mapper/ devices
   r|.*| — disables all other devices

   If you need to use other disk paths, specify them in the filter. For example, filter = [ "a|/dev/disk/by-id/.*|", "r|.*|" ]

   If the system disk uses LVM, add it to the filter:

   Example config for /dev/sda

   filter = [ "a|/dev/mapper/.*|", "a|/dev/sda|", "r|.*|" ]
   

7. Verify that the filter and configuration are correct with the command:

   vgs
   

   If everything is configured correctly, the VG list in the command output should contain no errors.

How to connect a storage

To connect the storage, enter Clusters → select the cluster → menu → Network storages:

1. Click Add storage → Add a network storage → SAN.
2. Specify its settings:
3. 1. Name.
   2. Storage tags. Read more in the Selecting storage for VM disks.

   3. Path to block device. For example, /dev/sdb.

      If you used UUID or WWID to connect the iSCSI target when you configured the SAN, specify the UUID or WWID path:

      • for UUID: /dev/disk/by-uuid/;
      • for WWID: /dev/disk/by-id/.

      If you used multipath, specify a path like /dev/mapper/.

4. Click Add.

VMmanager will automatically configure the LVM storage on the block device: it will create PV and VG. If a VG has been created on the block device, the storage will not be connected.