30 June 2021 Reading time: 19 minutes

Alexander Grishin

VMmanager Product Owner

What improvements in VMmanager 6 will help to get more profits

What improvements in VMmanager 6 will help to get more profits

 

ISPSystem

Alexander Grishin, VMmanager Product Owner, has explained the product features that will help to build a fault-tolerant and cost-effective virtual infrastructure. At the MeetUP, held on April 14, he shared the results of the product development over the year and future plans.

VMmanager 6: from panel to platform

 

Many are familiar with the previous, the fifth version of VMmanager. It came out about seven years ago. At that point, it was a good, simple and reliable web panel – a web interface over Libvirt. However, time is not standing still, and technology is moving forward. Meanwhile, the expectations of customers and administrators are rising. So today I will talk about the new version of our product – the VMmanager 6 virtualization platform. Because VMmanager 6 is more than just a panel. It is a powerful multi-component asynchronous microservice tool for virtual infrastructure administration.

Microservice architecture of the VMmanager platform

As applicable to business goals, VMmanager 6 provides the following:

  • Orchestration of virtual machines and containers;
  • Tool for monitoring, statistics and analytics on the parameters in the system;
  • Tracking tasks in the platform: by who, when and what tasks were run, what their status is;
  • User and role management;
  • Notifications of events to your email or Telegram;
  • Automatic provisioning of resources to the customer;
  • Backups management;
  • Automatic maintenance of hypervisors and nodes;
  • Hypervisor occupancy policies;
  • Accounting and management of IP addresses;
  • Integration with PowerDNS, DNSBL;
  • CEPH, SAN, IP-Fabric, High availability and much more.
VMmanager - a tool to build a virtual infrastructure based on LXD and KVM

Over the past year we have implemented a lot of new functionality. In conversations with clients, I often hear that VMmanager now and a year ago are two different products. It is actually quite true. Here are some of the features of VMmanager that I will talk about:

 

  • Ceph distributed storage;
  • SAN network storage;
  • Unbreakable clusters;
  • Lightweight and productive LXD containerization and ZFS;
  • Notifications to email and Telegram;
  • BGP-based IP fabric;
  • SaaS support.

 

I chose them not because they are buzzwords or new fancy technologies. Each of these improvements will help take your business to the next level and bring you closer to the main goal of any company making more profit.

Distributed Ceph storage in VMmanager

 

Serh is the most budget-saving way to set up network storage for your infrastructure. Ceph can be located remotely from the main virtualization cluster and not depend on it. It has the following advantages:

  • Fault tolerance: if one node fails, data is automatically copied to free nodes;
  • Replication: you can specify how many times to copy client data;
  • Scalability: physical server disks can easily be added to the cluster.

 

This way you get network storage easily and at a reasonable cost. You can start using Ceph in VMmanager 6 right now.

How Ceph fault-tolerant storage works. Data is divided into blocks. Each block is stored in two copies on different nodes.
Data recovery in Ceph. When one of the nodes fails, data replicas are duplicated to other nodes

SAN network storage (iSCSI/FC) in VMmanager

 

In addition to Ceph, there is a second way to solve the problem of data storage. This is the SAN (Storage Area Network) hardware solution. The SAN is a disk rack connected via Fibre Channel or iSCSI protocols to the hypervisor nodes.

 

SAN has all the advantages of Ceph. Nevertheless, because it is a hardware solution, it provides even better speed, reliability, scalability and ease of maintenance. SAN enables efficient use of rack space because more physical disks fit in the disk racks than in the server.

Unbreakable clusters in VMmanager

 

When using Ceph or SAN storage in VMmanager, you can create fault-tolerant unbreakable clusters. We have added the high availability technology, which will take care of the reliable operation of virtual machines. In case of a crash or a node failure, virtual machines will be automatically recovered on free nodes of the cluster.

When using Ceph or SAN storage in VMmanager, you can create fault-tolerant unbreakable clusters. We have added the high availability technology, which will take care of the reliable operation of virtual machines. In case of a crash or a node failure, virtual machines will be automatically recovered on free nodes of the cluster.

When implementing this functionality, we took into account the experience gained with VMmanager 5, and studied our competitors. In particular, we were inspired by this concept in VMware.

 

To implement fault tolerance, we needed to solve the classic problem of achieving consensus in distributed systems. Therefore, we adapted the algorithm from the Paxos family to the specifics of VMmanager.

Paxos algorithm operation in VMmanager

One of the nodes takes on the role of the leader. The leader makes decisions at the cluster level. The Paxos algorithm ensures that there can only be one leader in a cluster at any time.

 

In our case, two main factors are used to determine the availability of the node:

  • Connectivity at the control network level;
  • Connectivity at the network storage level.

 

A secondary factor is the external verification IP address. It allows you to determine the situation when a node has lost access to the control network, but the virtual machines still have access to the Internet or another required network. In this case, there is no need to take any recovery or relocation actions.

How fault-tolerant clusters are configured in VMmanager

 

A fault-tolerant Ceph storage is configured for the cluster. High availability can also be used with SAN
There are three nodes in the cluster, one of which is Master and the others are Slaves. To use high availability in VMmanager, two nodes would be sufficient
For each virtual machine, you can assign a priority setting. If the node is damaged, the VM with the higher priority will be recovered sooner

 

 

You can use additional fault tolerance settings – an external validation IP address.

It does not matter if you are using a Ceph software storage or a hardware SAN storage. Either of them allows building a fault-tolerant virtual infrastructure. This, in turn, helps guarantee a higher level of SLA for customers.

 

Read more about fault-tolerant clusters in VMmanager

 

High availability clusters are now available in VMmanager

LXD/LXC containerization

 

Infrastructure containers based on the LXD “lightvisor”

We received a request from our customers for an easier virtualization than KVM, so we added support for LXD containers to VMmanager. A close analogue of this technology is OVZ. However, LXD is a more modern and technologically advanced virtualization. It is included in the Linux kernel and does not require patches.

 

LXD is a very high-performance technology. It is often called a lightvisor, and this title is well deserved. We can check the speed for ourselves.

 

With LXD, you can provision services in seconds.

 

Creating an LXD container on the VMmanager testbench. A virtual machine is used as a node. Despite this, the container was created in 7 seconds. On a good node, you can achieve the speed of 1-4 seconds before delivery of the finished service to the client

LXD increases the return on equipment. We have done a comparison of KVM and LXD utilization.

 

Test conditions

The test involved a dedicated server based on an Intel(R) Xeon(R) CPU E3-1230 v3 with a 16 GB RAM configuration. It was used to run LXD containers and KVM servers.

 

We simulated an average client's VPS with a website. Instances were deployed with the following settings:

  • ОС CentOS 7,
  • PHP 74-fpm,
  • MySQL 5.5 database,
  • WordPress.

 

Parameters measured:

  • Deployment time;
  • Number of containers deployed.

 

With LXD, the speed of service provision increases tenfold
LXD provides high virtualization density
Reboot speed of a fully loaded and five times densely packed node with LXD is about 40 times faster than with KVM

High utilization is achieved primarily through more efficient use of CPU and RAM. But to make sure that storage does not become a bottleneck for your business, we have implemented support for ZFS storage for image storage.

 

The platform reuses the LXD image through ZFS Snapshots technology: only one instance of the OS is stored on the node. In addition, each container owner has only the difference between the parent image and the user image written to disk.

ZFS storage is used with LXD virtualization. Images for LXD containers take less space than KVM-based virtual machines

LXD has a limitation: this technology can only be used on Linux operating systems. You cannot use it to provide services on Windows. However, this virtualization is great for Linux tasks and can significantly increase the utilization of the three main resources: RAM, Storage, CPU.

 

More on LXD virtualization in VMmanager

 

As fast as Docker, as reliable as KVM. LXD containers – a new type of virtualization in VMmanager

 

Testing KVM and LXD technologies

 

Working options for using virtual machines, Docker, LXD and Kubernetes

 

Notifications to email and Telegram

The main purpose of the notifications is to increase the level of customer service. In VMmanager 6 you can set up notifications to Telegram or email. If a node, virtual machine, or task list error occurs, administrators will know about it before the client calls for help. The speed of response to incidents will increase.

Notification service in VMmanager

With the notification service you can set up messages on errors on nodes and virtual machines, as well as on exceeding storage, CPU, RAM parameters. This will help professionals respond proactively to avoid typical mistakes. For example, the node will not run out of storage unexpectedly, and the client will be satisfied with the level of service.

More about monitoring and notifications in VMmanager

 

Monitoring in VMmanager: measuring is managing

BGP-based IP fabric in VMmanager

IP-fabric is a tool for network abstraction and our know-how.

IP-fabric schematic diagram in the platform

How IP-fabric works

 

The first level is the usual private flat L3 network for hypervisors. Only your specialists (VMmanager administrators) have access to it – it is a closed circuit. It functions as the underlay (management network).

 

On top of the first level, overlay layers are created for each virtual machine. This is a peer-to-peer connection, on a /32 mask with a public address. Routing of such a public address is carried out at the underlay layer using the BGP protocol.

 

Networks are built in isolation for each customer.

 

IP-fabric offers the following advantages:

  • Financial benefit: public IP addresses savings. The larger the company's infrastructure, the greater the savings.
  • Increased security: the infrastructure is in a closed circuit, which is isolated from clients. If necessary, it can also be isolated from the Internet through a firewall.
  • Broadcast traffic is reduced: network performance increases several times over. This has a positive effect on customer satisfaction: everyone likes a network that works quickly and efficiently.

 

More about IP-fabric in VMmanager

 

IP-fabric: the new network configuration in VMmanager

SaaS support

 

VMmanager 6 supports SaaS services. You can package your specialists' knowledge and offer it as a service. Clients rarely just need to rent resources (CPU, storage, RAM). Sometimes they do not care if they take one virtual machine or two –– they just need to solve their problem. You can provide this solution with tools in the VMmanager platform:

  • OS repositories;
  • scripts.

 

OS repositories

In VMmanager, you can connect your repositories with custom OS. This way you can install and pre-configure the software.

You can connect your own repositories with custom OS and preconfigured software in VMmanager

Scripts

 

The script tool helps to provide services for each client and to pass the client's data for software fine-tuning.

 

Selecting scripts in VMmanager

In VMmanager, you can use a large number of variables already built in: email, get away, IP address and others.

 

Variables repository for SaaS use

At the script level you can:

  • Manage access;
  • Assign the owner of the script;
  • Hide the contents of the script from the end user in order to preserve intellectual property;
  • Use parameters to request data from the user;
  • Configure the multi-digit preset parameters. This allows you to give the user a choice of ready-made values.

 

Passing parameters to a script for fine-tuning an application on a VM
Creating preset parameter values in the script

All of these tools help customize the software for a particular user: with the settings, parameters, and integrations he needs. It does not matter if it is one or two virtual machines, with databases installed, or a set of software for web programming. The main thing is that the client will be able to use the finished service.

 

In the latest update, we have added support for Powershell scripts for VMs on Windows systems.

 

Powershell scripts for Windows

More about SaaS in VMmanager

 

CI/CD as a service in the VMmanager platform

 

How we automated testbench deployment using Gitlab and VMmanager to save 15 man-hours per week

VMmanager: results of the year and achievements

The size constraints of this report will not allow me to tell you about the slightly less significant improvements we added. You can read about some of them in the ISPsystem blog materials.

 

New features in VMmanager that appeared over the year. Twenty tools for infrastructure administration

 

VMmanager has evolved a lot over the year. Each of the added functionalities is driven by financial gain. It is designed to improve the quality of services and enhance customer service:

  • Increase user satisfaction;
  • Utilize physical resources better;
  • Enable quicker return on investment;
  • Save IP addresses;
  • Increase network performance;
  • Respond to incidents faster;
  • Expand the range of services provided, including SaaS;
  • Ensure fault tolerance and high SLA;
  • Ensure the security of your own infrastructure and that of the client's.

 

Finally, I will tell you about two more achievements:

 

Rock911 Labs specializes in software security audits. Its experts have recognized VMmanager as one of the safest virtualization platforms.

 

Renowned review aggregator G2 awarded VMmanager as High Performer in the virtualization product market last year.

 

VMmanager award by G2

A user-friendly informative interface

 

We are working to improve the interface further. You can now customize the appearance of tables in VMmanager: choose what information to display. For example, engineers can customize the interface to get more information about nodes or other entities.

 

VMmanager table settings. Users choose what information to display.

Планы

 

New informative design. We are already moving in that direction. Recently, we added a compact view to the tables and the ability to customize the display of information in the interface. In the future, we are planning to launch a completely new product interface.

 

Hyperconverged clusters — the ability to configure Ceph directly from the platform interface, on the same nodes used as hypervisors.

 

Policy for tag-based allocation of virtual machines, as well as DRS/sDRS balancing of already created VMs.

 

Proactive High availability — this functionality will be working in combination with DCImanager (a platform for managing physical equipment). It will predict the failure of a node and move critical virtual machines to a running node in advance.

 

Flexible networks (Bridge) he ability to create bridge bonds directly from the interface and to map their virtual machines. This is convenient for those who prefer a flat network. The release is expected in June 2020.

 

vIaaS (virtual infrastructure as a service) VMmanager will allow you to provide users with a virtual infrastructure, which is abstracted from the physical and network equipment provider. The user will be able to manage a pool of virtual machines, public IP addresses, create any number of private networks for virtual machines. This will be possible thanks to VxVLAN eVPN technologies.

Q&A

 

Does VMmanager support managing clusters in multiple datacenters? So that the images from which the virtual machines are deployed do not need to be moved between data centers, but stored at each location.

Yes, it does. OS templates are stored at cluster nodes.

 

I need to deploy ready-made SQL services, proxies, mail services, similar to what Yandex does. Are you planning to provide this functionality?

Precisely for this purpose, we have implemented the ability to provide SaaS services. You can do it on your own right now.

 

If there is a relevant request, we will consider providing ready-made services. We could digitize some of our engineers' knowledge. However, every business provides services to customers in a different way. Therefore, in my opinion, the right thing to do is to use a SaaS tool and configure it individually for your clients, the way that you think is right.

 

Try VMmanager 6