Reading time: 7 minutes
Business intelligence (BI) tools are needed to collect, analyze and interpret data within business processes. They help organizations make informed decisions, optimize efficiency, monitor performance and predict future trends. Creating tools to perform analytical tasks on your own can require a lot of time, effort, and a separate staff of technical specialists. BI systems help automate the process, making business intelligence convenient and flexible. These technologies allow both decision makers and business analysts to build graphical reports using data that was not originally designed for analytics.
How BI systems work
- All BI tools use the following algorithm:
- All types of data, both structured and unstructured, are extracted from various sources (transactional DBMS, web services, message brokers, text documents, etc.).
- The extracted data is converted into a convenient form for analytics.
- The converted data is uploaded to a specialized analytical repository.
- Data from the analytical repository is visualized in the form of reports and graphs.
BILLmanager already has a built-in mechanism for generating reports and graphs. The user receives several predefined reports out of the box. In addition, they can create their own reports. However, this mechanism is far from the principles on which modern BI-systems are built:
- The customization options are quite sparse;
- Data is extracted directly from the BILLmanager database;
- Table structures are complex;
- Just a SQL query alone is not sufficient for building charts;
- Fine-tuning requires developers to write code in C++.
BI system implementation in BILLmanager
We decided to provide our clients with a more flexible and modern tool that will meet all the requirements and principles of modern BI systems. It is closely integrated with BILLmanager, which allows you to take advantage of all the benefits that BI system offers to accomplish business objectives.
As a result of studying and testing various solutions (including by overseas developers), we came to the need to develop our own tools that would meet our requirements as well as the requirements of our clients - both current and future BILLmanager users.
We identified the following components, which are necessary to create your own BI system.
ETL (Extract, Transform, Load)
Before you visualize the data, you need to bring them to an appropriate view: download from different sources, combine tables, normalize, make additional transformations and transfer the result to a specialized analytical data storage. This is the task of the ETL component - which we have developed ourselves.
Initially, ETL is already configured to download certain data sets from BILLmanager, such as information about clients, their costs and consumed services. Because all settings are described in the configuration files, it should not be difficult for the user to connect new data sources or specify their own data conversion rules. Knowledge of programming languages is not required.
You can also use sources other than BILLmanager: databases, web services and spreadsheets. Different conversion rules can be specified for the received data. Such rules include, for example, conversions to other data types, formatting strings, clearing unnecessary data, or constructing missing data.
For periodic downloading, a schedule must be specified: when and how often you want to synchronize data from sources.
Analytical data storage
Traditional transactional DBMS are poorly suited for analytical queries, mainly because of their low processing speed. For analytical data storages, the so-called columnar DBMSs are often used as a replacement for traditional DBMSs. They are characterized by weak support for transactions or the lack thereof, as well as a special structure of data storage and processing, due to which such DBMS perfectly cope with analytical queries.
One widely known and actively maintained columnar DBMS is Clickhouse. Its distinctive feature is flexible scalability. This DBMS can be used both on virtual servers with relatively little RAM and in a large cluster. If the amount of stored data is small, you can most often make do with a small amount of computing resources.
Data visualization
The data obtained after processing need to be visualized: build and view reports and graphs, combine them into dashboards. We use Apache Superset for this purpose, which is one of the most flexible and customizable tools. It is a freeware solution used by many companies and organizations around the world. The open source code allows you to make changes and modify it to suit your needs.
Why the BI system in BILLmanager is convenient and efficient
A successful BI system requires that all of its components be successfully integrated with each other so that users do not have to deal with time-consuming deployment and pre-configuration.
When designing BI tools in BILLmanager, we wanted to satisfy the following requirements: ease of installation, configuration and operation:
- The BI system is installed from the graphical interface of the BILLmanager panel. It can be installed on the same server where BILLmanager is already installed (a trial version) or on a separate server (recommended production version). No additional actions by the panel and server administrator are required before installation.
- The BI system is integrated with BILLmanager. We initially provide a set of reports and graphs that are built using data from BILLmanager. There is integration of BILLmanager employees and BI users, which allows you to differentiate access to certain data, if necessary.
- Users of the BI system can add new data sets from different sources, build their own charts and reports without having to dive into technical details for maximum focus on business tasks.
- Provide users in the future with advanced analytics capabilities, flexible forecasting and recommendations based on the generated forecasts.
Implementation of a proprietary BI system in BILLmanager allowed us to make business analytics convenient and easy. Built-in tools and ready sets of dashboards, charts and reports allow both subject matter experts and management, as well as ordinary users to work with the system. Integration with external data sources allows you to supplement analytics with any information from outside.
Built-in BI tools for building analytical reports are available in any version of BILLmanager, even on the most affordable tariff plan.
Request a demo to get to know the BI system built into the BILLmanager platform.