How to add a custom processing module

The article describes the creation of the pmollama service processing module. The handler deploys local Ollama AI models in isolated Docker containers on a dedicated provider server. The following are covered:

• integration with the BILLmanager Python SDK (Software Development Kit);
• working with add-ons and parameters;
• support for CPU and GPU modes with resource limits through NVIDIA MPS;
• common errors when creating handlers.

General information

Processing modules manage the service lifecycle in external systems:

• integrated platforms;
• provider APIs;
• certificate authorities;
• domain registrars.

A standard BILLmanager processing module consists of an executable file and an XML description. Below are the features specific to the pmollama module.

Installation directory: /usr/local/mgr5/processing/
XML description: /usr/local/mgr5/etc/xml/billmgr_mod_pm<name>.xml

Required command:

• features — returns an XML response with the following parameters:
  • <itemtypes>;
  • <params>;
  • <features>;
  • <templates> (for SSL certificates).

Main commands (common for most processing modules):

• open — create a service;
• suspend — suspend a service;
• resume — resume a service;
• close — delete a service;
• setparam — change service parameters or switch the tariff;
• prolong — renew a service;
• sync_item — synchronize the service status with the external system;
• check_connection — check connection parameters to the external system;
• tune_connection — modify the form for adding a processing module.

Specific commands for domain registrars:

• transfer — transfer a domain;
• import — import existing domains;
• WHOIS — get domain owner information through WHOIS;
• update_ns — change DNS servers;
• get_contact_type — list contact types for a domain zone;
• tune_service_profile — configure contacts;
• validate_service_profile — validate contacts;
• cancel_prolong — cancel auto-renewal;
• uploaddocs, uploadext — upload documents;
• contactverify, domainverify, checkdomaindoc — document verification.

Specific commands for SSL certificates:

• approver — get email addresses to confirm domain ownership;
• usercreate — create an account with the certificate authority;
• reopen — reissue a certificate.

/usr/local/mgr5/processing/pmvpn --command open --item 42

XML description

The file name must look like billmgr_mod_pmXXX.xml, where XXX is the module name. The file must be copied to the /etc/xml directory relative to the BILLmanager installation path. The file contains the description of the module itself, as well as the description of additional forms and messages.

Explanations for XML file commands for service processing modules

features element — output

Returns an XML document containing the description of the processing module parameters and supported functions.

<doc>
  <itemtypes> <!-- List of supported product types. -->
    <itemtype name="itemtype1"/>
    ...
    <itemtype name="itemtypeN"/>
  </itemtypes>
  <params> <!-- List of processing module parameters. -->
    <param name="param1"/> <!-- Non-encrypted parameter. -->
    ...
    <param name="cryptedparam1" crypted="yes"/> <!-- Encrypted parameter. -->
    ...
    <customparam name="customparam1" unique="yes|no" defval=""/>
    ...
  </params>
  <features> <!-- List of supported functions. -->
    <feature name="feature1"/>
    ...
  </features>
</doc>

Attribute details:

• name for the itemtype element is specified by the internal name of the product or service type;
• name for the param element must match the input field name in the processing module settings form;
• crypted indicates that the parameter must be stored in the database in encrypted form;
• name for customparam is shown in the selection list when adding an additional parameter;
• unique with value yes indicates that only one parameter with the selected internal name can exist;
• defval defines the default value of the parameter (optional);
• name for the feature element indicates a supported processing module function.

get_suitable_module element — input and output

Input (stdin) — accepts an XML document with service parameters.

<?xml version='1.0' encoding='UTF-8'?>
<doc>
  <item> <!-- Service parameters. -->
    <pricelist>PR_ID</pricelist> <!-- Ordered tariff code. -->
    <param1></param1>
    ...
  </item>
  <skip_modules></skip_modules>  <!-- IDs of handlers to ignore. -->
</doc>

Output — returns an XML document of the following form:

<?xml version='1.0' encoding='UTF-8'?>
<doc>
  <modules>
    <module id="id1"/>
    ...
  </modules>
</doc>

check_connection element — input and output

Input (stdin) — accepts connection parameters (always unencrypted):

<?xml version='1.0' encoding='UTF-8'?>
<doc>
  <processingmodule> <!-- Handler parameters. -->
    <param1></param1>
    ...
  </processingmodule>
</doc>

Output — returns a BILLmanager XML error or <doc/>:

<?xml version='1.0' encoding='UTF-8'?>
<doc>
  <ok/>
</doc>

tune_connection, tuning_param, usercreate, tune_changepassword elements — input and output

Input — accepts the current XML form file (standard BILLmanager form description):

<doc>
  <form>
    <field name="api_url">
      <input type="text" name="api_url" required="yes"/>
    </field>
  </form>
</doc>

Output — returns the modified XML file. You can add fields, attributes, and hints.

<doc>
  <form>
    <field name="api_url">
      <input type="text" name="api_url" required="yes" placeholder="https://..."/>
    </field>
    <field name="extra_param">
      <input type="text" name="extra_param"/>
    </field>
  </form>
</doc>

import_pricelist element — output for different subcommand

For the --subcommand available command, returns:

<doc>
  <elem>
    <id>...</id> <!-- Tariff code on the platform or on the service provider side. -->
    <name>...</name> <!-- Tariff name. -->
    <name_ru>...</name_ru> <!-- Tariff name in ru localization. -->
    <itemtype>...</itemtype> <!-- Internal name of the tariff product type.-->
  </elem>
</doc>

For the --subcommand pricelist command, returns:

<doc>
  <elem>
    <id>...</id>
    <name>...</name>
    <itemtype>...</itemtype>
    <real_billtype>4</real_billtype> <!-- Fixed value. -->
    <internalname>...</internalname> <!-- Name of the tariff plan configuration. -->
    <billdaily>...</billdaily> <!-- on value for daily deduction. -->
    <minperiodtype>...</minperiodtype> <!-- Minimum order period type. -->
  </elem>
</doc>

Other subcommands: images, periods, details — build similar XML structures with tags name, intname, currency_code, period, etc.

check_param and check_addon elements — input

Input — accepts an XML file with old and new parameter values:

<doc>
  <item> <!-- List of old service parameter values. -->
    <param1>...</param1>
  </item>
  <newitem> <!-- List of new service parameter values. -->
    <param1>...</param1>
  </newitem>
</doc>

Output — returns <doc/> or an error.

transition_controlpanel element — output

Returns a list of panels:

<doc>
  <url></url> <!-- Transition link. -->
  <panelcount>2</panelcount>
  <panelname keyname="vds_panel">VDS Panel</panelname>
  <panelname keyname="vm_manager">VMmanager</panelname>
</doc>

pingip element — output

<doc>
  <ping_result>OK</ping_result> <!-- check result: OK if the IP address is reachable. -->
</doc>

Task description

The provider sells access to local AI models through BILLmanager. When ordering the service, the client gets an isolated Docker container with Ollama on a dedicated provider server. Each container is available through a unique URL with an authorization token.

The handler supports the following modes:

• CPU — a container from an image without CUDA, computations run on the CPU. No GPU is required;
• GPU — a container from a CUDA image with GPU mounted into the container and the share of the compute unit limited through NVIDIA MPS.

The mode is selected in the handler settings.

Architecture

BILLmanager
└─ pmollama handler (CPU mode or GPU mode)
   └─ SSH to the server (one server = one handler)
      └─ docker run hwdsl2/ollama-server[:cuda]   (one container per service)
         ├─ Caddy inside the container: auth proxy on OLLAMA_PORT (11434)
         │      └─ reverse_proxy → ollama serve on 127.0.0.1:41434
         ├─ --cpus / --memory / --storage-opt   (limits from add-ons)
         └─ GPU mode only:
            --gpus device=N (round-robin across GPUs)
            --ipc=host + /tmp/nvidia-mps (MPS client)
            CUDA_MPS_ACTIVE_THREAD_PERCENTAGE (SM share, gpu_pct add-on)
            CUDA_MPS_PINNED_DEVICE_MEM_LIMIT (VRAM limit, vram_gb add-on)

The client request passes through several layers before reaching the Ollama model:

Level 1. BILLmanager

BILLmanager is the billing core. It stores services, tariffs, and users, and calls the pmollama handler on payment, suspension, tariff change, etc. BILLmanager manages Docker and communicates with the server through the handler.

Level 2. pmollama handler

The handler is a script that:

• gets service parameters (add-ons, mode, settings);
• connects to the provider server over SSH and runs the required Docker commands.

Level 3. Provider server

A physical or virtual server where client containers run. Server requirements:

• only one handler with one set of SSH parameters must be bound to one server;
• Docker must be installed on the server, and in GPU mode — additionally NVIDIA Container Toolkit with the MPS daemon running;
• each container gets its own port from a range defined in the handler settings (the base_port parameter).

Level 4. Service Docker container

Each service uses a separate container from the hwdsl2/ollama-server image (CPU) or hwdsl2/ollama-server:cuda image (GPU). The container is isolated: it has its own resource limits, its own volume with models, and its own API key.

Limits are passed from tariff add-ons using command parameters docker run:

• --cpus — number of cores (add-on cpu_count);
• --memory — RAM size (add-on ram_gb);
• --storage-opt size= — disk quota (add-on storage_gb).

Level 5. Inside the container: Caddy and Ollama

Two processes run inside the container:

• Ollama (ollama serve) listens only on 127.0.0.1:41434 — the port is not exposed outside the container;
• Caddy listens on port 11434 and proxies requests to the internal Ollama port, requiring the header Authorization: Bearer <token>.

With this architecture, the client gets a unique URL with a token, and Ollama is protected from direct external access.

GPU mode specifics

In GPU mode, special flags and environment variables are added to the container:

• --gpus device=N — passthrough of a specific graphics card;
• --ipc=host and mounting /tmp/nvidia-mps — connecting the container to the NVIDIA MPS daemon running on the host. The CUDA_MPS_ACTIVE_THREAD_PERCENTAGE variable is the share of streaming multiprocessors (SM) that the container can use. It is set by the gpu_pct add-on. The CUDA_MPS_PINNED_DEVICE_MEM_LIMIT="0=<N>G" variable is the video memory limit in gigabytes. It is set by the vram_gb add-on.

How it all fits together: request path

Client request path to the Ollama API:

1. The client sends an HTTP request to the public service URL (for example, https://ai.example.com:11435/api/tags ) with the header Authorization: Bearer <token>.
2. The provider’s external proxy terminates HTTPS and forwards the request to the container port on the host.
3. Docker forwards the request to port 11434 inside the container, where Caddy receives it.
4. Caddy checks the token. If the token is valid, it proxies the request to 127.0.0.1:41434, where Ollama is listening.
5. Ollama processes the request and returns the response back to the client along the same chain.

Implementation result

• localai product type with add-ons:
  • CPU;
  • RAM;
  • disk;
  • GPU share;
  • VRAM limit;
  • fine-tuning of Ollama;
• pmollama handler in Python with CPU/GPU mode switch;
• full service lifecycle over SSH:
  • creation;
  • pause;
  • resume;
  • tariff change;
  • renewal;
  • deletion;
• unique token and URL for each service;
• resource usage statistics collection (CPU, RAM, and disk) through the docker stats command.

Service lifecycle and handler commands

Handler commands are called in response to actions by users and administrators.

SDK: calling commands and handling arguments

This section explains how BILLmanager launches handler commands, passes arguments, and processes results. Understanding this mechanism helps avoid common development mistakes.

Entry point and run()

BILLmanager starts the handler as a regular executable file and passes the command with the --command option:

/usr/local/mgr5/processing/pmollama --command open --item 1234 --runningoperation 5678

Inside pmollama.py an module object is created and .run() is called. The run() method is in the base class billmgr.modules.base.Module and performs the following steps:

def run(self) -> None:
    args = self._parse_args()            # 1. Parse argparse.
    if args.test:                        # 2. --test/-T → "(c) ISPsystem.com" and exit
        sys.stdout.write("(c) ISPsystem.com")
        sys.exit(0)
    self._before_run_command(args)       # 3. Preprocessing (features + implementation check).
    try:
        self._run_command(args)          # 4. Call the command.
    except Exception as err:
        exc.log_backtrace()
        xml_err = err if isinstance(err, exc.XmlException) \
            else exc.XmlException(err_type=args.command, err_value=str(err))
        self._on_raise_exception(args, xml_err)   # 5. Handle the error.
        sys.exit(1)

A significant part of errors occurs while one of the steps in the code above is running.

Mechanism for calling the command function

Commands are registered in the module constructor and stored in the self._commands: Dict[str, Callable] dictionary, where the key is the command name (open, close, etc.). The command execution step (_run_command) looks like this:

def _run_command(self, args):
    cmd = self._commands.get(args.command)
    if not cmd:
        raise exc.XmlException("unknown command", "", args.command)
    # From all argparse arguments, take only those whose names match the command function parameter names:
    params = {k: v for k, v in vars(args).items() if k in signature(cmd).parameters}
    cmd(**params)

The SDK passes only those arguments to the function that were declared in the signature. The function parameter names must match the dest values of the corresponding argparse arguments:

def open(item: int) -> None:          # Function receives --item
def set_param(item, runningoperation):# Function receives --item and --runningoperation
def stat(module: int) -> None:        # Function receives --module / -m
def check_connection() -> None:       # Function declares no arguments → receives nothing

If a function parameter is named different from the dest name of the argument, the function will fail. For example, when declaring def open(item_id):

• the SDK will not find the item_id parameter among the argparse arguments (there is item there) and will call open() without arguments;
• the call will end with a TypeError: open() missing 1 required positional argument error.

List of arguments passed by BILLmanager

The constructor of the module base class ProcessingModule.__init__ registers all possible arguments.

The parameter name must match the value from the dest: item, module, runningoperation column. A common mistake is to use a parameter name for the function that differs from the dest of the argument. For example, if you specify def open(item_id), the SDK will look for item_id. Since item is registered in argparse, the function will be called without arguments. The call will end with a TypeError: open() missing 1 required positional argument.

Processing the service command features and validating required methods

At the preprocessing step ( _before_run_command in the module object call example) the module performs the following functions:

• processing the service command features. The --command features command is a service request that the BILLmanager platform sends to the handler to get the list of supported capabilities. The SDK builds the XML description of these capabilities based on the registered commands and outputs it to stdout.

def _before_run_command(self, args):
    if args.command == "features":
        self.features()      # Builds the XML capabilities and prints to stdout.
        sys.exit(0)
    for feat, cmd in self._commands.items():
        if cmd is None:      # Determines the action order if the command is registered but the handler is not set.
            raise exc.XmlException("not_implemented", err_object="feature", err_value=feat)

• validation of required commands implementation. During initialization (ProcessingModule.init), the module sets None for the required service lifecycle commands:

  • open;
  • close;
  • resume;
  • suspend;
  • setparam.

# Inside ProcessingModule.__init__:
self._add_callable_feature(Feature.OPEN)      # _commands["open"]    = None
self._add_callable_feature(Feature.CLOSE)     # _commands["close"]   = None
self._add_callable_feature(Feature.RESUME)    # _commands["resume"]  = None
self._add_callable_feature(Feature.SUSPEND)   # _commands["suspend"] = None
self._add_callable_feature(Feature.SET_PARAM) # _commands["setparam"]= None

Before executing any command, the _before_run_command method walks through the dictionary of registered commands. If at least one of them still has the value None, execution is stopped with a not_implemented exception. This means the developer must override all five required commands, even if the module does not need some of them. For example, suspend or resume. 

Ways to register commands and features

The SDK provides three methods for registering module commands and features. The chosen method determines how the module will appear in the XML description (features-XML) and how the platform will interact with it.

When processing the --command features command, the SDK generates an XML description that includes:

• product types from itemtypes;
• the self._features set (registered features);
• parameters from get_module_param() (output as <param>) and get_module_custom_param() (output as <customparam>).

Finishing an operation: post*-functions

An asynchronous command does not change the service status automatically. To make BILLmanager switch the service to the required status and mark the current operation as completed, call the appropriate post* function from the billmgr.misc library after the command completes successfully:

If misc.postopen(item) is not called after the open function, the container is created and works, but the service remains in the processing status. BILLmanager considers the operation incomplete and calls the open. This leads to the error Conflict: The container name "/ollama_1234" is already in use when trying to start a container with the same name. To avoid this error, add misc.postopen(item) at the end of the open function.

Errors, auto-retry, and idempotency

If a command raises an exception, error handling is triggered (_on_raise_exception):

def _on_raise_exception(self, args, err):
    if not args.runningoperation:
        return                       # Synchronous commands are not called again.
    misc.save_runningoperation_error(args.runningoperation, err.as_module_error())
    MAX_TRY = 10
    res = db.get_first_record(
        "SELECT trycount FROM runningoperation WHERE id = %s", args.runningoperation)
    if res and res.as_int("trycount") >= MAX_TRY:
        misc.create_manual_task(args.item, args.runningoperation, args.command)

Command behavior on error differs:

• an asynchronous operation is retried automatically — up to 10 times, after which a manual task is created. Commands must be idempotent: rerunning them must not fail because part of the work is already done;
• synchronous commands (without runningoperation) are not retried. BILLmanager shows their error immediately in the form — so it is important to return a meaningful XmlException.

To ensure idempotency in close, suspend, and resume, all shell commands are wrapped in || true, and volume deletion uses raise_on_error=False:

ssh.run(f"docker stop {name} || true && docker rm {name} || true")

Returning errors to the interface: XmlException and msgerror

To show the user a clear message instead of low-level Python stack traces, raise an exception of type billmgr.exception.XmlException with a meaningful error type identifier in the err_type parameter:

raise XmlException("open_error") from e

The err_type value (in the example above open_error) is used to look up the text in the XML section <messages name="msgerror"> that matches the msg_error_<err_type> template:

<msg name="msg_error_open_error">Failed to start the Ollama container.</msg>

To duplicate the error to the standard output stream (stdout), override the _on_raise_exception method at the module entry point. The system will display it in synchronous forms:

def _on_raise_exception(self, args, err):
    super()._on_raise_exception(args, err)   # Write to runningoperation and retry.
    sys.stdout.write(err.as_xml())           # Output the error to the form.

Product type

A product type (itemtype) defines the service category in BILLmanager. It sets the add-ons, parameters, and system behavior when ordering, renewing, and changing the service. Without a product type, you cannot create a tariff and link a handler to it. See the article Product types for details.

Standard types (vds, vhost, domain) are not suitable for the pmollama handler, because they have their own lifecycle logic and do not contain the required add-ons. Create your own localai type.

Creating a product type

To create your own product type:

1. Open the platform web interface with an administrator account.
2. In the main menu, go to Products → Product types → click Add.
3. Fill in the fields:
   • Internal name: localai;
   • Name: LocalAI (Ollama).

The internal name localai is the identifier. Pass it to:

• the code (super().__init__(itemtypes={"localai"}));
• the XML configuration (<type name="localai"/>).

The value must match the one specified in the interface.

Linking the product type with the processing module

To link the product type with the processing module in two places at once:

1. In the module XML file, specify the <type name="localai"/> element inside the <plugin> section. This tells BILLmanager that this handler supports the localai type.

   <plugin name="pmollama">
     <group>processing_module</group>
     <params>
       <type name="localai"/>
     </params>
   </plugin>

   SDK inserts the type in response to the features command, and BILLmanager matches the data.
2. In the platform web interface:
   1. Go to Integration → Processing modules → click Add.
   2. Select the pmollama module. The pmollama module processes tariffs of the localai type.
   3. Specify the server connection parameters and operating mode.
   4. Select this handler in the tariff settings.

Add-ons and tariff parameters

In BILLmanager, the entities configured for a service are divided into several categories with different reading rules.

Tariff add-ons (measurable resources)

An add-on is a measurable numeric resource included in the tariff (CPU, RAM, disk). When the tariff changes, the add-ons change too. When creating a tariff for the handler, add-ons are the Docker container limits and GPU parameters.

To create an add-on, go to Products → Product types → select localai → Details tab. The internal name of the add-on in the interface must exactly match the name used by the processing module.

Reading add-ons in the open command

Use the misc.itemaddons(item) function from the SDK module. It performs an SQL query and returns a dictionary with add-ons.

addons = misc.itemaddons(item)
# {"cpu_count": Addon(value="4", measure="pcs", bill_type=...),
#  "ram_gb":    Addon(value="16", measure="gb", ...), ...}

When working with the misc.itemaddons(item) function, keep in mind the following:

• the add-on instance (Addon) is a named tuple NamedTuple(value, measure, bill_type). The value is available by index [0] or through the .value attribute, and the unit of measure is available by index [1] or through the .measure attribute;
• the value is stored as a string, so convert it to int or float manually;
• the dictionary contains only specified add-ons, so check whether the key exists before accessing it to prevent a KeyError.

Use the following pattern to safely read add-on values with default values:

def get_addons(item: int) -> dict:
    addons = misc.itemaddons(item)
    def i(key, default):
        return int(addons[key][0]) if key in addons else default
    return {
        "cpu_count":  i(CPU_COUNT,  CPU_COUNT_DEFAULT),
        "ram_gb":     i(RAM_GB,     RAM_GB_DEFAULT),
        "storage_gb": i(STORAGE_GB, STORAGE_GB_DEFAULT),
        "gpu_pct":    i(GPU_PCT,    GPU_PCT_DEFAULT),
    }

Use a function to read optional add-on values. The function returns None if the value is missing.

def get_ollama_tuning(item: int) -> dict:
    addons = misc.itemaddons(item)
    def opt(key):
        if key not in addons:
            return None
        try:
            v = int(addons[key][0])
        except (TypeError, ValueError):
            return None
        return v if v > 0 else None
    return {
        "OLLAMA_MAX_LOADED_MODELS": opt(OLLAMA_MAX_LOADED_MODELS),
        "OLLAMA_NUM_PARALLEL":      opt(OLLAMA_NUM_PARALLEL),
        "OLLAMA_CONTEXT_LENGTH":    opt(OLLAMA_CONTEXT_LENGTH),
    }

Tariff parameter ollama_models (string)

The comma-separated model list is a string and cannot be a numeric add-on. Set this tariff parameter through the dynamic metadata pricelist.pmollama.localai (requires enabling the PRICELIST_DYNAMIC_SETTINGS option). Use a separate function to read the value:

def get_default_models(item: int) -> str:
    params = misc.get_pricelist_params(misc.iteminfo(item)["pricelist"])
    return (params.get(OLLAMA_MODELS_PARAM) or "").strip()

Service parameters

A service parameter is a value that the module stores for a specific service when executing commands (open, setparam):

• container_name;
• container_url;
• container_token (stored encrypted).

Use the misc.save_param function to write values, and misc.itemparams to read them. Encrypted parameters are decrypted automatically:

misc.save_param(item, param="container_name",  value=name)
misc.save_param(item, param="container_url",   value=container_url)
misc.save_param(item, param="container_token", value=token, crypted=True)

params = misc.itemparams(item)
token  = params.get("container_token")

To configure service parameters, go to Products → Product types → select localai → Parameters tab. See Product type parameters for details.

Handler parameters (connection and mode)

These parameters are set when creating the processing module. The system stores them at the module level. Use the misc.get_module_params(module_id) function to read them. The system automatically decrypts encrypted fields. For example, a password:

module_id = misc.get_item_processingmodule(item)
params    = misc.get_module_params(module_id)
ssh_host     = params["ssh_host"]
ssh_user     = params["ssh_user"]
ssh_password = params["ssh_password"]
base_port    = int(params.get("base_port", "11434"))
public_url   = params["public_url"]
gpu_mode     = params.get("gpu_mode")

Reference information: parameter types and functions for reading them

hwdsl2/docker-ollama image and CPU/GPU modes

The module uses the ready-made hwdsl2/docker-ollama image instead of the standard ollama/ollama image. The structure of this image defines the environment variables and volumes that must be specified in the docker run command.

Image variants

The module automatically selects the image depending on the handler mode using the consts.image_for_mode() function.

Using Caddy as an authentication proxy and working with two ports

Inside the container two processes run:

1. The ollama serve process accepts connections only at 127.0.0.1:41434. The port is not exposed externally.
2. The caddy process accepts connections on the OLLAMA_PORT port (default 11434) and forwards requests to the internal port. All routes except the root route require the Authorization: Bearer <api_key> header.

External access is provided to the Caddy port. In the docker run command, you need to specify the parameter -p {host_port}:11434. If the token is invalid, Caddy rejects the request with 401 Unauthorized.

API key

The platform reads the authorization key from the OLLAMA_API_KEY environment variable and saves it to the /var/lib/ollama/.api_key file. If the variable is not set, the image generates a random key itself and saves it to the same file.

The processing module sets the value of the OLLAMA_API_KEY variable using the misc.random_string(32) function. This lets the module get the key right away and removes the need to read it from the container after startup.

Data volume

Models and the authorization key are stored in the /var/lib/ollama directory. Mount a named volume to this directory with the -v ollama_{item}:/var/lib/ollama parameter. Data in the volume is preserved when the container is recreated. The module uses this feature when running the setparam command to recreate the container.

Environment variables used by the image

The image startup script passes only the following environment variables to the Ollama process, if they are set:

Variables that are not supported by the startup script and will not be passed to the process:

• OLLAMA_GPU_OVERHEAD — the startup script does not pass this variable to the ollama serve process;
• GPU limits at the cgroups level — Docker does not support them without MIG or MPS technologies.

Mode switch and dependencies

The operating mode is defined by the handler’s select parameter gpu_mode with values cpu or gpu. The choice options are built dynamically by the tune_connection command. The processing module normalizes the parameter value and performs the following actions:

• in both modes, it selects the appropriate Docker image (image_for_mode):

  • for cpu — hwdsl2/ollama-server;
  • for gpu — hwdsl2/ollama-server:cuda;
• in gpu mode, it additionally builds the container launch parameters taking into account GPU resource limits (gpu.build_run_flags).

Sharing GPU resources between containers

In gpu mode, graphics processing unit resources are shared between containers in the following ways:

1. Cyclic GPU allocation (round-robin across GPUs). The --gpus device=N parameter is used, where the value of N is calculated as the remainder of dividing the service ID by the total number of graphics processors:

   N = item % number_of_GPUs.

   Where item is the integer service ID in BILLmanager.
2. Share of streaming multiprocessors (SM) through NVIDIA MPS technology. It is set using the CUDA_MPS_ACTIVE_THREAD_PERCENTAGE environment variable, whose value is taken from the gpu_pct add-on.
3. Limiting video memory through MPS. It is set using the CUDA_MPS_PINNED_DEVICE_MEM_LIMIT environment variable, whose value is formed based on the vram_gb add-on:

   CUDA_MPS_PINNED_DEVICE_MEM_LIMIT="0=<N>G" (add-on vram_gb).

MPS technology is a background process on the host (nvidia-cuda-mps-control -d), which containers connect to as clients. The container image does not install MPS and does not mount sockets. Therefore, in gpu mode, if the background MPS process is running on the host, the module adds the following parameters when starting the container:

 --ipc=host, -v /tmp/nvidia-mps:/tmp/nvidia-mps and -e CUDA_MPS_PIPE_DIRECTORY=/tmp/nvidia-mps

Where:

• --ipc=host — the container uses the host IPC namespace. This allows processes inside the container to interact with the MPS background process through shared memory and named channels;
• -v /tmp/nvidia-mps:/tmp/nvidia-mps — mounts the directory with MPS sockets from the host into the container at the same path. Provides access to the control sockets of the MPS background process;
• -e CUDA_MPS_PIPE_DIRECTORY=/tmp/nvidia-mps — sets an environment variable that points CUDA applications to the directory with MPS communication channels.

If the background process is not running, the module starts the container with a GPU without limits and writes the corresponding warning to the event log.

XML metadata of the module

The file xml/billmgr_mod_pmollama.xml describes how the handler appears in the platform:

• settings form;
• tariff form;
• localization;
• error messages.

The file name must match the pattern billmgr_mod_<binaryname>.xml, where <binaryname> is the name of the handler executable (pmollama).

<mgrdata>
  <!-- Registering the processing module and displaying it in the platform. -->
  <plugin name="pmollama">
    <group>processing_module</group>
    <author>Your Name</author>
    <params>
      <type name="localai"/>
      <priority lang="ru">100</priority>
      <priority lang="en">100</priority>
    </params>
  </plugin>
  
  <!-- Processing module settings form. -->
  <metadata name="processing.edit.pmollama" type="form">
    <form>
      <page name="methodprops">
        <field name="ssh_host">
          <input type="text" name="ssh_host" private="yes" required="yes"/>
        </field>
        <!-- ... rest of the fields ... -->
        <field name="gpu_mode">
          <select name="gpu_mode" />
        </field>
      </page>
    </form>
  </metadata>

  <!-- Localization and error messages. -->
  <lang name="ru">
    <messages name="msgerror">
      <msg name="msg_error_open_error">Failed to start the Ollama container.</msg>
      <!-- ... -->
    </messages>
  </lang>
</mgrdata>

Purpose of XML elements

The <plugin> element registers the processing module. The nested group element with the value processing_module assigns it to service processing modules. Specifying the <type name="localai"/> element is required.

The <metadata> element with the name processing.edit.pmollama defines the fields of the processing module settings form. The values of these fields are read by the misc.get_module_params() function. The private attribute with the value yes hides the entered value in the interface after saving.

The <metadata> element with the name pricelist.pmollama.localai defines the fields of the tariff plan form. The name of this element is formed using the pattern pricelist.<binaryname>.<itemtype>.

The <messages> section with the name msgerror contains error message texts. The message tag name must include the msg_error_ prefix and exactly match the value of the err_type parameter passed when generating the XmlException("open_error") exception.

Matching names in the configuration and in the code

Creating a module with Python SDK

Project structure

pmollama/
├── install.sh                # Installation script.
├── Makefile                  # File placement in /usr/local/mgr5 directories
├── requirements.txt          # Module dependencies for installation in a virtual environment.
├── pmollama.py               # Entry point, contains the module class and command registration.
├── ollama/                   # Package with logic, the platform places it in lib/python/ollama.
│   ├── __init__.py
│   ├── consts.py             # Constants.
│   ├── gpu.py                # Building Docker flags for GPU and MPS.
│   ├── misc.py               # Read functions for add-ons and parameters.
│   ├── ssh.py                # Wrapper for the paramiko library.
│   └── commands/             # Directory with files, each file implements one command.
└── xml/
    └── billmgr_mod_pmollama.xml

Splitting it into the ollama/ package and the pmollama.py entry point file is necessary for correct installation. The platform places the entry point in the processing/ directory and the package in the lib/python/ directory.

Entry point: pmollama.py

#!/usr/bin/env python3
import sys
import os
from enum import Enum

sys.path.insert(0, "/usr/local/mgr5/lib/python")
from billmgr.modules.processing import ProcessingModule, Feature
from billmgr.exception import XmlException
import billmgr.logger as logging
import ollama.commands as cmd

logging.init_logging("pmollama")
LOGGER = logging.get_logger("pmollama")

class ExFeature(Enum):
    DATACENTER = 'datacenter'

class OllamaModule(ProcessingModule):
    def __init__(self) -> None:
        super().__init__(itemtypes={"localai"})
        self.set_description("Ollama Docker on a dedicated GPU server")
        
        self._add_callable_feature(Feature.OPEN, cmd.import_func("open"))
        self._add_callable_feature(Feature.CLOSE, cmd.import_func("close"))
        self._add_callable_feature(Feature.SUSPEND, cmd.import_func("suspend"))
        self._add_callable_feature(Feature.RESUME, cmd.import_func("resume"))
        self._add_callable_feature(Feature.SET_PARAM, cmd.import_func("set_param"))
        self._add_callable_feature(Feature.STAT, cmd.import_func("stat"))
        self._add_callable_feature(Feature.CHECK_CONNECTION, cmd.import_func("check_connection"))
        self._add_callable_feature(Feature.CONNECTION_FORM_TUNE, cmd.import_func("tune_connection"))
        
        self._add_feature(ExFeature.DATACENTER)
        self._add_feature(Feature.PRICELIST_DYNAMIC_SETTINGS)

    def get_module_param(self):
        return {
            "ssh_host":     {},
            "ssh_user":     {},
            "ssh_password": {"crypted": "yes"},
            "base_port":    {},
            "public_url":   {},
            "gpu_mode":     {},
        }

    def _on_raise_exception(self, args, err: XmlException) -> None:
        super()._on_raise_exception(args, err)
        sys.stdout.write(err.as_xml())

if __name__ == "__main__":
    # Logic for launching the virtual environment and initializing the module.
    OllamaModule().run()

Key points of the entry point:

• specify the product type using super().__init__(itemtypes={"localai"}) ;
• register required commands through _add_callable_feature(Feature.X, fn) ;
• define parameters in get_module_param() , where the keys match the form field name attributes, and the ssh_password parameter contains {"crypted": "yes"} ;
• override _on_raise_exception to print the error text to stdout;
• make sure the module restarts in a virtual environment where os.execv replaces the current process with the interpreter from the virtual environment.

Detailed description of commands

The general form of a command is: the function receives the arguments declared by the SDK, performs actions over SSH, saves service parameters if needed, and обязательно finishes by calling a function from the post* family.

Below are examples of command implementations:

• open;
• close;
• suspend;
• resume;
• setparam;
• stat;
• check_connection;
• tune_connection.

The code follows the idempotency rules, use of misc.post*, and correct add-on reading described in the previous sections.

Building and installing the module

Installation consists of the following stages:

1. Install dependencies into an isolated virtual environment (venv).
2. Place files into directories under /usr/local/mgr5 using Makefile.
3. Clear the BILLmanager cache.

requirements.txt

paramiko==3.5.0

The file contains only the dependencies needed by the module in addition to the SDK. These packages are installed into the virtual environment, not into the system Python interpreter.

Makefile

MGR    = billmgr
PLUGIN = pmollama
BASE  ?= /usr/local/mgr5
SRC    = $(shell pwd)
VENV_PATH = $(SRC)/venv-pmollama

.PHONY: dist-prepare copy-lib

dist-prepare: $(DISTDIR)/processing/pmollama copy-lib

$(DISTDIR)/processing/pmollama: $(SRC)/pmollama.py
	@mkdir -p $(DISTDIR)/processing/
	cp -f $(SRC)/pmollama.py $(DISTDIR)/processing/pmollama
	sed -i '1s|^#!.*|#!$(VENV_PATH)/bin/python3|' $(DISTDIR)/processing/pmollama
	chmod 755 $(DISTDIR)/processing/pmollama

copy-lib:
	@rm -rf $(DISTDIR)/lib/python/ollama
	@mkdir -p $(DISTDIR)/lib/python/ollama
	cp -rf $(SRC)/ollama/. $(DISTDIR)/lib/python/ollama/
	@find $(DISTDIR)/lib/python/ollama -name __pycache__ -type d -prune -exec rm -rf {} +

include $(BASE)/src/isp.mk

install.sh

#!/usr/bin/env bash
. /usr/local/mgr5/lib/pkgsh/core_pkg_funcs.sh

InstallDeps() {
    Info "Installing dependencies..."
    PKGS="billmanager-plugin-python-libs python3-pip make"
    python3 -m venv venv-pmollama
    "./venv-pmollama/bin/pip3" install -r requirements.txt
}

InstallModule() {
    Info "Installing pmollama..."
    DESTDIR=/usr/local/mgr5/src/pmollama
    mkdir -p "${DESTDIR}" && cp -rfa ./* "${DESTDIR}"/
    cd "${DESTDIR}" && make install
}

InstallDeps
InstallModule

Checking the installation

# 1. Clear the XML cache and restart BILLmanager.
/usr/local/mgr5/sbin/mgrctl -m billmgr exit

# 2. Check the features command manually.
/usr/local/mgr5/processing/pmollama --command features

Server preparation and testing

Ports

Open the port range for container operation:

ufw allow 11434:12000/tcp

CPU mode

To work in CPU mode, install the Docker platform:

curl -fsSL https://get.docker.com | sh

GPU mode

To work in GPU mode, install Docker, the NVIDIA driver, NVIDIA Container Toolkit, and the MPS service.

# Install NVIDIA Container Toolkit.
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
apt-get update && apt-get install -y nvidia-container-toolkit
nvidia-ctk runtime configure --runtime=docker && systemctl restart docker

# Configure MPS (for gpu_pct and vram_gb parameters).
nvidia-smi -c EXCLUSIVE_PROCESS
ulimit -n 16384
export CUDA_MPS_PIPE_DIRECTORY=/tmp/nvidia-mps
export CUDA_MPS_LOG_DIRECTORY=/var/log/nvidia-mps
nvidia-cuda-mps-control -d

Interface (UI)

Module icon

The icon is shown in the processing module list and in the marketplace. The source file is placed in the dist/skins/common/plugin-logo/billmanager-plugin-pmollama.png directory

The isp.mk build script copies the file to the target directory: /usr/local/mgr5/skins/common/plugin-logo/billmanager-plugin-pmollama.png. File requirements:

• PNG format with a transparent background;
• the file name exactly matches the binaryname value (billmanager-plugin-pmollama.png). For example, for the pmollama module, the icon name will be billmanager-plugin-pmollama.png.

Icons for buttons in XML forms

BILLmanager uses the built-in SVG sprite. The icon name for a toolbar button is specified in the @img attribute.

<metadata name="localai" type="list">
  <toolbar>
    <toolbtn func="localai.open"    name="open"    img="t-on"     type="new"/>
    <toolbtn func="localai.suspend" name="suspend" img="t-lock"   type="group"/>
  </toolbar>
</metadata>

Diagnostics and errors

If the system does not work correctly, check the following points. Most problems fall into one of the categories described below.

The processing module is unavailable or the tariff plan cannot be ordered

• make sure the plugin name in the XML file, the executable file name in the processing directory, the binaryname value, and the metadata prefixes all match exactly. For example, pmollama;
• check that the type element in the XML file and the itemtypes parameter in the code match the internal name of the product type. For example, localai;
• for a custom product type, the datacenter option must be declared. Without it, the tariff plan will not appear in the list of available services;

• make sure the cache clear command has been run after making changes to XML files:

  mgrctl -m billmgr exit

• check that a manual run of the module with the --command features parameter returns the correct values for product types, features, and parameters (itemtype, features, and params).

The command ends with a not_implemented error

• make sure all five required commands are registered: open, close, suspend, resume, setparam.

The command ends with a TypeError: ... missing argument error

Make sure that:

• the function parameter name matches dest: item, module, runningoperation, ip_id (not ip);
• the file or function name matches the value passed to import_func(...).

The service is created but remains in an intermediate state or repeats

Make sure that:

• the open command ends with a call to misc.postopen(item);
• the commands are idempotent:
  • the docker stop and rm commands are executed with the || true operator;
  • volume deletion is performed with raise_on_error=False.

Parameters are not read or saved

Make sure that:

• all handler form fields are listed in get_module_param();
• add-ons are read with key existence checking int(addons[key][0]) if key in addons else default;
• in the check_connection and tune_connection methods, data is taken from the input XML file, not from the get_module_params function.