Creating an adapter for the AI assistant

The following terms are used in this article:

The article describes the operating principles, step-by-step instructions, and the API of the module for creating your own AI assistant adapter. For information on how to configure working with the AI assistant in the platform interface, see the article Working with the AI assistant.

What an adapter is

An adapter is a module that connects the chat with an external LLM provider (OpenAI-compatible API, Yandex GPT, your own service, and so on). The chat server does not contact the provider directly: it sends the request to a registered adapter, and the adapter returns the response in a single internal format.

The capabilities of the adapter mechanism are:

connecting any LLM without changing the server code;
using several adapters in one chat instance. For example, you can use different adapters for different providers;
supporting streaming (SSE) and standard response generation;
model management: the adapter can optionally pass a list of supported models for selection in BILLmanager settings;
implementing custom LLM logic. For example, function calling, request filtering, and so on;
passing the API key, URL, and system prompt from the project settings.

Custom adapters are loaded automatically when the chat server starts from the /app/server/adapters/ directory inside the Docker container. Rebuilding the image is not required — just place the adapter file in the container and restart the container.

Built-in adapters

After installing the "AI assistant" module, the following adapters are available in the platform:

Name	Description
`openai`	An adapter for communicating with OpenAI-like LLMs.
`deep-seek`	An adapter for communicating with DeepSeek.

Creating an adapter

Below is an instruction for creating an adapter for communicating with an LLM through the OpenAI package. The example uses the adapter name my-llm. Instead of my-llm, you can specify another name.
To create an adapter for communicating with an LLM through the OpenAI package:

Create the adapter file [adapter-name].adapter.js. For example, my-llm.adapter.js.

Create the adapter class and implement the required methods:

getName — get the adapter name;
getCompletion — get a response from the AI without SSE;
getCompletionStream — get a response from the AI with SSE.

A detailed description of the API for input and output parameters is available below;

Adapter creation example

// The OpenAI library is available for import. No additional package installation is required.
const OpenAI = require('openai');

class MyLLMAdapter {
  // Return the adapter name. This is the name the chat server uses.
  getName() {
    return 'my-llm';
  }

  // Receive a response from LLM without SSE.
  async getCompletion(params) {
    // Create a client for communicating with LLMs. Pass the URL and key parameters.
    const client = new OpenAI({
      baseURL: params.apiUrl,
      apiKey: params.apiKey,
    });
    // Process the system prompt if it is specified.
    const systemPromptMessage = params.systemPrompt
      ? {
          role: 'system',
          content: params.systemPrompt,
        }
      : undefined;
    // Request an answer from LLM.
    const response = await client.chat.completions.create(
      {
        model: 'deepseek-chat',
        messages: [systemPromptMessage, ...params.input].filter(Boolean),
        stream: false,
      },
      // The parameters also include abortSignal to abort requests.
      { signal: params.abortSignal },
    );
    // Receive a text response.
    const text = response.choices[0]?.message?.content ?? '';
    // Write logs using console.log.
    console.log('My LLM response: ', text); 
    //  Return a response in the format expected by the chat server.
    return {
      outputText: text,
      output: [
        {
          status: 'completed',
          role: 'assistant',
          content: [{ type: 'text', text }],
        },
      ],
    };
  }

  async *getCompletionStream(params) {
    // Create a client for communicating with LLMs. Pass the URL and key parameters.
    const client = new OpenAI({
      baseURL: params.apiUrl,
      apiKey: params.apiKey,
    });
    // Process the system prompt if it is specified.
    const systemPromptMessage = params.systemPrompt
      ? {
          role: 'system',
          content: params.systemPrompt,
        }
      : undefined;
    // Create a request to the AI with the flow option.
    const stream = await client.chat.completions.create(
      {
        model: 'deepseek-chat',
        messages: [systemPromptMessage, ...params.input].filter(Boolean),
        stream: true,
      },
      { signal: params.abortSignal },
    );

    let accumulatedText = '';

    // Processing partial responses.
    for await (const chunk of stream) {
      const delta = chunk.choices[0]?.delta?.content;
      if (delta) {
        accumulatedText += delta;
        // Return an object of type text_delta for each piece of text.
        yield { type: 'text_delta', delta };
      }
    }
    // Output the log via console.log.
    console.log('My LLM response: ', accumulatedText);
    // Return the completed object to complete the request..
    yield {
      type: 'completed',
      response: {
        outputText: accumulatedText,
        output: [
          {
            role: 'assistant',
            content: [{ type: 'text', text: accumulatedText }],
            status: 'completed',
          },
        ],
      },
    };
  }
}

module.exports = { MyLLMAdapter };

Copy the adapter file to the /app/server/adapters directory of the running chat container. To keep the added adapters after rebuilding the image, create your own image based on the chat Docker image.
Example of creating an image with Dockerfile
```
FROM docker-registry.ispsystem.com/ispsystem5/chat:1.0.0

WORKDIR /app
COPY ./my-llm.adapter.js ./server/adapters/my-llm.adapter.js
```
Where:
- docker-registry.ispsystem.com/ispsystem5/chat:1.0.0 — address to the public image of the chat container.
Create an image from the Dockerfile:
```
 docker build . -t my-llm 
```
Specify the tag of the created image in the script for managing chat images /usr/local/mgr5/etc/docker/services/aichat.sh. Replace the SERVICE_IMAGE variable with the tag of the new image SERVICE_IMAGE="my-llm"

Restart the chat container:

sh etc/docker/services/aichat.sh restart

In the BILLmanager web interface, go to AI asistant → Settings → select the AI assistant → click Edit.
In the Main section, in the Adaptor field, select the new adapter.
Set the required settings: API URL, API key. For more details, see the article Working with the AI assistant.
Save changes.

After configuration, the chat is ready to use. Requests will be sent through the created adapter.

Writing an adapter with external dependencies

The chat service imports files ending with .adapter.js . If this file needs dependencies (for example, the node_modules directory) or packages, they must also be moved into the container into the /app/server/adapters/ directory.
When the node_modules directory is moved into the container, the dependencies become available through the require import.

Import example

// The OpenAI library is available for import by default.
const OpenAI = require('openai');
// An example of importing a dependency when moving node_modules with the necessary dependencies to the /app/server/adapters/ directory
const customPackage = require('custom-package');

class MyLLMAdapter {
  ...
}

module.exports = { MyLLMAdapter };

Adapter API

Function descriptions

getName()

Returns the adapter’s unique string name.

Accepts	None
Returns	`string` — unique adapter identifier. The chat service uses the `deep-seek` and `openai` adapters by default.
Required	Yes

getCompletion(params)

Synchronous response generation without a stream.

Accepts	`AdapterCompletionParams`
Returns	`Promise<ModelResponse>`
Required	Yes

getCompletionStream(params)

Streaming response generation for SSE mode.

Accepts	`AdapterCompletionParams`
Returns	`AsyncIterable<ModelStreamEvent>` (the function must be an asynchronous generator )
Required	Yes

getModels()

Returns a list of models available for selection in the settings.

Accepts	None
Returns	`string[]`
Required	No

Interface descriptions

ModelStreamEvent

Stream event (for SSE) is one of the following types:

text_delta — intermediate text fragment.

Field	Type	Required	Description
`type`	`text_delta`	Yes	Event type.
`delta`	`string`	Yes	Added text fragment.

completed — generation finished (required final event).

Field	Type	Required	Description
`type`	`completed`	Yes	Event type.
`response`	`ModelResponse`	Yes	Final answer.

tool_call — tool call notification.

Field	Type	Required	Description
`type`	`tool_call`	Yes	Event type.
`tool`	`string`	Yes	Tool identifier to display in the interface.

AdapterCompletionParams

Field	Type	Required	Description
`apiKey`	`string`	Yes	Provider API key from the settings.
`input`	`ModelMessage`	Yes	Message history for the LLM.
`abortSignal`	`AbortSignal`	Yes	Signal for canceling requests when the connection to the client is lost.
`apiUrl`	`string`	No	Base API URL from the settings.
`model`	`string`	No	Model name from the chat settings.
`systemPrompt`	`string`	No	System prompt from the settings.

ModelMessage

Field	Type	Required	Description
`role`	`user \| assistant \| system`	Yes	Message author role.
`content`	`string`	Yes	Message text.

ModelResponse

Field	Type	Required	Description
`outputText`	`string`	Yes	Full assistant response text.
`output`	`ModelResponseOutputMessage[]`	Yes	Structured output.
`relatedContent`	`ModelResponseRelatedContent`	No	Additional content related to the response.

ModelResponseOutputMessage

Field	Type	Required	Description
`role`	`assistant \| user`	Yes	Message author role.
`content`	`{type: 'text', text: string}[]`	Yes	Text fragments.
`status`	`string`	No	Generation status (for example, `completed` ).

ModelResponseRelatedContent

Field	Type	Required	Description
`links`	`{ title: string, url: string }[]`	No	Sources used by the LLM when generating the response. Display them as links below the message text message.

Example of an adapter with function calling

Adapter example

//The OpenAI library is available for import. No additional package installation is required.
const OpenAI = require('openai');

// Adapter with native tool support: LLM can call an external API,
// and return the result to the dialog and (optionally) attach links to relatedContent.
class CustomToolAdapter {
  // OpenAI client cache by apiKey. This is necessary to avoid creating a client for each request.
  clients = new Map();

  // Maximum number of rounds "model → tool → model".
  maxToolRounds = 10;

  // Get or create an OpenAI client for the specified key and URL
  getClient(apiKey, baseURL) {
    const client = this.clients.get(apiKey);
    if (client) {
      return client;
    }

    const newClient = new OpenAI({
      apiKey,
      baseURL,
    });
    this.clients.set(apiKey, newClient);
    return newClient;
  }

  // Converting the response to the format expected by the chat server.
  mapResponse(outputText, relatedContent) {
    return {
      outputText,
      relatedContent,
      output: [
        {
          role: 'assistant',
          content: [{ type: 'text', text: outputText }],
          status: 'completed',
        },
      ],
    };
  }

  // Call an external tool via HTTP. The URL and response format are customized for your service.
  async callTool(content) {
    const response = await fetch(<CUSTOM_TOOL_URL>, {
      method: 'POST',
      body: JSON.stringify({ question: content }),
    });
    const data = await response.json();
    return data;
  }

  // Extracting the question argument from the JSON that the model passed to the tool call
  getArgForTool(toolCall) {
    try {
      const parsedArgs = JSON.parse(toolCall.function.arguments);
      return parsedArgs.question ?? '';
    } catch {
      return '';
    }
  }

  //  leave only function tool calls, discarding other types.
  getFunctionToolCalls(toolCalls) {
    return toolCalls.filter((item) => item.type === 'function');
  }

  // Messages collection for chat.completions: system prompt + dialogue history.
  getChatMessages(params) {
    const inputMessages = params.input.map((item) => ({
      role: item.role,
      content: item.content,
    }));

    if (!params.systemPrompt) {
      return inputMessages;
    }

    return [
      {
        role: 'system',
        content: params.systemPrompt,
      },
      ...inputMessages,
    ];
  }

  //  Tool description for the model. Replace the placeholders with the actual name and description.
  getTools() {
    return [
      {
        type: 'function',
        function: {
          name: <TOOL NAME>,
          description: <CUSTOM DESC>,
          strict: true,
          parameters: {
            type: 'object',
            properties: {
              question: {
                type: 'string',
                description: <CUSTOM DESC>,
              },
            },
            required: ['question'],
            additionalProperties: false,
          },
        },
      },
    ];
  }

  //  Adapter name. This is what the chat server is configured to use in the project settings.
  getName() {
    return 'custom-tool';
  }

  //  List of models to select in settings (optional method)
  getModels() {
    return ['gpt-oss-20b/latest', 'gpt-oss-120b/latest'];
  }

  //Getting a response from LLM without SSE, with a tool calling loop
  async getCompletion(params) {
    const client = this.getClient(params.apiKey, params.apiUrl);
    let messages = this.getChatMessages(params);
    let lastAssistantText = '';

// Links to documents returned by the tool (will go into relatedContent.links)
    const answerRelatedContent = {
      links: [],
    };

    for (let round = 0; round < this.maxToolRounds; round++) {
     	// Request to the model with a description of available tools.
      const completion = await client.chat.completions.create(
        {
          model: params.model,
          messages,
          tools: this.getTools(),
          parallel_tool_calls: true,
          stream: false,
        },     
       // abortSignal allows you to abort a request from outside.
        { signal: params.abortSignal },
      );

      const assistantMessage = completion.choices[0].message;
      lastAssistantText = assistantMessage.content ?? lastAssistantText;

      const toolCalls = this.getFunctionToolCalls(
        assistantMessage.tool_calls ?? [],
      );

  	// The model returned the final text without calling tools - we're done.
      if (toolCalls.length === 0) {
        const result = this.mapResponse(
          assistantMessage.content ?? '',
          answerRelatedContent,
        );
        return result;
      }

	// Execute each tool call and collect responses for the next round.
      const toolOutputs = [];
      for (const item of toolCalls) {
        if (item.function.name === <TOOL NAME>) {
          const question = this.getArgForTool(item);
          const toolResponse = await this.callTool(question);

      
		// Add links from the tool response, if any
          answerRelatedContent.links?.push(...toolResponse.useddocs);

          toolOutputs.push({
            role: 'tool',
            tool_call_id: item.id,
            content: toolResponse.answer,
          });
        }
      }

  
	   // Adding to the history: assistant's message with tool_calls and tools' responses.
      messages = [
        ...messages,
        {
          role: 'assistant',
          content: assistantMessage.content ?? null,
          tool_calls: toolCalls,
        },
        ...toolOutputs,
      ];
    }

	// If maxToolRounds are exhausted, return the last known text.
    const result = this.mapResponse(lastAssistantText, answerRelatedContent);
    return result;
  }

	// Stream generation (SSE) with tool calling support.
  async *getCompletionStream(params) {
    const client = this.getClient(params.apiKey, params.apiUrl);
    let messages = this.getChatMessages(params);
    let lastAssistantText = '';

    const answerRelatedContent = {
      links: [],
    };

    for (let round = 0; round < this.maxToolRounds; round++) {
      const stream = await client.chat.completions.create(
        {
          model: params.model,
          messages,
          stream: true,
          tools: this.getTools(),
          parallel_tool_calls: true,
        },
        { signal: params.abortSignal },
      );

      let finishReason = null;
      let content = '';
      // Collect tool calls by index from stream deltas.
      const toolCallsByIndex = new Map();

      for await (const event of stream) {
        const choice = event.choices.at(0);
        if (!choice) {
          continue;
        }

        const delta = choice.delta;

        // Text fragments of the response — returned to the client as text_delta.
        if (delta.content) {
          content += delta.content;
          yield { type: 'text_delta', delta: delta.content };
        }

        // Gradually build tool_calls from stream.
        if (delta.tool_calls) {
          for (const toolCallPart of delta.tool_calls) {
            const current = toolCallsByIndex.get(toolCallPart.index) ?? {
              id: toolCallPart.id ?? '',
              type: 'function',
              function: { name: '', arguments: '' },
            };

            if (toolCallPart.id) {
              current.id = toolCallPart.id;
            }
            if (toolCallPart.function?.name) {
              current.function.name = toolCallPart.function.name;
            }
            if (toolCallPart.function?.arguments) {
              current.function.arguments += toolCallPart.function.arguments;
            }

            toolCallsByIndex.set(toolCallPart.index, current);
          }
        }

        if (choice.finish_reason) {
          finishReason = choice.finish_reason;
        }
      }

      lastAssistantText = content || lastAssistantText;

      // The thread terminated without finish_reason - exiting.
      if (!finishReason) {
        return;
      }

      const toolCalls = [...toolCallsByIndex.values()];

      // The final response without tools is to end the stream with the completed event.
      if (toolCalls.length === 0) {
        const response = this.mapResponse(content, answerRelatedContent);
        yield {
          type: 'completed',
          response,
        };
        return;
      }

      // We are implementing tools and preparing the next round of dialogue. 
      const toolOutputs = [];
      for (const item of toolCalls) {
        if (item.function.name === <TOOL NAME>) {
          const question = this.getArgForTool(item);

          // Notify the UI about the tool call (optional).
          yield {
            type: 'tool_call',
            tool: 'doc',
          };

          const toolResponse = await this.callTool(question);
          answerRelatedContent.links?.push(...toolResponse.useddocs);

          toolOutputs.push({
            role: 'tool',
            tool_call_id: item.id,
            content: toolResponse.answer,
          });
        }
      }

      messages = [
        ...messages,
        {
          role: 'assistant',
          content: content || null,
          tool_calls: toolCalls,
        },
        ...toolOutputs,
      ];
    }

    // Backup option if the round limit is reached.
    const result = this.mapResponse(lastAssistantText, answerRelatedContent);
    yield {
      type: 'completed',
      response: result,
    };
  }
}

module.exports = { CustomToolAdapter };