kadi-core

---

The kadi-core module is a comprehensive toolkit for developers integrating with the KADI infrastructure. This module simplifies tasks such as managing agent.json files, spawning processes, interacting with brokers, and handling interprocess communication (IPC).

Features

• Agent JSON Management: Manage configurations for tools, agents, or systems using Kadi.
• Process Management: Support launching and managing subprocesses.
• Broker Interaction: Facilitate communications with brokers.
• IPC Support: Tools for interprocess communication across various programming languages.

Installation

Install kadi-core using npm:

npm install kadi-core

Documentation

Agent JSON Functions

• getAbilityJSON(abilityName, abilityVersion): Retrieves the agent.json for a specified ability.
• getAbilityJSONPath(abilityName, abilityVersion): Provides the file path to the agent.json for a specific ability.
• getAbilityVersionFromArray(abilities, name): Searches ability array provided, and returs the version number for name provided
• getAbilitiesDir(): Returns the directory path where abilities are stored.
• getProjectJSON(): Fetches the agent.json for the current project.
• getProjectJSONPath(): Gets the file path for the project's agent.json.
• getKadiCoreJSON(): Retrieves the agent.json for the Kadi core.
• getKadiCoreJSONPath(): Provides the file path to the Kadi core's agent.json.
• getKadiJSON(): Fetches the agent.json for the Kadi system.
• getKadiJSONPath(): Returns the file path for the Kadi system's agent.json.

Process Management Functions

• runExecCommand(name, version, command): Executes a command for initializing abilities.
• runSpawnCommand(name, version, command): Uses spawn to execute commands for subprocesses.

Usage Examples

import { getProjectJSON, runExecCommand, IPCManager, Broker } from 'kadi-core';

async function setupProject() {
  const projectConfig = getProjectJSON();
  console.log(projectConfig);

  await runExecCommand('example', '1.0', 'npm install');
  
  const ipc = new IPCManager();
  ipc.createInstance('python', 'pythonScript.py', 'pythonInstance');

  Broker.addBroker('ws://example.com', 'exampleBroker');
  let broker = Broker.getBroker('default');
    console.log(`Connected to ${broker.url}`);
    broker.send(BrokerMessageBuilder.setup("TestAgent", "A Broker Testing Agent", null, null));
}

Broker Functions

• Broker: Manages interactions with broker systems.
• IBroker: Broker instance interface with methods to manage its lifecycle and communications.
• BrokerMessageBuilder: Assists in constructing messages for broker communication.

Broker Functions

The Broker system facilitates interaction between different agents and services within the Kadi environment.

Broker Interface

• addBroker(url, name = 'default'): Adds a new broker connection. If a broker with the same name exists, it will be reused.
• disconnect(name = 'default'): Disconnects the broker specified by the name.
• deleteBroker(name = 'default'): Deletes the broker specified by the name and closes its connection.
• send(message, brokerName = 'default'): Sends a message through the broker specified by the name.
• addEventListener(event, listener, brokerName = 'default'): Adds an event listener to the specified broker.
• removeEventListener(event, listener, brokerName = 'default'): Removes an event listener from the specified broker.
• removeAllListeners(brokerName = 'default'): Removes all event listeners from the specified broker.
• getBroker(brokerName = 'default'): Retrieves a broker instance by name.

IBroker Instance

This class extends EventEmitter and manages individual broker connections:

• constructor(url, name): Initializes a new broker connection.
• send(message): Sends a message through the WebSocket connection.
• getConnectedAgents(): Retrieves a list of currently connected agents.

BrokerMessageBuilder

Utility class for constructing broker messages:

• create_message(type, data): Creates a JSON string with the specified type and data.
• message(to, content): Constructs a message directed to a specific peer.
• setup(name, description, limit, uuid): Creates a setup message for registering the broker.
• suspend(): Constructs a suspend message.
• finish(): Constructs a finish message.
• list(): Constructs a list message to request a list of connected agents.

IPC Functions

• IPCMessageBuilder: Constructs messages for IPC interactions.
• IPCManager: Manages setup and lifecycle of IPC connections.
• IAbilityIPC: Represents an IPC ability instance with methods to manage its lifecycle and communications.

IPCManager

Manages the lifecycle and setup of interprocess communications.

• createInstance(language, commandString, name = 'default'): Creates and manages a new IPC instance for the specified language.
• getInstance(name): Retrieves an existing IPC instance by name.
• shutdownInstance(name): Shuts down an existing IPC instance by name.

IAbilityIPC Instance

Manages an individual interprocess communication instance:

• start(): Starts the child process associated with this IPC instance.
• launch(): Sends the launch command to the associated process.
• sendMessage(message): Sends a JSON-formatted message to the child process.
• shutdown(): Sends a shutdown command to the child process and terminates it.

IPCMessageBuilder

Utility class for constructing IPC messages:

• createMessage(type, contents): Creates a JSON string with the specified type and contents.
• launch(commandString): Constructs a launch message with the specified command string.
• shutdown(): Constructs a shutdown message.
• message(content): Constructs a general message with the specified content.

Usage Examples

Here's how you might use the IPC and Broker functionalities:

Interacting with IPC

        console.log("Starting IPC Ability Interface test...");

        // Create Python IPC instance
        const pythonIPC = IPCManager.createInstance('python', 'echo "Hello, World!"', 'pythonTest');
        
        pythonIPC.on('error', (error) => {
            console.error("Handled Python Error:", error);
            // Consider appropriate error handling or recovery actions here
        });
        
        pythonIPC.on('critical-error', (error) => {
            console.error("Critical Error from Python:", error);
            // Handle critical errors, potentially restarting the process or alerting administrators
        });
        
        pythonIPC.on('message', (msg) => {
            console.log('Message from Python:', msg);
        });

        pythonIPC.on('broker', (msg) => {
            console.log('Broker message from Python:', msg);
        });

        pythonIPC.on('shutdown', () => {
            console.log('Python IPC has shut down.');
        });

        Broker.addBroker('http://your.broker.com', 'default');
        let brk = Broker.getBroker('default');
        broker = brk;

        pythonIPC.attachBroker(brk);

        brk.on('open', function open() {
            console.log(`Connected to ${brk.url}`);
            const jsonObject = BrokerMessageBuilder.setup("TestAgent", "A Broker Testing Agent", null, null);
            brk.send(jsonObject);

        });

        brk.on('message', function incoming(data) {
            let msg = JSON.parse(data);
            if(msg.type == 'setup'){
                (async () => {
                    try {
                        await GetLLMProxy();
                    } catch (error) {
                        console.error('Error during GetLLMProxy:', error.message);
                    }
                })();
            }
            // console.log('Received:', formatJSON(data));
        });

        brk.on('error', function error(error) {
            console.error('WebSocket error:', error);
        });

        brk.on('close', function close() {
            console.log('Disconnected from the server');
            process.exit(0); // Exit the process when the WebSocket connection is closed
        });

        // Send launch command

        //Utilize log files
        // pythonIPC.sendMessage(IPCMessageBuilder.launch(pythonIPC.commandString, process.cwd() + "/logs"));

        //No log files created
        pythonIPC.sendMessage(IPCMessageBuilder.launch(pythonIPC.commandString));
        

Handling IPC

const ipcManager = new IPCManager();
const ipcInstance = ipcManager.createInstance('node', 'app.js', 'nodeApp');
ipcInstance.on('message', message => console.log(message));

---

TODO:

• Once kadi-core and dependencies are done, kadi (and all commands), need to be rewritten, using the kadi-core module. this will be v1.0 release

• [ ] add getConnectedAgents to the Broker, it should call the IBroker method

• [x] add key to IBroker object

• [ ] IBroker on 'message' should listen for setup event, and update teh uuid and key for the IBroker object

• [ ] Extract Broker and IPC to their own node module and host on npm

• [x] Publish kadi-core to npm

• [x] Setup of python environment needs to happen once an IPC Inteface is created This include setting up the virtual environment and installing the required packages

• [x] Add method to IPCManger to set IPCInterfaces from agent.json. If IPCManger has empty list, it thorws an error that no IAbilityIPC's can be created. IPCManger passes the proper interface command to IAbilityIPC on creation from this list, IAbilityIPC does not need access directly to agent.json. Controlling app can update this list anytime a new IAbilityInterface is created, allowing the agent.json to be updated during runtime.

• [x] Need to adapt ipc.js to read from the kadi-core folder when trying to open IPC instances, instead of project folder. kadi-core shold be in the npm_modules folder (when deployed)

• [x] The test code needs to fully test the IPCManger interface, and not just the IPCAbility

• [x] If node fails, we need to make sure python IPC interface and child processes also close out.

• [x] Add buffer to error handeling in IAbilityIPC, mimic same process used in std input

• [x] Verify if Python IAbilityIPC needs to utilize buffering, and node passing '\n' at end of jso

• [x] Setup passthrough for broker messages to/from Language IPC interface

• [x] Add broker message builder/parser in python IAbilityIPC

• [x] Update BrokerMessageBuilder to use same format at IPCMessageBuilder

• [x] Update Broker so it extends EventEmitter, rather than have a property of event emitter (similar to IAbilityIPC)

• [x] update python IAbility so that it creates a timestamped command_output.log file, new file for each run, in the project root directory... not the kadi-core directory

• [x] Comment out debug messages

• [x] Add ability to allow for launched process to be piped to event handler (launch command should default to process, but allow user to set flag that will save all outputs to file in the event handler)