Node-RED Opik API Documentation

1. Overview

This document provides technical details for the two primary Node-RED flows designed to integrate with the Opik evaluation platform.

• Dataset Creation Flow: Automates the creation of question-and-answer datasets. It retrieves content from a specified knowledge base in ChromaDB, uses a Large Language Model (LLM) to generate Q&A pairs, and then creates and populates a new dataset in Opik.

• Evaluation Flow: Orchestrates the evaluation of an agent's performance. It triggers a test run against a specified dataset in Opik, using an LLM-based agent and scorer, and submits the detailed results back to an Opik experiment.

---

2. Prerequisites & Setup

Global Configuration

Before using these endpoints, you must configure several global context variables within your Node-RED environment. These variables store credentials and service URLs required by the flows.

Global Variable	Type	Description	Used By
openaiKey	String	Your secret API key for OpenAI (e.g., sk-...).	Both Flows
chroma_link	String	The full URL to your ChromaDB instance (e.g., http://your-chroma-db:8000).	Both Flows
opik_url	String	The base URL for the Opik API service.	Both Flows
opik_username	String	The username for authenticating with the Opik API.	Evaluation Flow
opik_password	String	The password for authenticating with the Opik API.	Evaluation Flow

Download the Flows

To get started, download the complete Node-RED flow configuration and import it into your environment.

You can download this flow here. (Right-click and "Save link as..." to download)

---

3. API Endpoints

3.1. Dataset Creation Flow

This flow exposes an endpoint to generate a new Q&A dataset in Opik from a knowledge base.

Endpoint: POST /generate-dataset-for-opik

Description: Initiates a process to fetch documents from a specified knowledge base collection, generate a set of question-answer pairs using an LLM, and upload them to a new, uniquely named dataset in Opik.

• Subflows Used: Question Generation Agent (d3f950f3d7083a33)

Request Payload

Key	Type	Required	Description	Example
topic	String	Yes	The name of the knowledge base collection in your system.	"Documents"
num_questions	Integer	No	The number of Q&A pairs to generate. Defaults to 5.	15

Success Response (201 Created)

Key	Type	Description	Example
status	String	Indicates a successful operation.	"success"
message	String	A human-readable confirmation message.	"Dataset 'docs-qa-175...8' created..."
dataset_name	String	The final, unique name of the dataset created in Opik.	"documents-qa-dataset-1753701963438"
items_added	Integer	The total number of Q&A pairs added to the dataset.	15
generated_questions	Array	An array of the generated Q&A pairs.	[{"source": "MANUAL", ...}]

Mapped Opik APIs

• POST /api/v1/private/datasets: To create the new dataset entry.
• PUT /api/v1/private/datasets/items: To add the generated Q&A pairs to the dataset.

Example Usage (cURL)

curl --location 'http://127.0.0.1:1880/generate-dataset-for-opik' \
--header 'Content-Type: application/json' \
--data '{
    "topic": "Documents",
    "num_questions": 10
}'

3.2. Evaluation Flow

This flow exposes an endpoint to trigger a new evaluation run for an agent.

Endpoint: POST /run-opik-evaluation

Description: This endpoint initiates a full evaluation harness. It finds or creates an experiment in Opik, fetches the items from the specified dataset, runs the main agent and scorer for each item, and submits all results in a single bulk request.

• Subflows Used: Opik Evaluation Harness (36d0e08ac7662b5b), Main KB Agent For Opik (d24c4d1df097038a), Opik Scorer (6166e65b7a46aea4).

Request Payload

Key	Type	Required	Description	Example
opik_dataset_name	String	Yes	The name of the dataset in Opik to use.	"documents-qa-dataset-1753701963438"
opik_experiment_name	String	Yes	The name for the experiment in Opik. It will be created if it doesn't exist.	"Weekly Regression Test"

Success Response (200 OK)

Key	Type	Description
status	String	Indicates the evaluation has completed.
message	String	A human-readable acknowledgment message.
items_evaluated	Integer	The total number of dataset items processed in the run.
experiment_name	String	The name of the Opik experiment where results were saved.

Mapped Opik APIs

• GET /api/v1/private/experiments: To find existing experiments.
• POST /api/v1/private/experiments: To create a new experiment if one is not found.
• POST /api/v1/private/datasets/retrieve: To fetch dataset details by name.
• GET /api/v1/private/datasets/{id}/items: To get all items within the dataset.
• PUT /api/v1/private/experiments/items/bulk: To submit all evaluation results, traces, and scores.

Example Usage (cURL)

curl --location 'http://127.0.0.1:1880/run-opik-evaluation' \
--header 'Content-Type: application/json' \
--data '{
    "opik_dataset_name": "documents-qa-dataset-1753701963438",
    "opik_experiment_name": "Weekly Regression Test - Documents KB"
}'

---

4. Error Handling

Status Code	Endpoint(s)	Error Message / Scenario	Possible Cause & Resolution
400 Bad Request	Dataset Creation	"Missing or invalid 'topic' in request body."	The topic field is missing or empty. Ensure it is a non-empty string.
404 Not Found	Dataset Creation	"Could not find a Knowledge Base with the name '...'."	The provided topic does not match any existing knowledge base name.
500 Internal Server Error	Both	An internal server error occurred.	Failed Connection: Node-RED cannot reach a required service (ChromaDB, OpenAI, Opik). Check service URLs in global context. Invalid Credentials: API keys or passwords are incorrect. Verify values in global context. Flow Logic Error: Check the Node-RED logs for detailed error messages from the specific node that failed.
422 Unprocessable Entity	Both	(Response from Opik API)	The data sent to the Opik API was syntactically correct but semantically flawed. Check the Node-RED logs for the full error response from Opik.

---

5. Related Guides

Now that you understand the API, here are some guides to help you set up the required services:

Node-RED Guides

• Configuring Global Environment Variables in Node-RED: A guide to setting up global context variables.

Opik Guides

• Creating an Opik Service on the UBOS Platform: A step-by-step guide to provision your own Opik instance, which is required for both flows.
• Creating a Dataset in Opik using Node-RED: A detailed walkthrough of the dataset creation flow.
• Running an Evaluation in Opik using Node-RED: A detailed walkthrough of the evaluation flow.
• The Opik Scorer Subflow: Documentation for the Opik Scorer subflow.
• The Classic Test Agent Flow: Documentation for the Classic Test Agent flow.