CALL AGENT
Experimental

Overview

The CALL AGENT operation is designed to communicate with an external LLM or AI Agent, such as Hermes, by calling the /v1/responses endpoint (default) or /v1/chat/completions, and sending a custom prompt. Because calling an AI Agent typically involves an initial configuration with system prompts and other settings, this operation requires minimal parameters; however, you can control certain key settings such as the temperature and reusing previous answers.

Two operations are supported: calling an agent and expecting a response or calling an agent asynchronously without waiting for a response. The former can be used for transforming data into a format that is easier to consume in the data pipeline, while the former can be used for starting a series of downstream actions, such as sending alerts or opening tickets.

Calling this operation sets a pipeline variables: @lastAgentResponseId. You can use this variable immediately after a CALL AGENT operation, even with the ASYNC option.

For more advanced control over the HTTP request being made to an agent, including using other OpenAI requests, use the APPLY HTTP operation instead.

An Agent operation may loop even after a pipeline is cancelled due to a timeout or when it is manually stopped. Calling the /v1/responses endpoint triggers a call to the /cancel operation, but this is on a best effort basis. You are responsible for configuring an agent's maximum loop setting to avoid large token consumptions.

Sending Data

You can send data to an LLM or Agent using different approaches depending on the capabilities of the agent and the amount of data. In general, two main techniques can be used: passing the data inline, or passing a URL to the data (Claim-Check).

Passing data inline

This method involves packaging the data as a flat file or a JSON/XML document first, or a base64 image content, and sending the data as part of the prompt or the FILE_DATA or FILE_URL parameter depending on the agent. Knowning which approach is best requires choosing the recommended approach from the agent documentation.

When passing data as a flat file or document, you need to first shape the data as such; use a combination of ADD COLUMN FORMAT and ZIP to construct a payload quickly in memory into a single row and column, then pass the data in the PROMPT directly or in the FILE_DATA parameter. Here is a typical implementation example:

SELECT * FROM DB [conn] (SELECT id, name, zipCode FROM customers);
ADD COLUMN 'csv' FORMAT CSV OPTIONS 'headers:0 delimiter:\t';
ZIP COLUMN 'csv' FORMAT CSV OPTIONS 'headers:1';
-- we now have a single row with a single column name
CALL AGENT [agent] PROMPT 'Summarize this data: {{csv}}';

You can use pipeline variables, data pipeline column names and DataZen functions as part of the prompt and other string parameters. However, if multiple rows exist in the data pipeline, only the first row will be considered; this operation does not loop for every row provided in the pipeline at this time. For example, this prompt is valid (assuming a pipeline data set with a column name payload exists, but only the first row will be processed):

CALL AGENT [agent] 
  PROMPT 'Analyze this payload and return a flat-file comma-separated 
  list of values found in the list node, and return a header line. Keep your 
  answer short with data only. Here is the data: 
  {{payload}} ';

Passing a URL Claim-Check

Another approach is to send the data to a file in the cloud and inform the agent which file name to download and inspect. This may be best for larger data sets but may also require additional agent configuration, including skills, to download the file. Here is a typical implementation example:

SELECT * FROM DB [conn] (SELECT id, name, zipCode FROM customers);

-- Save to a file, and replace the pipeline with the file name 
SINK INTO DRIVE [adls] FORMAT JSON CONTAINER 'data' FILE 'customers.json' REPLACE;

-- we now have a single row with the name of the file to load in the cloud
CALL AGENT [agent] PROMPT 'Download and summarize this data: {{container}}\{{file}}';

Sending Binary Data

Sending binary data can also be done using the above two techniques, but may require specific base64 encoding when sending data. The following example demonstrates how to read an existing image (it could be a PDF document too) and how to send it to an agent configured to process images:

-- get data from a cloud endpoint and retrieve raw bytes 
SELECT * FROM DRIVE [awss3] (myimage.bmp) 
WITH 
 CONTAINER 'images'
 FORMAT 'BYTES';

-- this call only works on the first row returned 
-- the bytes column is converted into a base64 output using the base64: field decorator
CALL AGENT [agent] 
     TEMPERATURE 0.1
     RESPONSE
     REASONING 'high'
     FILE_DATA 'data:image/bmp;base64,{{base64:bytes}}'
     PROMPT 'What is in this bitmap image? Provide a CSV output, one term per line, 
  of the relevant terms that describe this image. Return a header named "result". 
  Return the top 5 most relevant terms.  
 '
     CONVERSATION 'plan image analysis';

-- get the response
APPLY TX '..text' ;
-- denormalize the answer; use unixlf if needed 
APPLY TX 'headers:1 unixlf:1' ;

Processing an Agentic Response

Depending on the task given to the agent the output may be non-deterministic or highly predictable; for example, you can request an email being drafted, or if the prompt is well crafted the agent can return data using a very specific format and layout that you can turn into a data set with rows and columns directly using the APPLY TX operation.

Text Output

One way to process the output of an LLM or agent is to send it as-is to a target system or endpoint, such as requesting an email output or content that can be sent to a Slack channel for example. Here is an example that sends text to a Slack channel using DataZen:

CALL AGENT [devlap06-llm-hermes] 
 CHAT 
 TEMPERATURE 1.8
 MAX_TOKENS 80000
 PROMPT 'Send me a short recipe for a desert; pick the desert you want! 
Keep it simple and return a response that is Slack-friendy'
;

-- Get the result out of the response payload
APPLY TX '..content';

-- Send it to Slack by calling another pipeline 
START 'SEND TO SLACK' PARAMS 'message=#urlencode(#data.value(0,0))';

Data Output

Another way to process the output of an LLM or agent is by requesting the expected format; extracting the response from the agent becomes possible because the output schema is purposefully highly deterministic. Here is an example that provides a deterministic output:

-- when calling an LLM, you need to specify the model to load
CALL AGENT [devlap06-llm] 'openai/gpt-oss-20b'
 TEMPERATURE 0.1 
 CHAT -- calling the /chat/completions endpoint 
 PROMPT 'Return the first 10 Fibonaci numbers, one per row, as a CSV output with a header called Fibonaci. 
Do not include any other text or information; just the data.';

-- transform the response from the LLM 
APPLY TX '..content';
-- now transform the content which is a CSV output
APPLY TX 'headers:1';

CALL AGENT [CONNECTION] 'agent_name' 
	PROMPT '...'
	{ < RESPONSE | CHAT > }
	{ PER_ROW 
	  { WHEN '...' } 
	}
	{ FILE_URL '...'}
	{ FILE_DATA 'data:mime;base64,...' }
	{ DETAIL '< AUTO | LOW | HIGH >' }
	{ < RESPONSE_ID '...' | CONVERSATION '...' > }
	{ REASONING  '< AUTO | LOW | MEDIUM | HIGH >' }
	{ FILE_NAME '...' }
	{ MAX_TOOL_CALLS N }
	{ TEMPERATURE N }
	{ MAX_TOKENS N }
	{ TIMEOUT N }
	{ ASYNC }
	{ CONTINUE_ON_ERROR }
;

SELECT * FROM HTTP [weather] 
	(GET /forecast?forecast_days=7&daily=temperature_2m_max,precipitation_sum);

-- Save the HTTP payload (a JSON document) in a file 
SINK INTO DRIVE [adls] 
	FORMAT 'raw' COLUMN 'payload' 
	FILE 'forecast.json' 
	CONTAINER 'weather';

-- The agent is configured to read from a known cloud location
-- so we just need to provide the file name below 
CALL AGENT [hermes-connection] 'weather-parser-agent'
  RESPONSE 
  REASONING 'high'
  TIMEOUT 45
  TEMPERATURE 0.3
  FILE_URL 'https://.../forecast.json'
  PROMPT 'Read the file provided and extract the forecast found in this file, by date.
	Three fields should be returned: the date of the forecast, the temperature, 
	and the precipitation. 
	Strictly return the output as a CSV file delimited by a comma. No 
	other information should be returned and no header row should be returned either.
	The first column should be a date in the following format: yyyy-mm-dd'
	;

-- since we are getting back a CSV format, turn the output into 
-- rows and columns 
APPLY TX 'headers:0 delimiter:,' ;

SELECT * FROM HTTP [weather] 
	(GET /forecast?forecast_days=7&daily=temperature_2m_max,precipitation_sum);

-- Save the HTTP payload (a JSON document) in a file 
SINK INTO DRIVE [adls] 
	FORMAT 'raw' COLUMN 'payload' 
	FILE 'forecast.json' 
	CONTAINER 'weather';

-- The agent is configured to read from a known cloud location
-- and send Slack messages. 
-- In this example, the pipeline will not wait for a response
-- because this is an ASYNC call

CALL AGENT [openapi] 'weather-parser-agent'
  ASYNC 
  RESPONSE 
  TEMPERATURE 0.3
  REASONING 'medium'
  RESPONSE_ID 'resp_001'
  PROMPT 'Read the file at this location: forecast.json in the 
	blob container you already know about 
	and read its content to extract the forecast found in this file, by date.
	If an unusually high precipitation is expected, send a Slack message to the 
	Warnings channel.'
	;

-- Continue with the pipeline data...