DataZen Documentation
DataZen User Guide

CALL AGENT

Overview

The CALL AGENT operation is designed to communicate with an external LLM or AI Agent, such as Hermes, by calling the /v1/chat/completions endpoint, and providing a custom prompt. Because calling an AI Agent typically involves an initial configuration with system prompts and other settings, this operation requires minimal parameters.

Two operations are supported: calling an agent and expecting a response or calling an agent asynchronously without waiting for a response. The former can be used for transforming data into a format that is easier to consume in the data pipeline, while the former can be used for starting a series of downstream actions, such as sending alerts or opening tickets.

For more advanced control over the HTTP request being made to an agent, consider using the APPLY HTTP operation instead.

An Agent operation may loop even after a pipeline is cancelled due to a timeout or when it is manually stopped. You are responsible for configuring an agent's maximum loop setting to avoid large token consumptions.

Syntax

Calls an AI Agent's /completions endpoint to perform the desired action based on the prompt provided.

CALL AGENT [CONNECTION] 'agent_name' 
	PROMPT '...'
	{ ASYNC }
;

ASYNC

When specified, the pipeline does not wait for the agent's response

PROMPT

A free-text prompt to send to the agent, which may contain pipeline parameters

Example 1

SELECT * FROM HTTP [weather] 
	(GET /forecast?forecast_days=7&daily=temperature_2m_max,precipitation_sum);

-- Save the HTTP payload (a JSON document) in a file 
SINK INTO DRIVE [adls] 
	FORMAT 'raw' COLUMN 'payload' 
	FILE 'forecast.json' 
	CONTAINER 'weather';

-- The agent is configured to read from a known cloud location
-- so we just need to provide the file name below 
CALL AGENT [hermes-connection] 'weather-parser-agent'
  PROMPT 'Read the file at this location: forecast.json in the 
	blob container you already know about 
	and read its content to extract the forecast found in this file, by date.
	Three fields should be returned: the date of the forecast, the temperature, 
	and the precipitation. 
	Strictly return the output as a CSV file delimited by a comma. No 
	other information should be returned and no header row should be returned either.
	The first column should be a date in the following format: yyyy-mm-dd'
	;

-- since we are getting back a CSV format, turn the output into 
-- rows and columns 
APPLY TX 'headers:0 delimiter:,' ;

Example 2

SELECT * FROM HTTP [weather] 
	(GET /forecast?forecast_days=7&daily=temperature_2m_max,precipitation_sum);

-- Save the HTTP payload (a JSON document) in a file 
SINK INTO DRIVE [adls] 
	FORMAT 'raw' COLUMN 'payload' 
	FILE 'forecast.json' 
	CONTAINER 'weather';

-- The agent is configured to read from a known cloud location
-- and send Slack messages. 
-- In this example, the pipeline will not wait for a response
-- because this is an ASYNC call

CALL AGENT [hermes-connection] 'weather-parser-agent'
  ASYNC 
  PROMPT 'Read the file at this location: forecast.json in the 
	blob container you already know about 
	and read its content to extract the forecast found in this file, by date.
	If an unusually high precipitation is expected, send a Slack message to the 
	Warnings channel.'
	;

-- Continue with the pipeline data...