Call an AI Agent or LLM Inline

Overview

You can call an AI Agent or LLM inline within a data pipeline by leveraging one of the following commands: SELECT HTTP, APPLY HTTP, or CALL AGENT. Depending on the scenario, you may need to use the RAW_PAYLOAD option (only supported by the APPLY HTTP command) to send content as-is to the LLM; in other scenarios, you may need to build the JSON payload first (such as leveraging the ADD COLUMN FORMAT JSON or call a database engine to return a JSON column).

• APPLY HTTP: The APPLY HTTP operation can send commands to an LLM or AI Agent by calling an OpenAI endpoint, such as retrieving available models or sending a prompt. However, unlike the SELECT command, you can also send a payload in batches using the ZIP operation beforehand.
• CALL AGENT: The CALL AGENT is designed to call the /v1/chat/completions endpoint exclusively and allows a prompt to be specified. In addition, it can optioanlly operation as a "fire and forget" request using the ASYNC command, so that execution of the pipeline continues without waiting for a response. This is the most concise way of calling AI Agents such as Hermes.
• SELECT HTTP: The SELECT HTTP operation can send commands to an LLM or AI Agent by calling an OpenAI endpoint, such as retrieving available models or sending a prompt.

Return available LLM models

In this example, we return all the models the LLM endpoint supports using a SELECT HTTP command:

SELECT * FROM HTTP [llmendpoint]  (GET /models)
APPLY TX 'data';

Send a request to an AI Agent

In this example, we request an agent to start an asynchronous request based on data that is saved as part of the pipeline into a shared drive. When the data exceeds a few KBs, it is usually best to use a claim-check pattern by first creating a file then calling the agent referencing the file.

SELECT * FROM DB [sql2017] (SELECT * FROM sys.databases);

ADD COLUMN 'json' FORMAT JSON; 
ZIP COLUMN 'json' FORMAT JSON;

SINK INTO DRIVE [awss3] FORMAT 'RAW' COLUMN 'json' 
    FILE 'alldatabases.json' 
    CONTAINER 'trace';

-- This agent is already configured to connect to an S3 bucket and 
-- knows how to send Slack messages 
CALL AGENT [hermes] 'database-monitoring-agent'
    PROMTP 'Analyze the file alldatabases.json in the S3 bucket and determine if 
    any new databases were created within the last 24 hours. If so, send a Slack 
    message to the SQLAdmin channel with the names of the newly created databases.
    Keep your answers short and concise.'
    ASYNC;

Format and send a small JSON payload inline

In this example, the JSON payload is provided directly in the PAYLOAD property of APPLY HTTP.

APPLY HTTP [llmendpoint]
(POST /chat/completions)
WITH 
	PROCESSING 'replace'
	CONTENT_TYPE 'application/json'
	PAYLOAD (
{
	"model": "qwen/qwen3-30b-a3b-2507",
	"messages": [
		{
			"role": "system",
			"content": "You are a helpful AI assistant that can execute tools when needed.\n\nWhen a tool result is provided (role=tool), use it to answer the user. Do NOT call the same tool again with the same arguments - its result is already in the conversation. Only call a different tool if it is genuinely needed. Otherwise, produce a clear natural-language answer based on the tool result(s) already returned.",
			"tool_calls": null,
			"tool_call_id": null
		},
		{
			"role": "user",
			"content": "which model are you?",
			"tool_calls": null,
			"tool_call_id": null
		}
	],
	"temperature": 0.699999988,
	"max_tokens": 20000,
	"tools": [],
	"tool_choice": "auto"	
}
);