Call an LLM Inline

Overview

You can call an LLM inline within a data pipeline by leveraging the SELECT HTTP or APPLY HTTP operation. Depending on the scenario, you may need to use the RAW_PAYLOAD option (only supported by the APPLY HTTP command) to send content as-is to the LLM; in other scenarios, you may need to build the JSON payload first (such as leveraging the ADD COLUMN FORMAT JSON or call a database engine to return a JSON column).

You can also include tools as part of this operation if needed based on the LLM you use.

In this example, we return all the models the LLM endpoint supports using a SELECT HTTP command:

SELECT * FROM HTTP [llmendpoint]  (GET /models)
APPLY TX 'data';

In this example, the JSON payload is provided directly in the PAYLOAD property of APPLY HTTP.

APPLY HTTP [llmendpoint]
(POST /chat/completions)
WITH 
	PROCESSING 'replace'
	CONTENT_TYPE 'application/json'
	PAYLOAD (
{
	"model": "qwen/qwen3-30b-a3b-2507",
	"messages": [
		{
			"role": "system",
			"content": "You are a helpful AI assistant that can execute tools when needed.\n\nWhen a tool result is provided (role=tool), use it to answer the user. Do NOT call the same tool again with the same arguments - its result is already in the conversation. Only call a different tool if it is genuinely needed. Otherwise, produce a clear natural-language answer based on the tool result(s) already returned.",
			"tool_calls": null,
			"tool_call_id": null
		},
		{
			"role": "user",
			"content": "which model are you?",
			"tool_calls": null,
			"tool_call_id": null
		}
	],
	"temperature": 0.699999988,
	"max_tokens": 20000,
	"tools": [],
	"tool_choice": "auto"	
}
);