Databricks Is Databricks for Agents
The agent platform on Databricks: execution, context, governance, evaluation, AI Runtime with NVIDIA, and serving open models. Follow the capabilities and the connections between them.
On Databricks, an agent can query governed data, retrieve documents, remember earlier sessions, execute tools, and call a hosted model or a custom open-model endpoint. You can deploy the application, inspect its traces, evaluate changes, and train a model on managed GPUs in the same platform.
That is the case for “Databricks for agents.” The familiar data-platform capabilities are part of it: storage, SQL, pipelines, machine learning, and governance. Agent Bricks adds the application services around them. AIR and Model Serving extend the workflow to teams that want to own their model weights and inference configuration.
The useful question is how these capabilities connect. A source table can feed retrieval. A model endpoint can serve several agent applications. An execution can become an analytical record, an evaluation example, or reviewed training data. The connections let developers work on the whole system while retaining choices about the harness and model. [1]
I’m an AI Solutions Architect at Databricks. This walkthrough uses capabilities documented by 31 October 2026. The diagrams are explanatory; the trace explorer uses a labeled synthetic dataset and runs in your browser.
What Databricks provides for an agent
There are three common starting points: build against a hosted model, deploy an open model unchanged, or adapt an open model to your task. The application still needs context, tools, execution, evaluation, and access controls in all three cases. The model-development path is what changes.
Choose a path below. Each row identifies an actual Databricks capability. Select a row to inspect its inputs and outputs.
This is the broader platform claim. A trace viewer covers one part of the work. Databricks also provides the data systems, application runtime, model infrastructure, and GPU environment that sit around that record. You can use those services independently or connect them for an end-to-end implementation.
Agent Bricks runs your application
Agent Bricks is the developer platform for an agent built with your own framework or harness. The harness remains your code: its instructions, control flow, tools, and stopping conditions. Genie Agents offers a separate route for creating agents without writing that orchestration; a custom agent can also call a Genie Agent as a tool. [1]
For a custom application, Agent Runtime hosts the agent server on Databricks Apps. The deployment has an authenticated endpoint and runs as the app’s service principal. The CLI creates or reuses the declared session and memory resources and configures MLflow tracing. With DurableAgentServer, it also provisions the Runtime Store used for execution state. [2]
agentbricks --profile <profile> deploy my-agentDurableAgentServer handles synchronous, streaming, and background requests, with persistent run state for recovery after interruptions. A model call is only one operation inside that longer application run. External tool writes still need their own transaction and idempotency design. [3]
Databricks Sandbox is another execution primitive. It gives the agent an isolated place to run generated code and retain files in its persistent home directory. It can be invoked through the platform APIs or used as an agent tool. Sandbox is Beta, with its own region, resource, and runtime limits. [4]
Hosts the agent server, its request lifecycle, and orchestration.
Runs training, GPU experiments, and supported batch workloads.
Hosts the model process that answers inference requests.
These are separate deployments and allocations. An application can run on Agent Runtime and call a model on Model Serving; AIR comes into the picture when the team needs GPU development or training. Keeping those roles explicit makes the architecture easier to operate.
Context includes search, SQL, tools, and memory
Databricks exposes several ways to give an agent useful information. Their interfaces reflect different data needs.
| What the agent needs | Databricks capability | What comes back |
|---|---|---|
| Relevant passages from documents | AI Search | Ranked documents and source metadata |
| An analytical answer over structured data | Genie One MCP + Genie Ontology | An answer, SQL results, and cited sources |
| Conversation history | Agent sessions | The current interaction’s retained messages |
| Knowledge retained across conversations | Agent memory, backed by Lakebase | Retrieved memory entries |
| A read or action in an application | Managed/custom MCP services and tools | A tool result under the configured identity |
AI Search, formerly Vector Search, creates indexes over source tables. Delta Sync indexes can follow changes in their source. Hybrid keyword and similarity search, metadata filters, and reranking let a developer control retrieval. Exact identifiers and natural-language descriptions can therefore participate in the same search workflow. [5]
Genie One MCP provides an analytical tool to the outer agent. The agent asks a natural-language question; Genie uses governed definitions and relationships from Genie Ontology, runs SQL, and returns cited results. The current service is system.ai.genie_one_mcp, accessed through Unity Gateway. This makes the analytical capability reusable by an agent built in another framework or hosted outside Databricks. [6]
Managed memory uses Lakebase to retain entries across conversations, with listing and relevance search. Sessions retain the conversation in progress. A memory pipeline can distill entries from session data. The application selects a trusted actor identifier; store permissions define the security boundary, while actor IDs organize entries within it. Managed memory is Beta. [7]
In an implementation, I would choose the interface per tool. An individual order lookup needs a typed application call; a question about aggregate activity can go to Genie; a manual lookup can go to AI Search. That exposes more of the data platform to the agent without making the harness rediscover every schema and retrieval mechanism.
Govern model and tool access alongside data
Unity Catalog supplies the governance layer for data and model artifacts. Unity Gateway extends controls to supported AI services: model services, MCP services, functions, and connections. Teams can configure permissions, service policies, consumption limits, and usage tracking around those calls. [8]
The two layers address related tasks. A table grant controls access to data. A model or MCP service policy controls an AI service interaction. Agent Runtime’s app identity, or configured request-user authorization, determines whose permissions a tool call uses. These are concrete deployment settings, rather than a general promise that every resource automatically inherits the same access rules.
The endpoint type also matters. The documented capability matrix distinguishes hosted foundation-model endpoints, external models, and custom endpoints. For example, custom model endpoints do not inherit all of the legacy AI Gateway guardrail and fallback features. Check the path your application actually uses. [9]
Serve open models with your inference environment
A hosted model is a useful starting point: select a supported Foundation Model API or external service and focus on the application. Custom Model Serving is the alternative when you need your own checkpoint, an adapter, or control over the inference environment.
The documented vLLM route begins in an AIR notebook. Start the inference server on a GPU, query it, and validate the model’s behavior. The deployment then packages that command and environment for a separate Model Serving endpoint. Qwen3.5-4B has an A10 notebook starter with an A10G endpoint on AWS; other documented starters cover tool-calling models, embeddings, and speech. [10] [11]
A deployable release includes the weights or adapter, tokenizer, chat template, tool parser, dependencies, and inference flags. Registering only a checkpoint path leaves out parts of the system that affect its responses. Express deployments provide the environment packaging used by this route. The partial registration step below follows the model-logging step in the linked example. [12]
mlflow.register_model(
model_info.model_uri,
"agent_lab.models.my_model",
env_pack="databricks_model_serving",
)There is also a documented multi-adapter pattern. vLLM loads a base model and selects a packaged LoRA adapter per request. Several specialized agents can therefore share the same base-model service while retaining distinct adapters and evaluation records. [13]
The custom vLLM route is Public Preview at this snapshot. It has cloud and region constraints and no dynamic autoscaling. Training hardware and serving hardware must be selected separately, and throughput, queueing, tool-call formatting, and long-context behavior need workload testing. These are the operational details of bringing your own model, and they belong in the platform explanation.
Train and adapt models on AIR with NVIDIA
AI Runtime adds GPU model development to the agent stack. It became generally available on 21 October 2026. Teams can run training code, custom containers, and distributed workloads on managed GPU infrastructure, with access to governed data and experiment tracking in MLflow. Lakeflow Jobs can connect preparation, training, and evaluation in a repeatable workflow. [14]
The NVIDIA integration is specific: NeMo, NeMo Automodel, NGC containers, and Nemotron cookbooks. Artifact Registry provides a route for the container environment. Developers can use NVIDIA’s software while AIR handles the underlying GPU infrastructure. Genie Code and NVIDIA-specific Agent Skills can assist with authoring, debugging, and profiling the training code. [15]
This extends beyond a GPU notebook. Distributed training has communication and data-loading requirements; the NVIDIA announcement describes NCCL and RDMA networking as part of that stack. The selected recipe still determines the model format, optimizer, precision, parallelism, and checkpoint handling. Those configurations should travel with the MLflow run and the selected output artifact.
databricks air run --file workload.yaml --watchThe AIR examples include full supervised fine-tuning of Qwen3.5-0.8B on one H100 followed by vLLM serving, LoRA fine-tuning of GPT-OSS 20B on eight H100s, and GRPO post-training of Gemma4-2B. They demonstrate different workloads rather than a single training method for every agent. [16]
The development sequence is straightforward: prepare and version a dataset, run the selected training recipe, log its configuration and artifacts, evaluate the result, and deploy a compatible inference environment. The resulting endpoint is a model target for the agent. This gives a team an open-model path that includes adaptation as well as inference.
Agent executions are data you can compute over
The original data platform is useful here too. Agent runs contain prompts, retrieved passages, tool calls, responses, and reviewer feedback. Databricks documents OpenTelemetry ingestion into Unity Catalog tables, so those executions can be queried and joined to other accessible data. An agent can use that path while running outside Databricks. [17]
The explorer below makes this tangible. It contains 480 synthetic support traces with a planted harness regression. Switch between a single execution and the collection, then group the records. The displayed counts are calculated from the browser dataset, not a reported Databricks benchmark.
In this dataset, the v13 harness has 25 failed exchanges in 55 runs, compared with one in 63 for v12. The grouping identifies a pattern worth investigating. Model groups happen to have similar aggregate failure rates because that is how the demonstration was constructed; a real model decision would need a controlled comparison.
SQL provides the filtering, aggregation, and joins. AI Functions can add candidate categories or extracted fields to execution text, and Genie Code provides a natural-language interface for investigating traces. The result remains data that can feed a dashboard, a reviewed evaluation dataset, or another pipeline. This is where the Databricks analogy reaches beyond operational telemetry. [18]
MLflow connects observation to evaluation
MLflow supplies the tracing, feedback, dataset, and scoring workflow. A developer can inspect the model and tool spans; a reviewer can attach an assessment to the execution; representative cases can enter an evaluation dataset. Code-based checks and model-based judges then compare a changed prompt, tool implementation, harness, or model. [19]
For the tool-calling example, I would score argument correctness separately from whether the final reply matches the tool result. Keeping those checks distinct makes the evaluation report useful when deciding what to change. Production monitoring can apply scorers to a configurable sample of incoming traces; that service is Beta. [20]
How the capabilities connect
Now the trace-to-model loop has its proper place: it connects capabilities already described in the article. Drag the token or select a stage to follow a request through execution, analysis, evaluation, training, and serving.
Those handoffs need to be implemented. Log the dataset revision and split manifest with the training run, retain model and harness versions with evaluations, and tag executions with the endpoint release. Unity Catalog and MLflow provide the storage and tracking mechanisms; the application and pipeline code establish the actual relationships.
A public dataset for exercising the stack
Retail tool calling remains a useful example because Sierra’s τ²-bench environment supplies simulated interactions, tools, and task definitions. inclusionAI’s related AReaL-tau2-data release lists 11,395 retail SFT records and 563 retail RL records. The SFT inventory counts assistant turns with context, rather than independent full conversations. [21] [22]
I would use it to exercise the platform path: put the pinned source data and split manifests under Unity Catalog, build the tool-using harness, evaluate its model calls with MLflow, run a supported AIR adaptation recipe, and serve the selected artifact through the custom endpoint. Canonical benchmark tasks and any added retrieval or memory behavior should be reported separately.
The companion repository, databricks-agent-loop, runs this path end to end on a live workspace, using its own generated retail orders. A support agent on Agent Runtime handles 1,520 requests, and its traces land in Unity Catalog. SQL over those traces finds the harness change that broke 68 of 100 exchanges, and an MLflow evaluation proves the fix. AI Runtime then fine-tunes Qwen3.5-0.8B on the agent’s own passing traces and registers it with a vLLM serving entrypoint. Served through vLLM on 60 held-out exchanges, the fine-tuned model picked the right item and size every time, matching the hosted model it learned from. Each step is one script, and the README has the full results.
The dataset gives the walkthrough a concrete workload. The Databricks capabilities remain the subject: the same architecture can support a coding agent, a document agent, or another tool-using application with different data and evaluators.
Why Databricks is Databricks for agents
Databricks supplies both the online application services and the data-and-model development environment. The agent can reach governed data, run tools, retain state, and use a hosted or custom model. Developers can analyze its executions, evaluate changes, adapt open models on GPUs, and operate the resulting endpoints.
The value comes from the connections and the choices that remain available. A team can bring its harness, start with a hosted model, later serve an open checkpoint, and add training when the task warrants it. The data platform continues to support the context, execution records, evaluations, and training artifacts throughout that work.
That is the claim worth demonstrating: Databricks already covers the work implied by “Databricks for agents,” including agent execution, agent data, model development, and inference.
Availability snapshot: AIR is GA; custom vLLM serving is Public Preview; managed memory, Sandbox, and MLflow production monitoring are Beta in the linked October documentation. Browser explorers illustrate the architecture and documented recipes. The companion repository runs the loop on a live workspace, and its README reports those results. Interaction inspiration: David Aerne.