DataForge MCP server for Databricks
Build governed data pipelines on your Databricks compute with AI agents
Connect Claude Code, Codex, or another MCP client to DataForge. Your agent can build and operate the complete pipeline lifecycle while DataForge validates every write before it persists.
DataForge is an independent data engineering platform. This is the DataForge MCP server for DataForge pipelines running on Databricks; it is not the official Databricks MCP server.
Your data stays yours
The agent changes the platform, not your cloud ownership model
Customer data and processing remain in your cloud and Databricks account. The MCP server gives an agent a governed route into DataForge's control plane; it does not copy your data into a separate agent-managed platform.
- Compute
- Your customer-owned Databricks account
- Agent access
- OAuth-authenticated and constrained by the signed-in user's role
- Unit of change
- Governed DataForge objects, with column-level transformation rules
- Validation
- The same save-time checks used for human edits, with no AI bypass
A governed build process, not just a connector
Native Databricks MCP access is broad. DataForge adds the architectural boundaries.
Native Databricks MCP servers primarily help agents use provider data and services. The DataForge MCP server helps agents build and operate the governed pipelines that produce and deliver that data. Cloud permissions determine what an agent may access; DataForge adds a consistent architecture and validation process for the pipeline changes your team must maintain.
| Evaluation point | Native Databricks MCP Official documentation ↗ | DataForge MCP on Databricks |
|---|---|---|
| Primary purpose | Databricks provides managed MCP servers for Genie, AI Search, Databricks SQL, and Unity Catalog functions. The SQL server can read and write data through AI-generated SQL. | Build and operate ingestion, transformation, orchestration, and delivery as governed DataForge pipeline objects. Read the 10.3 overview. |
| Architecture | The native server governs access to provider services and objects. Its documentation does not prescribe one end-to-end architecture for ingestion, refinement, orchestration, and delivery; the team defines that implementation. | Alloy's declarative architecture and platform object model establish one consistent build path across ingestion, refinement, orchestration, and delivery. Agents and people work inside the same structure. |
| Identity and RBAC | OAuth or personal access tokens authenticate the caller. Unity Catalog permissions govern the data and tools the caller can use, and Unity AI Gateway can apply policies and monitoring. | OAuth authenticates the user. DataForge requires an explicitly selected project, rejects conflicting project arguments, and rechecks the user's required project role in the database on every scoped call. |
| Project isolation | Native permissions isolate the services and Unity Catalog objects available to the caller. Scope follows the configured server and the caller's catalog and schema privileges. | Every DataForge pipeline action is bound to one explicitly selected project. Object lookups and writes verify project ownership, preventing an ID from another project from being used accidentally. |
| Save-time validation | The provider validates the requested API or SQL operation and enforces object permissions. Pipeline-wide design and dependency checks depend on what the team builds around those tools. | Every write uses the same parse-and-save path as a human edit. DataForge checks schemas, expressions, types, relationships, and output compatibility before persistence. Invalid changes do not save. See the five evaluation criteria. |
| Dependencies and change scope | The agent can inspect provider metadata, but the implementation is responsible for resolving pipeline dependencies and controlling the blast radius of code, SQL, or object changes. | DataForge resolves upstream relationships and validates downstream effects across rules, relations, and output mappings. Column-level rules keep routine transformation changes narrow and block incompatible downstream output changes. |
| Processing sequence and state | Some native tools expose asynchronous status and polling. Coordinating state across a complete data-pipeline lifecycle remains part of the surrounding implementation. | DataForge tracks ingestion and processing state across the pipeline. Sequencing guards stop agents from building on metadata that is not ready, while transport-safe waits return progress, retry timing, and resumable next calls. |
| Embedded orchestration | Native MCP tools invoke provider services. The team composes those calls into its own pipeline orchestration and operating process. | Agents can configure schedules, start pipeline processing, wait for completion, interpret recovered retries, run outputs, and cancel approved work through the same governed platform lifecycle. |
| Data access posture | The SQL MCP server can read and write Unity Catalog data within the caller's permissions. Other managed servers expose narrower services such as Genie and AI Search. | Pipeline work occurs through governed metadata objects. AI access to customer data requires customer opt-in and is disabled by default; when enabled, the query tool accepts read-only SQL. Metadata queries use a dedicated read-only connection. |
| Sensitive and repeated actions | Provider permissions constrain available operations; additional approval or confirmation behavior depends on the exposed tool and agent configuration. | Destructive operations require explicit confirmation, approval-gated operations require a separate approval flag, and identical repeated calls are coalesced instead of executing the same write twice. |
| Audit and telemetry | Unity AI Gateway centralizes MCP usage and activity monitoring. Optional request and response logging can write interaction data to Unity Catalog tables. | Customers can opt in to shared MCP and Talos tool telemetry for caller, project, operation, outcome, latency, and release version. It records argument and result shape rather than customer values, and token or secret fields are redacted. Tour the guard pipeline. |
| Operating guidance | The agent uses the tools, descriptions, and documentation supplied by the provider server and the team's implementation. | The deployed release serves its matching agent guide, object taxonomy, status codes, and SQL conversion guidance on demand, reducing drift between an agent's instructions and the version it operates. |
| Best fit | Well suited to direct provider exploration, administration, and custom tasks where flexibility is the priority. | Hardened while building a live, multi-system ERP reporting solution, then used by a data-literate newcomer to extend it. Read the field story. |
Use native Databricks MCP and DataForge MCP together
Native MCP access gives agents a direct route to Databricks data and services. DataForge supplies the deterministic build plane for the agent runtime your team already uses, applying architecture, dependencies, orchestration, and validation to pipeline work.
DataForge runs with your Databricks account and inherits its security perimeter. It complements, rather than replaces, Databricks's security and governance model.
Built for Databricks
A governed architecture on the platform you already operate
Alloy stages run as Delta tables or views and remain queryable with Databricks SQL. Hub tables build under Unity Catalog, and Ember metadata can be queried through Lakehouse Federation.
How it works
One governed path from agent intent to a running pipeline
01
Connect as yourself
Claude Code, Codex, or another compliant MCP client connects through browser-based OAuth. The agent acts with the signed-in user's DataForge permissions instead of a shared privileged identity.
02
Build governed pipeline objects
The agent works across ingestion, transformation, orchestration, and delivery as structured DataForge objects designed to run on your Databricks compute.
03
Validate before anything persists
Schema constraints, SQL parsing, role checks, sequencing guards, and destructive-action confirmation run at the tool boundary. Invalid writes are rejected with structured recovery instructions.
Why teams evaluate it
Move faster without giving the agent an ungoverned back door
General-purpose agents can write files, notebooks, and SQL quickly. DataForge gives them the architecture, metadata context, and validation needed to build changes that fit the platform your team must operate after the demo ends.
Review MCP security and technical controls →- Complete pipeline lifecycleIngestion, transformation, orchestration, delivery, and operations share one governed platform.
- Structured recoveryWhen a write fails, the agent receives the reason, the next call to make, and when to retry.
- Shared controlsMCP clients, Talos, and human users operate through the same registry, permissions, validation, and telemetry.
Questions about DataForge MCP on Databricks
What is the DataForge MCP server for Databricks? +
It is the DataForge MCP server used with a DataForge deployment whose pipeline processing runs in the customer's Databricks account. It lets external AI agents build and operate governed DataForge pipeline objects while DataForge validates every write before it persists.
Is DataForge the official Databricks MCP server? +
No. DataForge is an independent data engineering platform. Its MCP server governs DataForge pipelines that run on customer-owned Databricks compute; it is not operated or endorsed by Databricks.
Does DataForge move customer data out of Databricks? +
No. Customer data and pipeline processing remain in the customer-managed cloud and Databricks account. The MCP server operates through DataForge's control plane and governed metadata rather than moving customer data into a separate AI service.
What can an AI agent do through the DataForge MCP server? +
An authenticated agent can work across ingestion, transformation, orchestration, delivery, and operations through the DataForge tool registry. Available actions follow the signed-in user's permissions, and every write passes the same save-time validation as a human edit.
Which MCP clients are supported? +
Claude Code, Claude on the web or desktop through custom connectors, and Codex are documented clients. The server uses OAuth 2.1 with PKCE and dynamic client registration, allowing compliant MCP clients to connect without a shared API key.
Does DataForge work with Unity Catalog? +
Yes. DataForge builds Alloy stages as Delta tables or views in the customer's Databricks environment. Hub tables build under Unity Catalog so teams can apply its lineage and access controls.
Can teams query DataForge stages and metadata from Databricks? +
Yes. ORE, MINERAL, ALLOY, INGOT, and PRODUCT stages are queryable with Databricks SQL. Ember metadata can also appear as Unity Catalog tables through Lakehouse Federation.
Is the DataForge MCP server identical on Databricks and Snowflake? +
The governed interaction model is shared, while the tool registry is platform-aware. Databricks-specific and Snowflake-specific tools appear according to the platform selected for the DataForge deployment.
Put governed agents to work