DataForge Platform
Outcrop, the DataForge agent that runs inside your network
It reaches the data behind your firewall.
Some of the most valuable data in an enterprise still sits on a server behind a firewall: the enterprise resource planning (ERP) system that runs the plant, the order system that predates the cloud, the operational database that security policy keeps off the internet. Outcrop is a single lightweight jar that runs in that network. It ingests files and database extracts into governed tables in your own Databricks or Snowflake account, and streams SQL Server change data into Databricks.
Your network
Your Databricks or Snowflake account
Platform processing builds every layer
INGOT holds the governed hub table for each source
One jar inside your network, one connection out
Outcrop installs as a Windows service through a standard installer, or runs as a managed container in cloud deployments. It starts, it calls out, and DataForge answers its heartbeat with the work to do next.
The connection you configure in DataForge and the pipeline that consumes it are the same object model, managed in one place. Outcrop is part of the platform rather than a separate product you buy. It is the component name, and the DataForge UI shows it as Agent today.
Installed once
A Windows service, or a managed container in cloud deployments. Java 21, one jar, one process.
Driven by heartbeat
Outcrop asks for work on a schedule DataForge controls, and commands arrive as responses to that request.
Updated by the platform
A release flag triggers an atomic jar replacement and self-restart, so the fleet stays current on its own.
Ingestion modes
One lightweight agent, every on-premise source
Outcrop is organized by how data moves. Four ingestion modes run through the same agent, with the same install, the same credentials handling, and the same scheduling.
Files
Outcrop watches a directory and picks up what arrives.
- Local and UNC network shares
- SFTP
- Amazon S3
- Azure Blob Storage
- File masks and multipart files
Databases, batch JDBC
Scheduled extraction over Java Database Connectivity (JDBC) from the engines already in your data center.
- SQL Server, Oracle, PostgreSQL, MySQL
- Amazon Redshift, Snowflake, SAP HANA
- Elasticsearch, Microsoft Access, Pervasive
- Generic JDBC for any engine with a driver
ERP, licensed plugins
Add-on connectors for the systems that run the business.
- SAP ECC
- NetSuite
- QuickBooks
- Licensed add-ons to the base agent, not included by default
Streaming CDC replication
Continuous change capture from SQL Server into governed tables, using log-based change data capture (CDC) or Change Tracking.
- SQL Server sources
- Log-based CDC and Change Tracking
- Merges applied over a Databricks Serverless SQL Warehouse
- Changes arrive within seconds
Some connections run straight from DataForge compute.
Salesforce, Kafka events, and Databricks Unity Catalog connect that way, with no agent in the path. Outcrop covers the sources that sit behind a firewall.
Streaming replication
Log-based CDC and Change Tracking, streaming into governed tables
Outcrop polls the source, detects key columns and refresh behavior from the source configuration, performs a full initial load, and then continuously merges inserts, updates, and physical deletes into the destination table. Changes arrive within seconds.
Two tracking methods
Log-based CDC polls the SQL Server CDC change tables, and auto-detects key columns and refresh type from the source's CDC configuration. Recovery is automatic when the log retention window expires.
A second tracking method, SQL Server Change Tracking, is the default. Change Tracking needs a primary key and snapshot isolation on the source, while log-based CDC needs full CDC enablement. In a locked-down production database, Change Tracking asks much less of your database administrators.
Where the changes are written
Merges are applied into the governed hub table for that source, INGOT._<sourceId>, over a Databricks Serverless SQL Warehouse. Very large tables can start with a batch-first onboarding pass and then switch to streaming from a saved watermark.
Tables with keys get row-level replication. Keyless tables fall back to full reloads.
What streaming replication covers
- Streaming replication covers SQL Server sources. Other database engines are ingested through batch JDBC extraction.
- The streaming target is Databricks. Snowflake deployments use batch ingestion.
Replication specialists cover more source engines, including Oracle, SAP, and mainframe systems.
Where replicated changes end up
A standalone replication product ends at a landing zone. The vendor's pipeline writes raw change data into a staging area, and your transformation tool picks it up from there. That leaves a second schema for someone to own and keep fresh.
DataForge replication ends inside your lakehouse, not beside it. Outcrop's merges are applied directly into the governed hub tables the declarative transformation layer already consumes. The moment a change is merged, it is sitting in a governed table that your rules already read and your lineage already tracks.
Verified claim, as of July 2026, based on public vendor documentation
DataForge is the only column-level declarative transformation platform with replication built in: a single outbound-only agent inside your network streams log-based SQL Server change data capture straight into governed tables in your own Databricks account, where DataForge rules pick it up. One product, one lightweight agent, no per-row fees.
What column-level declarative means
In DataForge, every transformation rule writes exactly one column and is validated at save time in one transactional metastore. Replicated data flows into that rule system directly. The alternative is a separate modeling tool with its own metastore and its own pricing meter.
The same platform your agents build in
The governed tables Outcrop streams into are the same governed platform your AI agents build in through the DataForge MCP server, under the same save-time validation. Replication, transformation, and agent development run under one validation layer.
Security and architecture
What your network team needs to know
These are architecture facts about how Outcrop connects and how it holds credentials. A security review can check each one against the product.
Network posture
- Outcrop initiates every connection, and commands arrive as responses to its heartbeat. The firewall requirement is outbound port 443, and nothing calls in.
- Outbound-only in-network agents are the converged pattern in this category, and several vendors publish their own version of it.
- Source connections default to a read-only intent, and datalake credentials follow a write-only pattern.
Credentials and configuration
- The agent's configuration file is encrypted and bound to the host machine GUID, so a copied file is useless anywhere else.
- Credentials are held encrypted server-side and fetched per use rather than stored in the clear on the host.
- The Databricks warehouse connection supports OAuth machine-to-machine authentication.
A single lightweight jar
In-network data planes in this category are typically containerized platform deployments or multi-service runtimes: a Kubernetes cluster to operate, or a set of services to size, patch, and monitor. Outcrop is one jar running as a Windows service.
1
jar
One process, installed by a standard Windows installer or run as a managed container.
443
outbound port
That is the whole firewall conversation, and there are no inbound rules to write or review.
4
ingestion modes
Files, batch database extraction, licensed ERP plugins, and streaming SQL Server replication, through the same agent.
Pricing
Replication is included in the platform
Outcrop ships with DataForge, and replication is priced with the platform rather than as a second product with its own meter. DataForge does not meter rows replicated.
Fivetran meters monthly active rows and counts deletes as billed rows. HVR 5.7 reaches end of life on December 31, 2026, which puts a hard date on a migration decision for teams running it against on-premise sources.
Other vendors price on capacity, compute, or volume rather than rows.
See DataForge pricing →What Outcrop handles on its own
Replication earns its reputation for maintenance in the failure cases: an expired retention window, a column added upstream, a warehouse sized for a load that finished hours ago. Each of those has a mechanism behind it.
Auto-update with the platform
A heartbeat flag triggers an atomic jar replacement and a self-restart. Agents inside your network track platform releases without a ticket to your IT team.
Automatic recovery from retention expiry
If the CDC log retention window or the Change Tracking retention window expires, the stream reinitializes itself.
Automatic schema-change handling
Schema changes on the source are applied under distributed locks, and a column added upstream is handled automatically, with no one paged.
One scheduling knob
Outcrop runs a self-tuning stream scheduler with a single strategy choice: Latency, Cost, or Balanced. It also applies backpressure against batch workloads, so streams and scheduled loads are not competing for the same agent capacity.
Reach the data behind the firewall
If you have a SQL Server behind a firewall and a Databricks account, the path is short: install Outcrop as a Windows service, open outbound 443, point it at your database, and pick a scheduling strategy.
Direct answers about Outcrop
What is Outcrop?
Outcrop, the DataForge agent that runs inside your network, is the component that reaches the data behind your firewall. It is a single lightweight jar that installs as a Windows service inside your network, or runs as a managed container in cloud deployments. One agent covers file ingestion, batch database extraction, and streaming SQL Server replication. Outcrop is a named component of the DataForge platform, not a separately sold product.
Why does the DataForge interface say Agent?
Outcrop is the name of the component, and it is shown as Agent in the DataForge UI today. The interface and the documentation are catching up to the name. Wherever DataForge says Agent, that is Outcrop.
Which databases can Outcrop stream changes from?
Streaming replication covers SQL Server sources, using log-based CDC and Change Tracking. Other database engines are ingested through batch JDBC extraction.
Does streaming replication work with Snowflake?
No. The streaming target is Databricks, where merges are applied over a Databricks Serverless SQL Warehouse. Snowflake deployments use batch ingestion.
How quickly do changes arrive?
Within seconds, for streaming replication. Outcrop merges changes continuously after the initial full load.
What does Outcrop need from our firewall?
Outbound port 443. Outcrop initiates every connection and commands arrive as responses to its heartbeat, so there are no inbound firewall rules to open or review. Its configuration file is encrypted and bound to the host machine.
What can Outcrop ingest besides databases?
Files from local and UNC network shares, SFTP, Amazon S3, and Azure Blob Storage, including file masks and multipart files. Licensed plugins cover SAP ECC, NetSuite, and QuickBooks. Some DataForge connections, including Salesforce, Kafka, and Databricks Unity Catalog, run directly from platform compute and do not use Outcrop at all.
Is Outcrop priced separately?
No. Outcrop is a component of the DataForge platform rather than a separately sold product, and DataForge replication has no per-row fees.
How is Outcrop kept up to date?
Outcrop updates itself with the platform. When a release ships, a heartbeat flag triggers an atomic jar replacement and a self-restart, so agents inside your network track platform releases without a ticket to your IT team.
Solution guides
Evaluate DataForge by platform goal
Enterprise data platform
Enterprise data platform for governed analytics at scale
DataForge helps CDOs, CFOs, and data platform leaders scale analytics without assembling separate ETL, orchestration, observability, lineage, and cost-control tools.
Data pipeline platform
Data pipeline platform for complex enterprise source systems
DataForge helps data teams build, extend, orchestrate, and observe enterprise data pipelines while preserving a consistent architecture across every source and output.
Data engineering platform
Data engineering platform with architecture built in
DataForge is a data engineering platform with automatically enforced architecture, replacing separate ETL, orchestration, observability, catalog, and infrastructure tools while processing data in your cloud.
Data orchestration platform
Data orchestration platform without manually assembled DAG sprawl
DataForge orchestrates data pipelines from structured pipeline definitions, dependency metadata, scheduling, and execution history instead of manually maintained DAGs.
Data observability platform
Data observability platform with lineage, quality, audit, and cost context
DataForge observability ties code, orchestration, quality rules, alerts, lineage, audit trails, and cloud cost visibility back to the platform metadata.
Agentic data engineering
Agentic data engineering platform with save-time validation for AI agents
DataForge lets AI coding agents such as Claude Code and Codex build and operate governed data pipelines through an MCP server that validates every write at save.
Data replication
Data replication that ends inside the governed platform
DataForge replicates SQL Server change data through Outcrop, the DataForge agent that runs inside your network, and merges it directly into governed tables that the transformation layer already consumes.