Introducing Outcrop: The DataForge Agent That Runs Inside Your Network

Getting data out of old systems, especially on-premise enterprise resource planning (ERP) systems that have not been updated in over twenty years, is still one of the most time-consuming and human-oriented tasks in data engineering. Some of the most valuable data in a company is stored on a SQL Server that is never allowed to face the internet: the ERP that runs the plant, or the order system that predates the cloud. The security team locked it down for good reasons.

Databricks cannot reach that server natively, and neither can a cloud pipeline. Something has to run inside the network.

So the usual answer is to buy a replication product, take on a second vendor with its own pricing meter, and have it drop the changes somewhere your transformation tool can pick up later.

That was our answer too for a while, and we did not enjoy giving it. We sell a platform that is supposed to take you from the source system to the finished model, and for on-premise SQL Server the story started with somebody else’s product.

Today we are announcing Outcrop, the DataForge agent that runs inside your network, and with it a capability much of the market does not know DataForge has: ingestion and replication, built into the platform. One lightweight agent ingests files, extracts from databases and ERP systems on a schedule, and streams SQL Server changes into governed tables. This post explains what Outcrop covers, how the streaming path works, and why the destination matters as much as the source.

Outcrop, the DataForge agent that runs inside your network

Outcrop is a lightweight remote agent that reaches the data behind your firewall. We are putting a name on functionality that already exists: you will see it labeled simply as the Agent in the DataForge interface today. It is a component of the platform, not a separate product you buy.

Outcrop is deliberately small, and setup is minimal. It is a lightweight application with a normal Windows installer, and it also runs as a Docker container; almost every team just installs it on a Windows virtual machine inside the network. There is no container platform to deploy, no Kubernetes cluster to operate, and no multi-service runtime to babysit. Other vendors run agents inside the network as well; theirs are typically containerized platform deployments, and ours is one application on one VM.

Its security posture is strict and simple to explain to a network team:

• Outbound only. Outcrop initiates every connection, and commands arrive as responses to its heartbeat. The firewall requirement is outbound port 443. Nothing calls in, ever, so there are no inbound rules to review.
• Encrypted, machine-bound configuration. The agent’s config file is encrypted and bound to the host machine, so it is useless if copied elsewhere.
• Fleet auto-update. When the platform releases, Outcrop installs the new version and restarts itself. Your on-premise agents track platform releases without a ticket to your IT team.

One agent, every on-premise source

Four ingestion modes run through the same agent, with the same install, the same credential handling, and the same scheduling:

• Files. Outcrop watches local and UNC network shares, SFTP, Amazon S3, and Azure Blob Storage, with file masks and multipart files.
• Databases, batch. Outcrop runs scheduled JDBC extraction from roughly ten engines, including SQL Server, Oracle, PostgreSQL, MySQL, Redshift, Snowflake, and SAP HANA.
• ERP systems. Licensed plugins cover SAP ECC, NetSuite, and QuickBooks.
• Streaming replication. Log-based capture streams SQL Server changes into governed tables within seconds. This is the newest mode, and the next section walks through it.

Streaming replication: log-based CDC and Change Tracking

This is the part of 10.3 that brings Outcrop up to par with dedicated replication products where it counts: streaming, log-based replication from SQL Server, with both CDC (change data capture) and Change Tracking setups.

Outcrop streams SQL Server changes using log-based CDC. It polls the source’s CDC functions, auto-detects key columns and refresh behavior from the CDC configuration, performs a full initial load, and then continuously merges inserts, updates, and physical deletes into the destination table. Changes arrive within seconds. If the log retention window expires, it recovers on its own. If somebody adds a column upstream, the schema change is handled automatically under distributed locks.

10.3 also adds a second tracking method, SQL Server Change Tracking, and it is the default. Change Tracking asks much less of your database administrators: it requires only a primary key and snapshot isolation, rather than full CDC enablement on the source. If the retention window expires, the stream reinitializes itself. For teams that could never get CDC turned on in a locked-down production database, this is the difference between a replication project and a configuration screen. The warehouse connection also now supports Databricks OAuth machine-to-machine authentication.

Two limits apply today. Streaming replication covers SQL Server sources, and the streaming target is Databricks. Tables with keys get row-level replication; keyless tables fall back to full reloads. Replication specialists cover Oracle, SAP, and mainframes, and we do not match their connector count. Where DataForge differs is what receives the data on the other end, and that is the next section.

The destination matters as much as the source

A standalone replication product ends at a landing zone. The vendor’s pipeline writes raw change data into a staging area, and then your transformation tool takes over: another schema, another hand-off, and another place where ownership and freshness blur. The replication layer and the transformation layer meet at that staging area and nowhere else.

DataForge’s replication ends inside your lakehouse, not beside it. Outcrop’s merges are applied directly into governed hub tables in your own Databricks account, the same tables the DataForge transformation layer consumes. There is no landing zone and no staging hand-off inside the product. The moment a change is merged, it is in a governed table that your rules, your lineage, and your downstream outputs already know about.

The plumbing patterns of modern replication, an outbound-only agent and log-based capture, have become the industry standard, and we are glad they have. What has not become standard is what picks the data up when it gets there: a column-level declarative rule system in the same product, consuming the same governed tables the merges are applied to.

A scheduler you set with one knob

Streaming replication usually comes with a tuning project attached: polling intervals, batch sizes, warehouse sizing, and the standing question of whether you are paying for idle compute. We replaced that with a self-tuning stream scheduler and a single strategy choice: Latency, Cost, or Balanced.

The scheduler also applies backpressure against batch workloads so your streams and your scheduled loads are not fighting each other for the same warehouse. You state your intent once, and the platform does the tuning continuously.

Pricing: no per-row fees

DataForge replication has no per-row fees. Rows replicated is not a number we meter, so a busy source system does not become a billing event.

That matters right now for two reasons. In March 2025, Fivetran changed how monthly active rows are calculated, a change that raised multi-connector bills by 40 to 70 percent for affected customers and counts deletes as billed rows. And HVR 5.7 reaches end of life on December 31, 2026, which puts a hard date on a migration decision for teams running it against on-premise sources.

Per-row metering is a real cost-predictability problem, and the industry has more than one answer to it: several vendors price on capacity, compute, or volume rather than rows, so no per-row fees is not unique to us. It is simply how DataForge replication is priced.

Replication and transformation in one platform

As of July 2026, based on public vendor documentation, DataForge is the only column-level declarative transformation platform with replication built in. A single outbound-only agent inside your network streams log-based SQL Server change data capture straight into governed tables in your own Databricks account, where DataForge rules pick it up. One product, one lightweight agent, no per-row fees.

In DataForge, every transformation rule writes exactly one column and is validated at save time in one transactional metastore; that is what column-level declarative transformation means. Other platforms with replication attached transform at the level of visual flows, mappings, or table-level pipeline definitions. Your replicated data flows into that column-level rule system, and it is the architecture the rest of the platform, including our AI tooling, is built on.

That AI tooling is part of the same release: 10.3 also ships the DataForge MCP server. The governed tables Outcrop streams into are inside the same governed platform your AI agents build in, under the same save-time validation. That platform is Talos, the AI control plane your agents enter through the MCP server. Replication, transformation, and agent development run under one validation layer in one platform.

Getting started

If you have a SQL Server behind a firewall and a Databricks account, the path is short: run the Outcrop installer on a Windows VM, open outbound 443, point it at your database, and pick a scheduling strategy. Change Tracking mode needs only a primary key and snapshot isolation to begin, and the first full load flows into governed tables your transformations can consume as soon as it completes.

You can read more about the platform architecture on our platform page, see what your agents can do with the MCP server, or start a trial and watch the first changes arrive within seconds. If you are staring down a per-row bill or an end-of-life replication product, we would be glad to talk through the migration.

Moving your own data out of your own database was never supposed to be a product you buy. With Outcrop it is the first ten minutes of building a model.