Data Platform Modernization: A Reference Architecture We Actually Ship

Almost every architecture choice on a data platform is downstream of four decisions. If a leadership team gets these four right, the rest of the platform is mostly a procurement exercise. If any of these four are unresolved, no amount of tooling sophistication recovers the program.

For most mid-market clients in 2026, the architecture we converge on looks remarkably similar. Sources land into raw object storage (S3 / GCS / ADLS) via Fivetran, Airbyte, or hand-rolled connectors for the long tail. A lakehouse engine — Snowflake or Databricks, depending on the AI ambition — provides the compute. dbt owns the transformation layer end-to-end, with tests as a build gate, not an afterthought. A catalog (Unity, Atlan, or DataHub) is the system of record for definitions, lineage, and access. BI is split: governed dashboards in one tool, exploratory work in another, and the line between them is enforced by which models the BI tool can hit.

There is nothing exotic about this. The discipline is in what we say no to. We do not stand up a separate streaming platform until a real sub-second use case arrives. We do not pre-build a feature store before the second ML model. We do not buy reverse ETL until we have a customer for the data being reverse-ETL'd. Every component in the platform earns its way in by being on the critical path of a working use case.

The Snowflake-vs-Databricks decision is rarely about benchmarks. It is about the team and the workload mix. We pick Snowflake when the workload is dominated by SQL analytics, the team is SQL-fluent and Python-light, and the AI roadmap is mostly off-the-shelf or vendor-hosted. We pick Databricks when the workload meaningfully includes ML training, when the team has Python and Spark fluency, or when the company has a serious data science investment that needs to live next to the analytical data.

Both can do the other's job. Both have closed the feature gap by a wide margin. The decision is about which platform your team will operate well, sustainably, in year three — not which one has the better demo.

We have run the post-mortem on enough failed modernizations to see the same three failure modes:

A typical Polance data platform engagement runs in three phases. Phase one is two to four weeks of architecture design and decision capture — we land the four decisions above, get explicit sign-off from the operating leadership, and produce a one-page reference architecture the engineering team will live with. Phase two is the build, typically twelve to twenty weeks, with our senior engineers on the keyboard alongside the client team — we do not write code that only we can maintain. Phase three is a deliberate handover and stabilization period, where we step out of the critical path and the client team runs the platform under our review for a final four to eight weeks.

We are deliberately on the work, not above it. Architecture decisions made by people who are not also implementing them tend to be elegant in the deck and unworkable in the repo.

FinOps for data platforms is mostly common sense applied consistently. The patterns that move the needle: warehouse sizing tied to actual query patterns, not vendor-suggested defaults; auto-suspend everywhere; cost per use case as a first-class observability metric; and a quarterly architecture review that is allowed to retire components that did not earn their cost. None of this is sophisticated. All of it requires somebody senior to care about it for the duration of the engagement.