2025 — My Year in Data
A retrospective on building a full Data Mesh environment for my team at MercadoLibre, migrating 6+ pipelines to production, and planting the seeds of a data-driven culture initiative.
Where It Started
At the beginning of 2025, I inherited something both exciting and humbling: a Data Mesh Environment (DME) with potential but almost no data products. The DME — a governed BigQuery environment — existed on paper. It had the right architecture, the right governance model, the right tooling. What it didn’t have was data.
203 production tables, 162 published views, and 93 active scheduled jobs by year’s end. None of those existed in January.
This is a post about what it took to get there, what I learned along the way, and what I would do differently.
The Infrastructure Year
If I had to describe 2025 in a single word, it would be foundation. The year was almost entirely about building the plumbing before anyone could turn on the faucet.
The core work was migrating six data domains from the sandbox — a collection of unmonitored stored procedures and undocumented BigQuery tables — to the Data Mesh. Each migration followed the same architecture: Bronze (raw extraction) → Silver (typed, validated, business-enriched) → Gold (aggregated data marts). Every layer uses a different BigQuery dataset, enforces audit fields, and runs on scheduled data-platform jobs with automated alerting.
flowchart LR SRC[Operational sources] --> B[Bronze<br/>raw extraction] B --> S[Silver<br/>typed · validated · enriched] S --> G[Gold<br/>aggregated marts] G --> C[Consumers<br/>dashboards · self-service]
Here’s what shipped:
IT Asset Management — The Canonical Migration
The first major migration was our IT asset management system. Before the migration, the entire 192K+ device inventory lived in a single legacy stored procedure — unversioned, unmonitored, and brittle.
After the migration: 29 tables (10 Bronze, 14 Silver, 5 Gold), three platform jobs running twice a day, and a wide asset mart with 243+ columns that any team can query without asking me for a JOIN. The stored procedure is retired.
That migration became the template. The lesson: the hardest part of a migration is not the SQL — it’s building the case that the new system is better than the old one. For asset management, that case was clear (uptime, discoverability, self-service). For other domains, it required more patience.
A High-Stakes Findings Domain
One domain tracks operational findings, remediation timelines, and risk scores across the organization. Before the migration, the data was trapped in proprietary system exports, available to only one team.
After: 28 tables on a daily refresh, with a Gold mart that dashboards consume directly. It pre-calculates risk scores, aging buckets, and compliance rates — things that used to require a custom SQL request routed through the owning team.
A Freshness-Critical Domain
Another domain has a modest data volume but a strict freshness requirement: operational teams need it refreshed several times a day, not overnight.
This pipeline runs five times a day across 22 tables. Its Gold mart gives operations teams a consolidated, near-real-time view without querying the source system directly.
Ticketing, Employee Lifecycle, Licensing — The Long Tail
The remaining three migrations (ticket management, employee lifecycle, and compliance licensing) followed the same pattern but presented different challenges.
A ticketing domain had 30 source tables and 7 legacy stored procedures that other systems depended on. The Bronze layer migrated cleanly; the Silver and Gold layers required rewriting years of accumulated business logic.
A people-data domain involved sensitive fields from day one. We spent a full sprint designing the model so sensitive fields were excluded or masked before landing in Silver — and that masking framework became the reference for every later pipeline.
A compliance domain was the newest. Its data arrives via REST API rather than a database, so Bronze needed a custom Python extractor. By year’s end the orchestrator runs several times a day and feeds a Gold mart used across the organization.
Technical Highlight: The Compliance Pipeline
One pipeline deserves its own paragraph: the compliance tracking pipeline.
It is built on an internal tracking system whose data model is genuinely complex — 21 Bronze extractions feeding 16 Silver tables, with the output landing in four Gold marts that power a dashboard used across the organization.
The reason this pipeline stands out is the architecture decision: rather than building one massive reporting stored procedure, we separated concerns across layers. Bronze is dumb (raw extraction, no transformations). Silver is typed and business-enriched. Gold is aggregated for consumption. Each layer can be rebuilt independently. When a source schema changes, only Bronze needs updating.
This is textbook medallion architecture. It still took three months to get right.
The Governance Layer
Building pipelines is the visible part of data engineering. The invisible part is what makes them trustworthy.
Midway through the year, a governance audit revealed that 145 of our production tables were flagged as low-quality in our internal data-quality dashboard — meaning their descriptions were either missing or too generic to be useful. A table called FACT_DEVICE_INVENTORY with the description “device data” is technically documented. It is not actually useful.
We started a systematic enrichment project: generate business-context descriptions for every column across seven domains, apply them through the data platform, and track the quality score monthly. It’s unglamorous work. It’s also the difference between a data product and a data warehouse.
Our governance framework measures three things: Discoverability (can someone find and understand your data?), Efficiency (does it cost what it should to maintain?), and Integrity (is it accurate and reliable?). By year’s end, we had a baseline for each. The targets for 2026 are set.
Seeds Planted for 2026
Two initiatives started in late 2025 that will define 2026:
The data-literacy initiative — a program to grow data-tool adoption across the organization, with a measurable year-end target. The first leader mapping was done in December, and the first hands-on sessions are scheduled for Q1 2026. More on this in a separate post.
The datalake toolkit — A set of Claude skills I’m building to help the team validate jobs before promoting to production, scaffold new pipelines according to architecture standards, and troubleshoot failing jobs automatically. The goal is to embed architectural governance into the development workflow, not just the review process.
What I’d Do Differently
Communicate value earlier. I spent most of the year building pipelines and very little time explaining what those pipelines enabled. By the time I had 200+ tables in production, the value was obvious to me and invisible to almost everyone else. Architecture without narrative is infrastructure without users.
Define “done” before starting. Each pipeline took longer than expected not because of technical complexity, but because the definition of done kept expanding. A Bronze extraction became “also do the Silver.” A Silver migration became “also update the Gold mart.” Scope creep in data engineering looks exactly like good intentions.
Ship smaller. The temptation with a greenfield environment is to design the perfect system from the start. The better move is to get one domain to production, learn from it, then apply those lessons to the next one. We did this eventually. I wish we had done it from day one.
Closing
2025 was the year the DME went from potential to production. The numbers are real — 203 tables, 93 scheduled jobs, six domains migrated, and a governance framework in place. But numbers don’t capture what it actually means: the first time an analyst pulled the risk scores they needed without filing a request, or an operations team checking a near-real-time view during an audit without involving the source-system team.
Data infrastructure is most valuable when it’s invisible. When a team can answer their own questions without knowing the system’s name. When the pipeline runs on schedule and nobody notices because there was nothing to notice.
That’s what we built in 2025. Next year, we make it count.
Note: table and schema names in this post are illustrative and generalized — not real production identifiers. Metrics reflect real outcomes from my team’s work.