Databases and pipelines that stay fast.
The high-volume processing pipelines and clean architectures that fuel your entire tech ecosystem.
- A SQL Server or Azure SQL database is slow, deadlocking, or growing faster than its design.
- Orders or events get lost, duplicated, or stuck between services, and every fix is manual.
- A nightly job no longer finishes by morning.
- SQL Server 2016 is out of support and the audit noticed.
Why this keeps happening
Most databases are designed once, at a size they will never be again. The indexes match the queries of the first release. The purge job deletes row by row because that was fine at a million rows. Then the volume arrives, the hot table deadlocks every Tuesday, and the nightly job runs into the morning. Everyone knows which table it is; nobody has the time, or the evidence, to change it safely.
Pipelines fail the same way. An event is processed twice, a consumer falls behind, a replay floods a downstream service, and the symptom shows up as a stuck order and a support ticket three teams away.
Upgrades wait for the same reason. The last one took a weekend and broke two reports, so the next one is always next quarter, until the version leaves support and the risk becomes someone else’s finding.
How we work on it
Evidence before changes. We read the query plans, the deadlock graphs, and the consumer lag before touching anything, and we rank the fixes by payoff so the first change is the one that matters.
Load test the fix, not the hunch. Every index, partition, batch, or migration change is run at well beyond production volume before it ships, because a change that works at today’s size and fails at next year’s is not a fix.
Recovery by procedure. Replays, backfills, and rollbacks are written down and rehearsed, so your on-call team can run them without calling us.
What we do in Data Engineering
High-performance databasesMake the database that runs the business fast again, and keep it fast when the volume doubles.
Prior experience. The index and purge redesign on the order tables behind 4,700+ stores at Walmart Global Tech; the figures are on the card below.
Fits best in: retail and logistics, SaaS.
Unified data pipelinesKeep orders, inventory, and events moving between systems in real time, with no lost or duplicated records and a recovery your team can run.
Prior experience. Kafka event flows across five picking services for an order-fulfillment platform, with idempotent consumers and event replay so stuck orders recovered by procedure instead of by hand.
Fits best in: retail and logistics, SaaS.
Seamless data migrations over the cloudMove off an unsupported SQL Server version, or into Azure SQL, without a weekend outage or a slower database afterwards.
Method. A compatibility assessment, a timed cut-over rehearsed on a copy of production with a rollback per step, and performance baselined before and compared after.
Fits best in: retail and logistics, SaaS.
The work behind it
Prior experience: work by the people behind Data Fact Tree in previous roles.
Walmart Global Tech · 2022 to 2025
~1M rows per runDeadlocks ended and purges rebuilt on hot order tables
Composite and covering index redesign on the most-queried, deadlock-prone order tables, and a rewrite of the canceled-order purge to partition-aware batch deletes across 10 clusters.
Recurring deadlocks stopped; weekly purges handled about 1 million rows per run across 10 clusters and were stress-tested to 3 million.
Read the full story →How an engagement is shaped
We quote within 48 hours of a technical consultation.
Database health check
Findings with evidence, index and query plan, purge and partition strategy, a ranked fix list
Retained advisory
Design and code review, incident support, hiring help
If the same table deadlocks every week, send us the deadlock graph and we will tell you what we see. Get in touch.