This post is a retrospective on work I did with a small team on a healthcare analytics platform, written as much for my own understanding as for anyone else. Details are generalized, and the code samples are simplified reconstructions written for this post, not production code. It assumes familiarity with React, client-side data fetching, and REST pagination.

How Claude was used in writing this post

Short version: the engineering is mine, and so is the first draft. Claude helps me get it published.

  • Engineering and source material. All work described here was done by me. The background and technical detail come from reviewing my own PRs, their summaries, and our post-mortems.
  • Research. Claude helped gather outside documentation and sources. I read each one myself to judge relevance and applicability before it informed the post.
  • Drafting. With the material gathered, I sat down and wrote the rough draft manually in Neovim.
  • Editing. Claude then assisted with edits, organization, and structure.

Like most of us I don't have much spare time, and these write-ups happen on weekends. Claude is what lets me actually finish them instead of leaving them half-written. They exist primarily for my own learning and growth as an engineer.

1. The problem

The platform serves clinical users (care managers, physicians, nurses) who work against a roster of patients attributed to their care organization. Rosters range from tens of thousands of rows to several hundred thousand, and each row carries well over a hundred fields.

There were two distinct user stories to serve. Some users need to get in, find one patient, and get out in under a minute. Others live in the product all day and use it as their primary work surface. Both groups are medical professionals, mostly non-technical, already stretched thin, and any friction in the workflow contributes directly to the burnout problem the product is supposed to help with.

The interaction that mattered most was search. A user types a name and expects the row to appear as they type, the way it does in a spreadsheet.

2. Why not just paginate?

Server-side pagination was the default answer, and it was the backend team's initial instinct. It is the simplest solution to "large dataset, small payloads," and it is the right default for most applications. We rejected it for this product, but it is worth being honest about the reasoning and the alternatives.

The core problem is what pagination does to search and filtering semantics. If the client holds 5,000 of 100,000 rows, every search has to round-trip to the server. The feedback loop becomes: type, submit, spinner, results, and then a reset back to page one when the search is cleared. Each interaction costs a network round trip instead of a keystroke.

Worse than slow feedback is misleading feedback. If a user filters or sorts the subset that happens to be loaded, they get an answer that looks complete and is silently wrong. For clinical workflows, a confidently wrong "no results" is worse than a slow correct one.

Pagination also degrades a cluster of smaller interactions that users read as product quality: column sorting resets scroll position, multi-select across pages needs bookkeeping, exports need a separate server path, and "how many match this filter" becomes an API feature instead of a property the client already knows.

Alternatives we considered

To be fair to pagination, there were ways to make it feel better, and they are the right call in other contexts:

We chose the contrarian option: get the entire dataset into the browser and make that fast enough to be acceptable. The rest of this post is about what "fast enough" took.

3. The building blocks

Two library choices made the client-side approach plausible at all.

A virtualized data grid (we used MUI X Data Grid Pro, but AG Grid and TanStack Table offer the same capability). Virtualization means the grid only renders the rows currently visible in the viewport, plus a small buffer, and recycles DOM nodes as the user scrolls. 100,000 rows in memory is fine; 100,000 rows in the DOM is not. With virtualization the DOM holds maybe 20-40 row elements regardless of dataset size, so scroll performance is decoupled from row count. The MUI virtualization docs cover the mechanics. This is the load-bearing decision: without row virtualization, nothing downstream matters.

TanStack Query for fetching and caching. It is the de facto standard for server state in React, and the specific features that mattered were client-side caching keyed by query identity and fine-grained cache manipulation via setQueryData. That second one becomes important later, both as the mechanism that made things work and as the thing that caused our best bug.

One constraint worth mentioning: the backend API stayed conventionally paginated (limit/offset). The API is a product surface consumed by third parties, and they get standard REST semantics. All of the "load everything" behavior lives in the client and a thin server-side transformation layer. I think this was the right separation: the aggressive client behavior is an implementation detail of one consumer, not a property of the API.

4. Version 1: measure first

The naive version, one request for ~90,000 rows, failed in the obvious ways: a very long wait with no progress feedback and a payload in the tens of megabytes with all-or-nothing failure semantics.

So v1 fetched sequentially in 4,000-row chunks through TanStack Query's useInfiniteQuery, unwrapping each page into the grid as it arrived. Before optimizing anything, we profiled a single chunk request end to end:

Layer Operation Avg time
Backend (Django) database query execution ~10ms
Backend (Django) pagination ~480ms
Backend (Django) DRF serialization ~1,350ms
Server middle layer total ~1,977ms
Client total per 4k-row request ~2,005ms

The surprise: the database was essentially free at ~10ms. Roughly 75% of backend time was Django REST Framework serialization, turning model instances into JSON. This matches what others have documented about DRF serializer overhead on large querysets (Haki Benita's "Why is Django REST Framework so slow?" is the canonical writeup). If we had guessed instead of measured, we would have optimized the query layer and gained nothing.

I also built a small production-safe testing harness that let us vary the batch size at runtime and profile total load time and per-request time. Batch size tuning is a real tradeoff curve: bigger batches amortize per-request overhead (auth, serialization setup, HTTP overhead) but increase time-to-first-feedback and make each failure more expensive. We landed on ~5,000 rows per request, roughly 5MB per payload.

v1 shipped with a total cold load around 1-3 minutes depending on roster size, but with progress feedback and instant revisits within a session thanks to the client cache. Usable, not good.

5. The memory question

The obvious objection to all of this: what does holding the full dataset do to browser memory?

Fully loaded, the tab's footprint measured around 150MB. To decide whether that was acceptable, we looked at what comparable applications already demand: browser-based EMRs and CRMs that our users run all day commonly sit in the hundreds of megabytes per tab. Our users' hardware floor was hospital workstations, which are not powerful, but which already run these heavier applications.

I want to be careful not to overstate this argument. "Other apps are worse" is a justification with a ceiling, and the ceiling is real: at 500,000+ rows with hundreds of fields each, you are staring at potentially 500MB+ of JS heap, and somewhere in that range the approach stops working, especially on low-end hardware. Honest mitigations if we hit that wall, roughly in order of preference:

  1. Trim fields: the grid does not need every one of 100+ keys; a column-driven field selection could cut payload and heap substantially.
  2. Time-window the data where the domain allows it (event-style data has a natural time axis; a roster does not).
  3. Concede and move to a server-side row model for the largest customers.

We accepted the tradeoff knowingly for datasets in the ~100k range, and it has held up in production, but I would describe this as a decision with a known expiration condition rather than a solved problem.

6. Version 2: interactive first, complete later

v1's remaining problem was that "usable" only began when loading finished. v2 restructured loading around a principle I would now reach for by default: make the app interactive on the first response and treat completeness as a background concern.

The sequence:

  1. Fetch the first 100 rows and render immediately. The grid is interactive within a couple of seconds, and the response metadata (total count) tells us exactly how many background batches remain.
  2. Fetch the remaining batches in parallel rather than as a dependent sequential chain.
  3. Merge completed batches into grid state without disturbing whatever the user is doing: active filters, sorts, and selections must survive each merge.
  4. Show explicit progress ("12,000 of 96,000 loaded") so users know search results may be incomplete while loading.
  5. Track failed batches individually and surface a retry action. Never silently drop a batch: a partial dataset that admits it is partial is fine, a partial dataset pretending to be complete is the misleading-results problem all over again.

Point 4 deserves emphasis because it is the cheap fix for the correctness problem that made us reject pagination. During the loading window, client-side search has the same "silently incomplete results" hazard as a paginated grid. The progress indicator converts a correctness bug into a communicated limitation, which for a 15-second window is an acceptable contract.

The cache-shape decision

There is a subtlety in step 3 that shaped everything after it. Users navigate away and back constantly, and nobody will wait through a reload because they clicked into a detail page. TanStack Query's cache solves this, but caching the paginated response shape would mean re-running the unwrap-and-merge processing on every mount.

So the loader normalizes as it goes: batches are merged into a single flat cache entry, an array of rows under one query key, rather than being left as pages. Once loading completes, any component can consume the full dataset as a plain array with zero processing, and navigation back to the grid is instant.

Simplified to its skeleton, the whole v2 loader is about this much code (the production version adds retry policies, progress events, and abort handling): 

// TanStack Query v5. `fetchRows` POSTs { offset, limit } and returns
// { rows, totalCount }.
const BATCH_SIZE = 5_000;

async function loadRoster(queryClient: QueryClient, cacheKey: QueryKey) {
  // First 100 rows: the grid renders now, and totalCount tells us
  // exactly how many background batches remain.
  const first = await fetchRows({ offset: 0, limit: 100 });
  queryClient.setQueryData<Row[]>(cacheKey, first.rows);

  const offsets: number[] = [];
  for (let o = 100; o < first.totalCount; o += BATCH_SIZE) offsets.push(o);

  // Parallel batches, each merged into the single flat cache entry
  // as it lands. Failures are recorded per batch, never swallowed.
  const failedOffsets: number[] = [];
  await Promise.all(
    offsets.map(async (offset) => {
      try {
        const batch = await fetchRows({ offset, limit: BATCH_SIZE });
        queryClient.setQueryData<Row[]>(cacheKey, (rows = []) => [
          ...rows,
          ...batch.rows,
        ]);
      } catch {
        failedOffsets.push(offset);
      }
    })
  );

  return { failedOffsets }; // drives the user-facing retry action
}

Note that the merge is a functional update over whatever is currently in the cache. That detail is about to matter.

One wrinkle: because rows contain protected health information, request parameters travel in POST bodies rather than URL query strings. A common misconception (one I held) is that this is about encryption; it is not, since TLS encrypts query strings and bodies equally in transit. The actual reason is that URLs persist in places bodies do not: server access logs, browser history, referrer headers, and any analytics tooling that records page URLs (OWASP calls this out directly). The practical cost is that a lot of HTTP-layer and framework tooling assumes GET for reads: HTTP-level caching keys on URLs, and framework data caches are generally built around GET semantics (Next.js, for example, does not cache POST route handler responses). TanStack Query itself is method-agnostic since cache identity comes from the query key you construct, but we had to be deliberate about deriving keys from the request body instead of getting URL-based identity for free.

With v2, the grid was interactive in a couple of seconds, but a cache-cold full load still took 2-3 minutes on large rosters. The client had stopped being the bottleneck; each request was still paying that ~1.4s serialization cost on the backend.

7. The race condition

Before the backend work, the bug. This is the part of the project I would keep even if I had to cut everything else, because it is a clean example of a failure mode that emerges when two individually-correct patterns compose.

The platform has a tagging feature: users apply tags to rows, and tag mutations use optimistic updates so the UI responds instantly. The optimistic update writes the new tag directly into the TanStack Query cache entry for the grid data via onMutate, then the server request confirms it.

Separately, the progressive loader was also writing to that same cache entry, merging in each background batch as it completed.

Each pattern was correct in isolation. Composed, they raced: a user applies a tag while batches are still streaming in, the optimistic update lands in the cache, and then a later merge step replaces the list state and silently wipes the tag. From the user's perspective, tags applied during the loading window just disappeared.

The collision, reduced to its shape:

// Writer 1 - tag mutation, the documented optimistic-update pattern:
onMutate: ({ memberId, tag }) => {
  queryClient.setQueryData<Row[]>(cacheKey, (rows = []) =>
    rows.map((row) =>
      row.id === memberId ? { ...row, tags: [...row.tags, tag] } : row
    )
  );
};

// Writer 2 - leftover v1 merge logic that still fired when the load
// completed, rebuilding list state from the raw fetched pages:
queryClient.setQueryData<Row[]>(cacheKey, () =>
  pages.flatMap((page) => page.rows)
);
// The fetched rows predate the tag. The rebuild makes the fetch
// response the source of truth, and the optimistic tag is gone.

What makes this bug instructive:

The fix was to make the two writers aware of each other: the defunct v1 rebuild logic was stripped out, tag mutations became aware of the loader's cache structure, and every merge treats the current cache entry as the source of truth for rows the user has already touched: 

// One merge discipline for every writer: append rows that are new,
// never rebuild rows that are already present.
queryClient.setQueryData<Row[]>(cacheKey, (rows = []) => {
  const known = new Set(rows.map((row) => row.id));
  return [...rows, ...batch.rows.filter((row) => !known.has(row.id))];
});

The general lesson I took: when multiple actors write to a shared cache, every writer needs to know the cache's full lifecycle, or writes need to go through a single merge point. "Last write wins" is a fine default until two writers have different ideas about what the whole value is.

Sidebar: the rollback memory tradeoff

The optimistic update implementation had its own size-driven tradeoff worth recording. The textbook TanStack rollback pattern is snapshot-and-restore: copy the cache entry in onMutate, restore it in onError. With a multi-megabyte cache entry, snapshotting means briefly doubling memory for every mutation. We instead mutated the cache directly and stored only the reverse diff needed to undo the specific change. More code and more care in exchange for not paying a full-copy tax on every tag click; at small data sizes I would take the snapshot pattern every time.

8. Owning the backend contract

The endgame was accepting that the frontend could not fix the remaining 2-3 minutes alone, because the cost was per-request backend serialization.

The serialized output for a given customer's roster changes on a data-pipeline cadence (hours), while it was being recomputed on every request. That is a caching problem, but caching the existing endpoints was awkward because they conflated two contracts: third-party consumers need live queries with full filter/sort/ pagination semantics, while the grid needs exactly one thing, the full dataset, as fast as possible. Every filter parameter a cached endpoint accepts multiplies cache keys and shrinks hit rates.

So the endpoints were split:

Two supporting pieces made the cached endpoints safe to rely on.

Scheduled cache warming. A teammate built a scheduled job (an AWS Lambda on a timer) that makes real requests against the cached endpoints on the same cadence as the TTL, with the TTL set slightly longer than the warming interval so the cache never goes cold in practice. Without warming, one unlucky user per expiry window pays the full multi-minute cost; with it, effectively nobody does. The honest caveat: cache warming is a scheduled process that can itself fail, so the system needs the slow path to still work and ideally alerting on warmer failures.

A near-miss worth recording. While wiring the new cached route into the backend, an existing path-matching predicate guarding a data-correctness filter almost excluded the new path silently: 

# Guard that restricts member endpoints to currently-active members:
if request.path.endswith("/members/"):
    queryset = filter_currently_attributed(queryset)

# The new cached route is "/members/cached/", which does not end with
# "/members/". The guard silently skips it, and the cached endpoint
# returns inactive patients. No error, no failing request, just wrong data.

It was caught in self-review of the change. The takeaway is about string-matching on URL paths as a routing-adjacent mechanism: suffix and substring predicates encode assumptions that new routes will violate silently. Explicit route sets, or at minimum tests that enumerate which paths a predicate matches, are cheap insurance.

The frontend then routed the two consumers to their two contracts: the grid reads the cached endpoint (through a thin server-side proxy that enriches responses with user-generated data like tags, which changes far more often than the roster), while an interactive patient-search dialog elsewhere in the app kept using the live endpoint because it genuinely needs fresh, filtered queries. One dataset, two access patterns, two endpoints; conflating them had been the original mistake.

9. Results

Metric Before After
Cold full load (~100k rows) 2-3 minutes under 15 seconds
Time to interactive grid end of load ~2 seconds
Revisit within a session full reload instant (client cache)
Search/filter/sort latency n/a (incomplete data) keystroke-speed, full dataset

The stack that produces the 15 seconds: server-side response caching with scheduled warming (eliminates per-request serialization), parallel ~5MB batches (fills the pipe on cache-cold loads), and client-side normalized caching (eliminates repeat loads entirely). Each layer covers a different failure of the layer below it.

The system has now been in production for over two years across customers with very different roster sizes, which I take as reasonable evidence the tradeoffs were priced correctly for this domain, though not as evidence they generalize.

10. When to copy this, and when to just paginate

This pattern earned its complexity under a specific set of conditions, and I would not reach for it without most of them:

And the costs, plainly: you take on distributed-systems-flavored problems inside a browser tab (the race condition), a memory budget with a real ceiling, a second set of backend endpoints to maintain, and a cache-warming process that must be monitored. Pagination has none of those problems. It just also does not deliver spreadsheet-speed search over a complete dataset, and that was the product.

Further Reading

Library versions and behaviors described are as of early 2026; framework caching semantics and grid APIs change, so verify against your own stack.