Paginating Large IndexedDB Result Sets with Cursors

Calling store.getAll() on a table with fifty thousand rows works fine in development and then freezes the main thread and inflates memory in production. The whole result set is deserialized into a single array before your code runs, and on mobile that materialization can cost hundreds of milliseconds and tens of megabytes. The fix is to page through records with a cursor, fetching only the slice you render. This guide builds a reusable keyset-pagination helper in TypeScript. It is part of Query Optimization & Cursors within IndexedDB Architecture & Advanced Patterns.

Root cause: getAll materializes everything

IDBObjectStore.getAll() and IDBIndex.getAll() are defined to collect every matching record into one array and hand it to your success callback in a single step. The structured-clone deserialization of each value happens up front, so the cost scales with the total result size, not with what you actually display. On a list view that shows twenty rows at a time, fetching all fifty thousand is pure waste — and on a memory-constrained device it can trigger eviction of other data, a failure mode explored in Debugging Storage Eviction in Progressive Web Apps.

A cursor (IDBObjectStore.openCursor) is the opposite: it yields one record at a time and lets you stop whenever you have enough, deserializing only the rows you touch.

Step-by-step: a keyset getPage helper

The robust approach is keyset pagination (also called seek pagination): instead of asking for “the page starting at offset 2000,” you remember the last key you saw and ask for “the next twenty rows after this key.” IDBKeyRange.lowerBound(lastKey, true) builds exactly that range — the true makes the bound exclusive, so the row you already have is skipped.

interface Page<T> {
  rows: T[];
  // The key to pass as `afterKey` to fetch the following page.
  nextCursor: IDBValidKey | null;
}

function promisifyRequest<T>(req: IDBRequest<T>): Promise<T> {
  return new Promise((resolve, reject) => {
    req.onsuccess = () => resolve(req.result);
    req.onerror = () => reject(req.error);
  });
}

/**
 * Fetch one keyset page from an object store, ordered by primary key.
 * Pass `afterKey = null` for the first page, then feed back `nextCursor`.
 */
function getPage<T>(
  db: IDBDatabase,
  storeName: string,
  afterKey: IDBValidKey | null,
  pageSize: number,
): Promise<Page<T>> {
  return new Promise((resolve, reject) => {
    const tx = db.transaction(storeName, 'readonly');
    const store = tx.objectStore(storeName);

    // Exclusive lowerBound => start strictly after the last key we returned.
    const range =
      afterKey === null
        ? null
        : IDBKeyRange.lowerBound(afterKey, true);

    const rows: T[] = [];
    let lastKey: IDBValidKey | null = null;

    const cursorReq = store.openCursor(range, 'next');
    cursorReq.onsuccess = () => {
      const cursor = cursorReq.result;
      if (cursor && rows.length < pageSize) {
        rows.push(cursor.value as T);
        lastKey = cursor.key;
        cursor.continue(); // advance exactly one record
      } else {
        // Either we filled the page or ran out of records.
        const nextCursor = rows.length === pageSize ? lastKey : null;
        resolve({ rows, nextCursor });
      }
    };
    cursorReq.onerror = () => reject(cursorReq.error);
    tx.onerror = () => reject(tx.error);
  });
}

Driving it through an entire store is just a loop that feeds nextCursor back in:

async function forEachPage<T>(
  db: IDBDatabase,
  storeName: string,
  pageSize: number,
  onPage: (rows: T[]) => void,
): Promise<void> {
  let cursor: IDBValidKey | null = null;
  do {
    const page = await getPage<T>(db, storeName, cursor, pageSize);
    if (page.rows.length > 0) onPage(page.rows);
    cursor = page.nextCursor;
  } while (cursor !== null);
}

Paginating over an index, not the primary key

To order by something other than the primary key — say a createdAt field — open the cursor on an index instead. Pagination over an index requires care because index keys are not unique; the section on stable ordering below explains why you also need the primary key as a tiebreaker, which is exactly what Creating Compound Indexes for Multi-Field Filtering sets up.

function getPageByIndex<T>(
  db: IDBDatabase,
  storeName: string,
  indexName: string,
  afterKey: IDBValidKey | null,
  pageSize: number,
): Promise<Page<T>> {
  return new Promise((resolve, reject) => {
    const tx = db.transaction(storeName, 'readonly');
    const index = tx.objectStore(storeName).index(indexName);
    const range =
      afterKey === null ? null : IDBKeyRange.lowerBound(afterKey, true);

    const rows: T[] = [];
    let lastKey: IDBValidKey | null = null;
    const req = index.openCursor(range, 'next');
    req.onsuccess = () => {
      const cursor = req.result;
      if (cursor && rows.length < pageSize) {
        rows.push(cursor.value as T);
        lastKey = cursor.key; // the index key, used as the next bound
        cursor.continue();
      } else {
        resolve({ rows, nextCursor: rows.length === pageSize ? lastKey : null });
      }
    };
    req.onerror = () => reject(req.error);
  });
}

Why keyset beats advance(offset)

A cursor exposes cursor.advance(n) which skips n records, tempting you to emulate SQL OFFSET. Resist it. advance(2000) still walks the cursor over two thousand records internally — the cost grows linearly with the page number, so deep pages get progressively slower. Keyset pagination jumps straight to the right key via the B-tree range and stays O(pageSize) no matter how deep you are.

Offset paging also breaks under mutation. If a record before your current position is deleted between page loads, every later record shifts down by one, so advance(40) now skips a row you never showed the user. Keyset paging anchors on a concrete key, so insertions and deletions elsewhere never cause a row to be silently skipped or duplicated.

Verification

Assert that pages are the expected size, are contiguous, and never overlap:

async function verifyPagination(db: IDBDatabase): Promise<void> {
  const pageSize = 20;
  const seen = new Set<string>();
  let total = 0;

  await forEachPage<{ id: string }>(db, 'records', pageSize, (rows) => {
    console.assert(rows.length <= pageSize, 'page exceeded pageSize');
    for (const r of rows) {
      console.assert(!seen.has(r.id), `duplicate row across pages: ${r.id}`);
      seen.add(r.id);
    }
    total += rows.length;
  });

  console.assert(seen.size === total, 'overlap detected between pages');
  console.info(`Paged ${total} rows in chunks of ${pageSize}`);
}

To confirm the memory win, open DevTools → Memory and record a heap snapshot while running a getAll() load versus the paged loader over the same store. The getAll() path shows a large transient array of cloned objects; the paged path holds only one page at a time. The Performance panel will also show the long blocking task on the main thread for getAll() that the cursor approach eliminates.

Edge cases and a fallback

Stable ordering. Cursors iterate in key order, so ordering is deterministic only as long as the key you page on is unique. Paging on a non-unique index (many rows share a createdAt) can return the same row on two pages or skip one at a page boundary. The fix is a compound key [createdAt, id] so the primary key breaks ties — set up exactly as in the Indexing Strategies for Fast Queries guide.

Direction ‘prev’. To page backward, open the cursor with direction 'prev' and swap lowerBound for IDBKeyRange.upperBound(beforeKey, true). The rows come back in descending order, so reverse the array before rendering if your UI expects ascending order.

Fallback for small stores. Keyset paging only pays off above a few hundred rows. If a store is known to be small and bounded, getAll() with a count argument (store.getAll(null, 50)) is simpler and perfectly fine. Reserve cursor pagination for the unbounded, user-generated tables where the row count can surprise you.

Frequently Asked Questions

Related

• Query Optimization & Cursors — the parent guide on cursor-driven reads and range queries.
• Creating Compound Indexes for Multi-Field Filtering — build the [field, id] keys that make index paging stable.
• Indexing Strategies for Fast Queries — choosing the right key paths for the index you page over.
• IndexedDB Transaction Management — keeping per-page transactions short and valid.
• IndexedDB Architecture & Advanced Patterns — the broader data-layer design these patterns belong to.