Building Snappy

Problem: You shipped 50,000 rows to the browser and now you're sorting and filtering in JavaScript. Each operation takes 200ms+ and blocks the UI.

Search-as-you-type Architecture

The pattern: debounce user input → send query to server → server does the heavy lifting (SQL, full-text search) → return minimal JSON → render results.

// Debounced search — the right pattern for large datasets
function SearchInput({ onResults }) {
  const [query, setQuery] = useState('');
  const controller = useRef(null);

  useEffect(() => {
    if (!query || query.length < 2) return;

    // Cancel previous in-flight request
    controller.current?.abort();
    controller.current = new AbortController();

    const timer = setTimeout(async () => {
      try {
        const res = await fetch(
          \`/api/search?q=\${encodeURIComponent(query)}&limit=20\`,
          { signal: controller.current.signal }
        );
        const data = await res.json();
        onResults(data.results);
      } catch (e) {
        if (e.name !== 'AbortError') throw e;
      }
    }, 150); // 150ms debounce — sweet spot

    return () => clearTimeout(timer);
  }, [query]);

  return <input value={query} onChange={e => setQuery(e.target.value)} />;
}

DuckDB — The Secret Weapon

DuckDB is an embeddable analytical database that runs sub-100ms queries on millions of rows. It can run in-process (no separate server), supports SQL, and even compiles to WASM for the browser. For dashboards and analytical tools, it's transformative.

• Pagination — simpler, better for SEO, more predictable memory usage
• Infinite scroll — better UX for feeds and exploration, but harder to implement correctly (memory management, scroll position restoration)

Problem: You're parsing a 5MB FHIR bundle or running a complex data transform. The entire UI freezes for 800ms while the main thread grinds through it.

What to offload to workers

• FHIR bundle parsing (1,180 patients? That's worker territory)
• Heavy data transforms, aggregations, grouping
• Search indexing (build a Fuse.js index off-thread)
• CSV/Excel file parsing
• Image processing or compression

// worker.js — runs on a separate thread
self.onmessage = function(e) {
  const { type, payload } = e.data;

  if (type === 'PARSE_FHIR_BUNDLE') {
    // Heavy work happens here — main thread stays responsive
    const patients = payload.entry
      .filter(e => e.resource.resourceType === 'Patient')
      .map(e => ({
        id: e.resource.id,
        name: e.resource.name?.[0]?.text || 'Unknown',
        birthDate: e.resource.birthDate,
        conditions: extractConditions(e.resource),
      }));

    self.postMessage({ type: 'PARSED', patients });
  }
};

// main.js — stays snappy
const worker = new Worker('worker.js');

worker.postMessage({
  type: 'PARSE_FHIR_BUNDLE',
  payload: hugeFhirBundle  // 5MB of FHIR data
});

worker.onmessage = (e) => {
  if (e.data.type === 'PARSED') {
    renderPatientList(e.data.patients); // UI updates instantly
  }
};

Problem: You're rendering 2,000 provider locations as <div> markers in the DOM. Pan and zoom are at 8fps. The page uses 400MB of RAM.

Clustering with Supercluster

Never render individual pins at wide zoom levels. Supercluster groups nearby points into clusters that expand as the user zooms in. This keeps the rendered element count constant regardless of dataset size.

// Mapbox GL + Supercluster — minimal clustering example
import mapboxgl from 'mapbox-gl';

const map = new mapboxgl.Map({
  container: 'map',
  style: 'mapbox://styles/mapbox/light-v11',
  center: [-98.5, 39.8],
  zoom: 4,
});

map.on('load', () => {
  map.addSource('providers', {
    type: 'geojson',
    data: providersGeoJSON,  // 10,000+ points
    cluster: true,
    clusterMaxZoom: 14,
    clusterRadius: 50,
  });

  // Cluster circles
  map.addLayer({
    id: 'clusters',
    type: 'circle',
    source: 'providers',
    filter: ['has', 'point_count'],
    paint: {
      'circle-color': ['step', ['get', 'point_count'],
        '#51bbd6', 100, '#f1f075', 750, '#f28cb1'],
      'circle-radius': ['step', ['get', 'point_count'],
        20, 100, 30, 750, 40],
    },
  });

  // Individual points (only visible at high zoom)
  map.addLayer({
    id: 'points',
    type: 'circle',
    source: 'providers',
    filter: ['!', ['has', 'point_count']],
    paint: { 'circle-radius': 6, 'circle-color': '#3b82f6' },
  });
});

The Stack

• Mapbox GL JS / MapLibre GL — WebGL-based maps, handles millions of features
• Deck.gl — data visualization layers on top of maps: heatmaps, hexbins, scatter plots, arc layers — all GPU-accelerated, millions of points at 60fps
• Supercluster — fast geospatial point clustering, works with any map library