Bulk reload is an orchestration problem, not a bigger VM wish
InsertAll of multi-million ledger rows on one driver is a known failure mode. The notebook looks simple. The driver dies. Someone asks for a bigger cluster. The next reload dies later, with a larger bill.
Treat bulk reload as orchestration: batched parallel page ingest, progress monitoring, and entity filters for operable reruns. Separate one-time rebuild paths from incremental watermarks. Publish the pattern to prod only after batch-level observability exists. OOM here is a design smell in notebook API usage, not proof that "Fabric is flaky."
Figure 1. Multi-million InsertAll on one driver is an orchestration smell. Batch ingest belongs in an operable pipeline. Source: Microsoft Learn: copy data activity.
Figure 2. Batch with progress and entity filters, then promote the pattern like any other scheduled pipeline. Source: Microsoft Learn: pipeline overview.