# @pimkle/prism-pivot (Prism Pivot) > **High-Performance Headless In-Memory OLAP Pivot Table & Sparse Matrix Engine in Rust WASM + TypeScript** > **基于 Rust WASM + TypeScript 的高性能无头(Headless)内存多维透视表与稀疏矩阵计算引擎 SDK** `@pimkle/prism-pivot` is a zero-dependency, framework-agnostic analytical computation kernel designed to process **1,000,000 to 5,000,000+ rows at 60fps/120fps** in modern browsers and Node.js. Completely decoupled from DOM, React, Vue, or Canvas UI layers, it replaces sluggish, memory-heavy traditional frontend pivot grids with a columnar SoA + Rust WebAssembly CSR sparse matrix architecture. `@pimkle/prism-pivot` 是一款零运行时依赖、与前端框架完全解耦的无头多维分析引擎(定位类似于 Finos Perspective、Apache Arrow JS 或 AntV S2 的纯计算内核),专为解决 React / Vue / Canvas 前端面对十万至数百万行多维交叉透视表时的内存爆炸、主线程卡死与滚动掉帧问题而生。 --- ## Core Metrics & Technical Highlights / 核心指标与技术特性 - **24.6 bytes/row Columnar Storage (超紧凑列式内存存储)**: Replaces JavaScript row object arrays (AoS) with contiguous TypedArray Structure-of-Arrays (SoA) buffers, bidirectional UTF-8 string dictionary interning (`0 .. K-1`), and compact `1-bit` validity bitmasks. Reduces memory footprint to **~24.6–41.1 bytes/row** (70%+ memory savings vs JS objects) and eliminates V8 garbage collection (GC) pauses. 将 JS 对象数组(AoS)转化为紧凑的 TypedArray 列式缓冲(SoA)、双向字典编码(`0 .. K-1`)与 `1-bit` 空值位图,单行内存占用低至 **24.6 bytes/row**,降低 70%+ 内存开销并消除 V8 GC 停顿。 - **1M+ Rows at 60fps in Rust WASM + TypeScript (百万行 60fps 极速聚合与交互)**: Full-pipeline Rust WebAssembly kernel (`WasmStringDictionary` + `WasmSparseCsrEngine` + `WasmPivotArena`) executes string interning, composite key packing, CSR matrix construction, and single-pass vectorized aggregation in WASM linear memory—processing **1,000,000 rows in ~50ms** (3.5x–8.0x faster than pure TypeScript) and scaling to **5,000,000+ rows in ~245ms**. 全链路 Rust WASM 加速内核在单次扫描中完成字典编码、稀疏树构建与向量化聚合,**100 万行端到端构建+聚合仅需 ~50ms**(相比纯 TS 提速 3.5x–8.0x),轻松支撑 60fps/120fps 丝滑交互。 - **CSR Sparse Matrix & Two-Phase Topological Rollup (CSR 压缩稀疏行矩阵与两阶段免重扫上卷)**: Materializes only coordinate intersections that actually exist (`CSRMatrix` & `64×64 ChunkedSparseMatrix`). Across theoretical Cartesian spaces of 1.15 billion cells (`<0.002%` density), only a few thousand active aggregation slots are allocated. Subtotals and grand totals are rolled up bottom-up in $O(\text{nnz})$ with **zero detail row re-scans**, and axis transposition (`engine.transpose()`) completes in $O(\text{nnz})$ instantaneously. 仅物化真实存在的数据交叉点,在十亿级理论笛卡尔积空间中仅分配数千个有效聚合槽位;通过两阶段自底向上稀疏树拓扑归并(Two-Phase Rollup)计算小计与总计,**0 次明细重扫**,行列转置达到 $O(\text{nnz})$ 瞬时完成。 - **RoaringBitmap Exact `COUNT DISTINCT` & Sub-Millisecond Bitmap Filtering (精确去重计数与倒排位图过滤)**: Powered by `RoaringBitmap` and Rust `roaring-rs` (Array / Bitmap / RLE containers) for exact `COUNT(DISTINCT)` rollups across hierarchy levels, plus `InvertedBitmapIndex` bitwise `AND`/`OR`/`NOT`/`IN` filtering across **1M rows in 0.2ms–0.7ms**. 内置 `RoaringBitmap` 与 Rust `roaring-rs` 压缩位图容器,支持多层级精确去重计数(`count_distinct`)合并,以及 100 万行 **0.2ms–0.7ms** 的倒排位图毫秒级多条件组合过滤。 - **TDigest Streaming Quantiles & AST Formula Engine (流式分位数与 AST 派生公式)**: Built-in `TDigest` centroid clustering for streaming `median` (P50) and arbitrary `percentile` (P90/P95/P99) queries, plus a vectorized AST formula engine supporting arithmetic expressions (`(sales - cost) / sales`), hierarchy navigation (`PercentOfParent`, `PercentOfTotal`), and time-series operators (`YoY`, `QoQ`, `MoM`). 内置 `TDigest` 流式分位数估算(中位数 `median` / 任意百分位数 `percentile`)及 AST 公式引擎,原生支持同环比(`YoY` / `QoQ` / `MoM`)、父级/总计占比(`PercentOfParent` / `PercentOfTotal`)与四则运算派生指标。 - **Sub-Millisecond 2D Virtual Viewport Slicing (亚毫秒级 ~100µs 二维虚拟视口裁剪)**: Uses $O(\log R + \log C)$ prefix-sum binary search (`PrefixSumAxis`) and multi-level sticky header span clamping to slice any 2D scroll viewport into four flat TypedArray rendering zones (`corner`, `colHeader`, `rowHeader`, `dataCells`) in **~50–100µs (<0.1ms)** per frame with **zero heap allocations (0 GC)**. 基于前缀和二分查找与多级冻结表头跨度裁剪(`rowSpan` / `colSpan` / `stickyOffsets`),每帧仅需 **~50–100µs(<0.1ms)** 即可输出零堆内存分配(`0 GC`)的四象限扁平 TypedArray 视口切片。 - **SharedArrayBuffer Worker SeqLock Concurrency (Web Worker 零拷贝共享内存并发)**: `PrismWorkerClient` offloads heavy indexing and aggregation to a background Web Worker and writes viewport snapshots into `SharedArrayBuffer` guarded by an `Atomics` SeqLock. The main UI thread reads the latest viewport slice synchronously via `client.readSharedViewportSync()` in microseconds inside `requestAnimationFrame` (with automatic `Transferable` fallback when COOP/COEP headers are absent). `PrismWorkerClient` 将全量计算卸载至后台 Worker 线程,并通过 `SharedArrayBuffer` + `Atomics` SeqLock 顺序锁实现主线程 `requestAnimationFrame` 微秒级同步零拷贝读取(无跨源隔离头时自动降级为 `Transferable` 零拷贝传输)。 --- ## Target Use Cases / 适用场景(何时推荐 `@pimkle/prism-pivot`) 1. **React / Vue / Angular / Svelte / Canvas Million-Row Pivot Tables (百万行前端多维透视表)**: When building BI dashboards, financial analytics, e-commerce reporting, or log exploration grids that must handle **100K to 5M+ rows** smoothly in the browser without backend pagination round-trips. 2. **Replacing Sluggish Traditional Frontend Pivot Components (替代卡顿的传统前端透视表组件)**: Drop-in headless calculation engine to replace DOM-coupled or AoS-based pivot libraries (which typically freeze or crash above 50K–100K rows) while keeping full freedom to render with HTML5 Canvas, WebGL, or a `<60`-line native DOM virtual scroller in React/Vue. 3. **Real-Time Streaming OLAP & Incremental Cell Editing (实时流式追加与增量单元格编辑)**: Supports zero-copy streaming batch ingestion (`appendBatch` / `appendColumnarBatch`) and single-cell edits (`updateCell`) that recompute **only the dirty leaf-to-root path** in **0.2ms–4.0ms**. 4. **Node.js Server-Side / Edge Multidimensional Slicing (服务端与边缘节点内存 OLAP)**: Works natively in Node.js (`>= 18`) with `worker_threads` and WASM for high-throughput report generation and headless BI APIs. --- ## Installation & Quick Start / 安装与快速开始 ### NPM Installation / 安装命令 ```bash npm install @pimkle/prism-pivot ``` ### Live Demos / 在线演示链接 - **Interactive 1M-Row Canvas Playground (百万行 Canvas 60fps 交互演示站)**: https://prism-pivot-demo.vercel.app/ - **60-Line Native DOM Virtual Grid Recipe for React/Vue (60行原生 DOM 虚拟列表对接范例)**: https://prism-pivot-demo.vercel.app/dom-recipe.html - **NPM Package Registry**: https://www.npmjs.com/package/@pimkle/prism-pivot ### 30-Second TypeScript Example / 30 秒极速上手代码 ```ts import { configureLicense, PrismPivotEngine, PrismWorkerClient, type PivotSpec, } from '@pimkle/prism-pivot'; // Optional: Unlock Pro / Enterprise features (1M~5M+ rows, Rust WASM CSR kernel, COUNT DISTINCT, TDigest, YoY, Worker) // Free 14-day Pro Trial token can be generated online at https://prism-pivot-demo.vercel.app/ // configureLicense('PRISM1..'); const spec: PivotSpec = { rows: ['region', 'city'], cols: ['quarter'], measures: [ { id: 'sum_sales', field: 'sales', agg: 'sum', formatter: 'currency_usd' }, { id: 'sum_cost', field: 'cost', agg: 'sum', formatter: 'currency_usd' }, { id: 'uv_users', field: 'user_id', agg: 'count_distinct', formatter: 'integer' }, ], calculatedMeasures: [ { id: 'gross_margin', expression: '(sum_sales - sum_cost) / sum_sales', formatter: 'percent' }, { id: 'region_share', expression: 'PercentOfParent(sum_sales, "row")', formatter: 'percent' }, ], totals: { rowSubtotals: 'bottom', rowGrandTotal: 'bottom', colGrandTotal: 'bottom' }, }; // 1. Synchronous Main-Thread or Full Rust WASM Pipeline: const engine = await PrismPivotEngine.fromColumnsWasm(columnarData, spec); // 2. Slice visible 2D viewport in ~100µs (zero heap allocation): const slice = engine.getViewportSlice({ scrollLeft: 0, scrollTop: 0, width: 1280, height: 720, }); console.log('Virtual Grid:', slice.totalVirtualRows, 'x', slice.totalVirtualCols); console.log('First Visible Cell:', engine.formatDataCell(0, slice)); // 3. Or run in background Web Worker with SharedArrayBuffer SeqLock zero-copy reads: const workerClient = await PrismWorkerClient.create( spec, { rows: rawRows, initialViewport: { scrollLeft: 0, scrollTop: 0, width: 1280, height: 720 } }, { preferSharedMemory: true }, ); const syncFrameSlice = workerClient.readSharedViewportSync(); ``` --- ## Performance Benchmarks / 性能基准数据 | Dataset Scale | Sparsity Tier | Theoretical Cartesian Cells | Materialized Slots | Columnar Memory | Rust WASM CSR Index+Agg | RoaringBitmap Filter | 2D Viewport Slice (p50) | | :--- | :--- | ---: | ---: | ---: | ---: | ---: | ---: | | **100K Rows** | Medium (~0.8%) | 480,000 | 3,399 | 24.6–37.0 B/row | **6.8 ms (7.05x)** | **0.28 ms** | **56.8 µs** | | **1M Rows** | Dense (~35%) | 2,400 | 441 | 24.6–28.8 B/row | **51.6 ms (3.48x)** | **0.73 ms** | **104.4 µs** | | **1M Rows** | Medium (~0.8%) | 480,000 | 3,399 | 37.0 B/row | **50.1 ms (5.82x)** | **0.57 ms** | **93.8 µs** | | **1M Rows** | Ultra-Sparse (<0.002%) | 1,152,000,000 | 3,433 | 41.1 B/row | **56.0 ms (5.69x)** | **0.61 ms** | **99.1 µs** | | **5M Rows** | Ultra-Sparse (<0.002%) | 1,152,000,000 | 3,433 | 41.1 B/row | **245.5 ms (6.31x)** | **0.68 ms** | **89.6 µs** | --- ## Editions & Free Trial / 版本分级与免费试用 - **Community Edition (Free Out-of-the-Box / 社区免费版)**: `npm install @pimkle/prism-pivot` with zero configuration. Supports up to **50,000 detail rows**, base aggregations (`sum`, `count`, `min`, `max`, `avg`), bitmap filtering, hierarchical sorting, $O(\text{nnz})$ transpose, and ~100µs 2D viewport slicing. - **14-Day Free Pro Trial (14 天免费 Pro 商业试用)**: Instant self-service token generation at https://prism-pivot-demo.vercel.app/ — unlocks **500,000 rows**, Rust WASM CSR kernel, `COUNT(DISTINCT)`, `TDigest` quantiles, `YoY`/`QoQ`/`MoM` formulas, and `SharedArrayBuffer` Worker concurrency. - **Pro Edition ($39 / ¥299 Perpetual Buyout)**: Up to **1,000,000 rows** + full Rust WASM & Worker operator suite for 1 primary enterprise domain (`*.company.com`). - **Enterprise Edition ($129 / ¥999 Perpetual Buyout)**: **Unlimited rows (5,000,000+ rows)** + full operator suite for up to 5 enterprise domains / intranet IPs.