LRU treats all cache misses as equal. That assumption breaks in disaggregated storage and raw-data engines: fetching one item may require a small network read, while another requires a large transfer, tokenization, and type conversion. A good eviction policy should minimize total reconstruction cost, not merely the number of misses.
Change the objective first
For cached item (i), the relevant signals include:
- probability or time until reuse;
- item size;
- cost to fetch or reconstruct it;
- cache capacity consumed while it waits.
This creates a three-way tension. Keeping an expensive item is attractive, unless it is huge or unlikely to be reused. A hit ratio can therefore improve while monetary cost or latency gets worse.
| Metric | What it reveals | What it hides |
|---|---|---|
| Hit ratio | Fraction of requests served from cache | Size and miss cost |
| Byte hit ratio | Fraction of requested bytes served | Compute and remote pricing |
| Fetch cost | Penalty of misses | Cache CPU and memory overhead |
| End-to-end latency | User-visible outcome | Why the policy behaved that way |
In plain terms: losing ten cheap items can be better than losing one item that is painfully expensive to rebuild.
What LBSC contributes
LBSC frames eviction for storage-disaggregated databases as a cost-aware learning problem. Its offline oracle, BeladySizeCost, uses future reuse together with size and fetch cost to produce strong decisions. A LightGBM model then learns to imitate those decisions online from observable history such as recency, frequency, size, and cost.
This separation is useful:
- The oracle asks what a strong cost-aware policy could achieve with future knowledge.
- The model asks how closely online features can approximate that policy.
- Baselines such as LRU or GDSF show whether learning adds value beyond established heuristics.
The reported simulation results showed meaningful total-cost reductions against heuristic baselines, but the exact percentage is less important than the evaluation structure.
The evidence to examine carefully
Synthetic cost weakens external validity
An access trace can provide sequence and object size without recording the real cost of every remote fetch or computation. Assigning synthetic costs is useful for controlled experiments, but the central heterogeneity then comes from the experimenter’s model. A deployment claim needs measured costs from a real path.
Workloads drift
A learned policy can encode yesterday’s reuse pattern. Evaluation should include phase changes, new objects, cost changes, and a retraining delay. Compare not only steady-state savings but also the damage during adaptation.
Inference is not free
Model CPU, feature maintenance, memory metadata, and retraining belong in the cost equation. A small improvement in fetch cost may not justify complexity on a latency-sensitive cache path.
Turning the idea into a credible follow-up
A tempting proposal is to combine NoDB’s positional and binary caches with learned cost-aware eviction. The research question is sound: tokenization and type conversion create heterogeneous reconstruction costs. The initial scope, however, should be much smaller than “build a new database cache.”
Minimum viable experiment
- Use only a chunked CSV binary cache.
- Define an entry as
(file, column, chunk). - Instrument actual read, tokenization, conversion, and memory costs.
- Capture a trace from a real query runner.
- Replay the trace through LRU, GDSF, a simple cost heuristic, an offline oracle, and the learned policy.
- Only integrate online eviction if replay shows a durable gap.
This order separates policy quality from database engineering. If the measured costs are nearly uniform or unpredictable, the proposed advantage disappears early and cheaply.
An ablation plan
| Experiment | Question answered |
|---|---|
| LRU vs cost heuristic | Does cost awareness matter without ML? |
| Heuristic vs learned policy | Does learning add value? |
| Learned policy without plan features | Does query context help? |
| Fixed vs drifting workload | How quickly does the model go stale? |
| Replay vs integrated engine | Does systems overhead erase simulator gains? |
The offline oracle is a ceiling, not a fair deployable competitor. The strongest practical baseline should remain visible in every chart.
Stop conditions are part of the design
The project should stop or change direction if:
- reconstruction costs have little variance;
- cheap features cannot predict reuse or cost;
- a simple size-cost heuristic matches the model;
- inference and metadata consume the measured savings;
- benefits vanish under workload shifts.
Writing these conditions before implementation protects the research question from becoming a demo that must “win.”
Review card
- Optimize the cost of misses, not miss count alone.
- Reuse, size, and reconstruction cost pull eviction decisions in different directions.
- An oracle defines opportunity; a deployable baseline defines practical value.
- Synthetic costs support controlled analysis but not full deployment claims.
- Trace replay should precede deep engine integration.
- Ablations, drift tests, overhead, and stop conditions make a learned-systems result credible.
LRU 把每次 cache miss 視為相同。這個假設在 disaggregated storage 與 raw-data engine 不成立：一個 item 可能只需少量 network read，另一個卻要大量 transfer、tokenization 與 type conversion。好的 eviction policy 應最小化 total reconstruction cost，而不是只數 miss 次數。
先改 objective
對 cached item (i)，重要 signal 包含：
- reuse probability 或下次 reuse 的時間；
- item size；
- fetch／reconstruction cost；
- 等待期間占用的 cache capacity。
三者會互相拉扯。昂貴 item 值得保留，除非它太大或幾乎不會再用。因此 hit ratio 可能變好，但 monetary cost 或 latency 反而變差。
| Metric | 看得見 | 看不見 |
|---|---|---|
| Hit ratio | 從 cache 回覆的 request 比例 | Size 與 miss cost |
| Byte hit ratio | 從 cache 回覆的 byte 比例 | Compute 與 remote pricing |
| Fetch cost | Miss 的代價 | Cache CPU 與 memory overhead |
| End-to-end latency | 使用者結果 | Policy 為何這樣選 |
白話來說： 丟掉十個很便宜的 item，可能比丟掉一個重建代價極高的 item 更好。
LBSC 的貢獻
LBSC 把 storage-disaggregated database 的 eviction 定義為 cost-aware learning problem。Offline oracle BeladySizeCost 使用 future reuse、size 與 fetch cost 產生強決策；LightGBM model 再從 recency、frequency、size、cost 等可觀察 history，學習在線近似 oracle。
這個拆法很有用：
- Oracle 問：知道未來時，cost-aware policy 最多能做到什麼？
- Model 問：online feature 能多接近這個上限？
- LRU、GDSF 等 baseline 問：learning 是否勝過成熟 heuristic？
Paper simulation 對 heuristic baseline 顯示有意義的 total-cost reduction；比單一百分比更值得留下的是這個 evaluation structure。
證據要小心看的地方
Synthetic cost 限制 external validity
Access trace 可能只有 sequence 與 object size，沒有每次 remote fetch 或 computation 的真實成本。Synthetic cost 適合 controlled experiment，但核心 heterogeneity 也就來自實驗者假設。要做 deployment claim，仍需量真實 data path。
Workload 會 drift
Learned policy 可能記住昨天的 reuse pattern。Evaluation 應包含 phase change、new object、cost change 與 retraining delay；不只比 steady-state saving，也要比適應期間造成的損失。
Inference 不是免費
Model CPU、feature maintenance、memory metadata 與 retraining 都要放進 cost equation。在 latency-sensitive cache path 上，小幅 fetch saving 未必值得系統複雜度。
把想法變成可信的後續研究
把 NoDB positional／binary cache 與 learned cost-aware eviction 結合是合理題目：tokenization 和 conversion 確實可能形成 heterogeneous reconstruction cost。但第一版不該直接變成「打造新的 database cache」。
Minimum viable experiment
- 只處理 chunked CSV binary cache。
- Entry 定義為
(file, column, chunk)。 - Instrument 真實 read、tokenization、conversion 與 memory cost。
- 從真實 query runner 收 trace。
- Offline replay LRU、GDSF、simple cost heuristic、oracle 與 learned policy。
- Replay 顯示穩定 gap 後，才做 online integration。
這個順序把 policy quality 與 database engineering 分開。如果實測 cost 幾乎一致或無法預測，研究優勢可以很早、很便宜地被否證。
Ablation 計畫
| Experiment | 回答的問題 |
|---|---|
| LRU vs cost heuristic | 不用 ML，cost awareness 是否已有效？ |
| Heuristic vs learned policy | Learning 是否真的增加價值？ |
| 拿掉 plan feature | Query context 是否有幫助？ |
| Fixed vs drifting workload | Model 多快過時？ |
| Replay vs integrated engine | Systems overhead 是否吃掉 simulator gain？ |
Offline oracle 是上限，不是公平的 deployable competitor；最強 practical baseline 應該一直留在每張 chart。
Stop condition 也是設計的一部分
遇到以下情況就應停止或換方向：
- Reconstruction cost 幾乎沒有 variance；
- 便宜 feature 無法預測 reuse 或 cost；
- Simple size-cost heuristic 已追上 model；
- Inference 與 metadata 吃掉 savings；
- Workload shift 後 benefit 消失。
Implementation 前先寫下這些條件，可以避免研究問題最後變成「一定要贏」的 demo。
複習卡
- 應優化 miss cost，而不只是 miss count。
- Reuse、size 與 reconstruction cost 會把 eviction decision 拉向不同方向。
- Oracle 定義 opportunity；deployable baseline 定義 practical value。
- Synthetic cost 支持 controlled analysis，但不足以支撐完整 deployment claim。
- 深入改 engine 前應先做 trace replay。
- Ablation、drift test、overhead 與 stop condition 讓 learned-systems 結果可信。