LRU treats all cache misses as equal. That assumption breaks in disaggregated storage and raw-data engines: fetching one item may require a small network read, while another requires a large transfer, tokenization, and type conversion. A good eviction policy should minimize total reconstruction cost, not merely the number of misses.
Change the objective first
For cached item (i), the relevant signals include:
- probability or time until reuse;
- item size;
- cost to fetch or reconstruct it;
- cache capacity consumed while it waits.
This creates a three-way tension. Keeping an expensive item is attractive, unless it is huge or unlikely to be reused. A hit ratio can therefore improve while monetary cost or latency gets worse.
| Metric | What it reveals | What it hides |
|---|---|---|
| Hit ratio | Fraction of requests served from cache | Size and miss cost |
| Byte hit ratio | Fraction of requested bytes served | Compute and remote pricing |
| Fetch cost | Penalty of misses | Cache CPU and memory overhead |
| End-to-end latency | User-visible outcome | Why the policy behaved that way |
In plain terms: losing ten cheap items can be better than losing one item that is painfully expensive to rebuild.
What LBSC contributes
LBSC frames eviction for storage-disaggregated databases as a cost-aware learning problem. Its offline oracle, BeladySizeCost, uses future reuse together with size and fetch cost to produce strong decisions. A LightGBM model then learns to imitate those decisions online from observable history such as recency, frequency, size, and cost.
This separation is useful:
- The oracle asks what a strong cost-aware policy could achieve with future knowledge.
- The model asks how closely online features can approximate that policy.
- Baselines such as LRU or GDSF show whether learning adds value beyond established heuristics.
The reported simulation results showed meaningful total-cost reductions against heuristic baselines, but the exact percentage is less important than the evaluation structure.
The evidence to examine carefully
Synthetic cost weakens external validity
An access trace can provide sequence and object size without recording the real cost of every remote fetch or computation. Assigning synthetic costs is useful for controlled experiments, but the central heterogeneity then comes from the experimenter’s model. A deployment claim needs measured costs from a real path.
Workloads drift
A learned policy can encode yesterday’s reuse pattern. Evaluation should include phase changes, new objects, cost changes, and a retraining delay. Compare not only steady-state savings but also the damage during adaptation.
Inference is not free
Model CPU, feature maintenance, memory metadata, and retraining belong in the cost equation. A small improvement in fetch cost may not justify complexity on a latency-sensitive cache path.
Turning the idea into a credible follow-up
A tempting proposal is to combine NoDB’s positional and binary caches with learned cost-aware eviction. The research question is sound: tokenization and type conversion create heterogeneous reconstruction costs. The initial scope, however, should be much smaller than “build a new database cache.”
Minimum viable experiment
- Use only a chunked CSV binary cache.
- Define an entry as
(file, column, chunk). - Instrument actual read, tokenization, conversion, and memory costs.
- Capture a trace from a real query runner.
- Replay the trace through LRU, GDSF, a simple cost heuristic, an offline oracle, and the learned policy.
- Only integrate online eviction if replay shows a durable gap.
This order separates policy quality from database engineering. If the measured costs are nearly uniform or unpredictable, the proposed advantage disappears early and cheaply.
An ablation plan
| Experiment | Question answered |
|---|---|
| LRU vs cost heuristic | Does cost awareness matter without ML? |
| Heuristic vs learned policy | Does learning add value? |
| Learned policy without plan features | Does query context help? |
| Fixed vs drifting workload | How quickly does the model go stale? |
| Replay vs integrated engine | Does systems overhead erase simulator gains? |
The offline oracle is a ceiling, not a fair deployable competitor. The strongest practical baseline should remain visible in every chart.
Stop conditions are part of the design
The project should stop or change direction if:
- reconstruction costs have little variance;
- cheap features cannot predict reuse or cost;
- a simple size-cost heuristic matches the model;
- inference and metadata consume the measured savings;
- benefits vanish under workload shifts.
Writing these conditions before implementation protects the research question from becoming a demo that must “win.”
Review card
- Optimize the cost of misses, not miss count alone.
- Reuse, size, and reconstruction cost pull eviction decisions in different directions.
- An oracle defines opportunity; a deployable baseline defines practical value.
- Synthetic costs support controlled analysis but not full deployment claims.
- Trace replay should precede deep engine integration.
- Ablations, drift tests, overhead, and stop conditions make a learned-systems result credible.