Journal Safari: Can You Compress a Cell Into a Word?

I miss reading papers.

In grad school we had a journal club we called Journal Safari: pick a genuinely cool paper, read it end to end, and argue about it out loud. The deeper I get into industry, the more I notice that habit atrophying — I skim abstracts, I trust the headline number, I move on. So I’m rebuilding the habit in public. Every couple of weeks I’ll pick one recent paper from a top-tier venue, read it the way I used to, and write up what survived the reading. Single-cell, population genomics, AI in healthcare, or maybe something random, on rotation.

First up is a paper with an irresistible pitch.

Why This Paper

Single-cell foundation models are a genre now. The template has calcified: take a transformer, pretrain it on tens of millions of single-cell RNA profiles, then fine-tune (or run zero-shot) on whatever downstream task you like. scBERT kicked it off in 2022, scGPT scaled it to 33 million cells, scFoundation and scPRINT pushed past 50 million. Each one has a twist.

CellVQ (Wang et al., Nature Communications, 2026) has two twists, and they map directly onto the two words in its title: comprehensive and interpretable. It’s pretrained on 68 million cells with roughly 500 million parameters, and its headline claim is that it beats scGPT, scFoundation, scBERT, UCE, and CellFM across clustering, annotation, property prediction, perturbation, and drug response — mostly zero-shot.

The reason I picked it: the field has a live skeptic problem. This paper cites, as one of its own references, a 2025 result showing that single-cell foundation models often fail to beat simple baselines. So CellVQ is arguing against an active critique that the whole genre might be a very expensive way to underperform a linear model. That’s a paper worth reading closely, because it’s picking a fight I care about the answer to. Also, I’m a sucker for drama in the form of academic sparring.

What They Actually Did

Here’s the coffee-length version.

Take a cell’s expression vector — twenty-thousand-ish genes, mostly zeros. Randomly keep 50–80% of the genes (cheap augmentation, forces the model not to lean on any single gene), run it through a small transformer, and pool it down to one vector per cell. Standard so far.

Then comes the load-bearing trick. Instead of keeping that continuous vector, you snap it to the nearest entries in a learned codebook of 65,536 slots. That discrete code — think of it as a “word” for the cell — is the whole idea. A decoder then has to rebuild the full expression profile from just the code, including the genes you dropped and all those zeros, using a Zero-Inflated Negative Binomial likelihood. (ZINB, if you’ve done microbiome count data, is the same friend: it models the excess zeros explicitly instead of pretending they’re real low counts.)

Why discretize at all? Because the discrete code is supposed to buy three things the authors name up front as their motivation:

• Batch effects. Cells from different labs and protocols get mapped into the same codebook — a shared vocabulary — so the argument is that batch differences compress out.
• Interpretability. A cell is now a short code you can count, compare, and group, instead of a twenty-thousand-dimensional blur.
• Sparsity. Paired with ZINB, the decoder reconstructs the zeros on purpose.

Then, as the “interpretable” payoff, they bolt on a knowledge graph that reads the model’s attention weights to claim which genes define which cell state.

The Part That Genuinely Convinced Me

I want to be fair before I’m skeptical, because there’s a real result here.

The strongest single piece of evidence isn’t the clustering pictures (foundation models always make pretty UMAPs). It’s the property-prediction panel. When you ask these models to predict a cell’s tissue, disease, age, or sex from its embedding, scGPT and scFoundation score below raw gene expression — meaning their fancy embeddings actually threw away biological signal that was sitting right there in the counts. CellVQ stays above raw expression. That’s the best argument that the discrete code retains biology rather than smearing it. If discretization were just lossy compression, this is exactly the number that would crater, and it doesn’t.

Credit where it’s due: the reproducibility hygiene is also better than average. Code is on GitHub under an MIT license, there’s a Zenodo archive, and the pretraining corpus is a documented pull from CZ CELLxGENE. More on that last part later.

What I’d Push On

Now the skeptical read, because a close reading is worthless if it’s just a summary.

The batch-effect “win” is a rescaled ruler. CellVQ reports a metric it calls “AvgBatch,” which turns out to be 1 divided by a silhouette score. That inversion takes a modest 0.098-to-0.072 gap and reports it as a “37.3% improvement.” There’s no bio-conservation axis and — the part that bugs me most — no comparison against the tools people actually use for batch correction, like Harmony or scVI. You can’t claim to have solved heterogeneity while declining to race the field’s standard runners.

The best drug-response number is standing on a leaky split. The headline is a Pearson correlation of 0.923 on drug response. But it comes from a random five-fold cross-validation that lets the same cell line land in both the training and test sets. When the thing you’re predicting has already seen a near-twin during training, your performance is inflated by construction. The field’s honest version of this test is to leave entire cell lines out — and that’s not what they did.

“Interpretability” is doing a lot of unpaid labor. The knowledge graph reads attention weights and narrates that the top genes are known markers. But attention-as-explanation is a contested idea in machine learning for good reason (Attention is not Explanation is practically a genre of its own now), and here it mostly confirms markers we already knew rather than discovering anything. Worse, the “gene regulatory networks” it recovers are built from genes that aren’t transcription factors — which makes them plain co-expression wearing a regulation costume. Two genes going up together is not one gene controlling the other.

And the seams show. The model was called ChrisCell all the way through peer review and got renamed CellVQ after acceptance. The parameter count wanders between 500, 511, and 513 million depending on which sentence you read; the donor count is 452 in one place and 465 in another. None of this is fatal, but it’s the kind of thing that makes you check the numbers that do matter.

Here’s the part I found bracing. Nature Communications publishes its peer reviews, so after I’d written down my objections I went and read what the actual referees said — and they’d raised the same ones. One reviewer flagged the leaky drug-response split almost verbatim. Another called the interpretability claims “circular.” A third noted the baselines disagree with an independent benchmark. All four referees ultimately signed off, but the two hardest quantitative objections were argued away in prose, not re-run, and the strongest evidence in the paper exists specifically because reviewers demanded it. “Accepted after argument” is not the same as “every objection resolved by re-analysis,” and the transparent review file is the honest reader’s cheat sheet for telling the two apart.

The Data-Infrastructure Coda

I do data infrastructure for a living, so I can’t read a paper like this without asking where 68 million cells actually live.

The shape of the problem is telling. It’s 68 million cells by ~20,000 genes, extremely sparse, and training samples a different random subset of genes for every cell on every step. You cannot hold that in memory; you need larger-than-memory, sliced, streaming access to a sparse array. Which is a solved problem: the pretraining corpus came from CZ CELLxGENE, whose Census is built on TileDB-SOMA — cloud-native sparse arrays with AnnData and Seurat interop. Sparsity there is a feature you exploit, not a tax you pay. The substrate this paper implicitly runs on is one the community already knows well.

The less-solved problem is reproducibility. To actually re-run CellVQ, you’d hunt across four different hosts — the cell corpus in one place, test data in another, code on GitHub, an archive on Zenodo — and reconcile the ChrisCell rename yourself. Here, reproducibility is a scavenger hunt. The honest, boring fix already exists and it’s called a Docker image with pinned dependencies; the fact that so few papers ship one is more a culture problem than a tooling gap. I’d love it if “hand me a container that reproduces Figure 3” were a submission requirement, not a favor.

Is a discrete cell code the right bottleneck for biology, or does snapping every cell to the nearest codebook entry throw away exactly the continuous gradients — the trajectories, the in-between states — that biologists care most about? I honestly don’t know yet, and that uncertainty is why the paper is worth the read. The property-prediction result says the code keeps more than I’d have guessed. The linear-model-ties-you asterisk says we might be scaling past the point of diminishing returns. Both can be true.

That’s the whole appeal of doing this in the open: I get to be wrong out loud and let the literature correct me. See you in two weeks with the next one.