Can you tell what a column means from the values alone?

Here is the question I actually had. You ingest thousands of spreadsheets from sources that do not agree on anything. Headers are cryptic, abbreviated, or missing. Can you embed the cell values and recover what each column is for?

It is a tempting idea because it would solve a real problem cheaply. It is also a question that a whole subfield has been answering since 2019, and the answer is more specific than yes or no.

The values alone are not enough, and this is provable

Sherlock is the canonical value-only approach. It treats semantic type detection as classification, trained on 686,765 columns from a large corpus against 78 semantic types, characterizing each column with 1,588 features covering statistical properties, character distributions, word embeddings, and paragraph vectors. It reaches a support-weighted F1 around 0.89. Genuinely good.

Now the failure that matters more than the score.

Consider a column containing Florence, Warsaw, London, Braunschweig. Is it location? city? birthPlace? All three are reasonable. No model reading only those four strings can decide, because the answer is not in the values. It is in what the rest of the table is about.

This is called semantic overlap, and it is not an edge case. It is the normal condition in enterprise data, where a column of numbers might be a quantity, a price, an account identifier, or a year, and the strings are identical either way.

The formal way to say it: single-column prediction is an under-determined problem. You are asking a function to map from a value distribution to a meaning when the mapping is not injective.

So the honest answer to “can cell embeddings glean what a column does” is no, not from the cells alone, and this is a property of the problem rather than a limitation of current embedding models.

Context is what closes the gap

Sato was built specifically to fix this, on the premise that a column’s table context carries additional descriptive power. It combines Sherlock’s single-column signal with topic modeling over the whole table to capture global context, plus structured prediction across neighboring columns for local context. It reaches 0.925 support-weighted F1 and 0.735 macro F1, a significant margin over the prior state of the art.

The macro number is the interesting one. Macro F1 weights rare types equally with common ones, and value-only models degrade badly on types with few training examples. Context helps most exactly where you have least data, which in a heterogeneous ingest pipeline is most of your long tail.

Starmie pushed this further with contrastive learning. It trains column encoders from pretrained language models with no labels at all, using a multi-column pretraining strategy so that each column’s embedding is contextualized by the columns around it. On table union search it beats prior approaches by 6.8 in MAP and recall, and critically it also beats its own single-column ablation by a clear margin. Same architecture, same training, the only difference being whether the encoder sees the neighboring columns.

There is a robustness result here too that I found more persuasive than the headline numbers. Under increasing sparsity, context-capturing methods hold steady while a purely column-based approach loses roughly half its MAP@10 at 15% sparsity. Real spreadsheets are sparse. Half your cells are blank, merged, or footnote garbage. If your representation depends only on the values in a column, sparsity degrades it directly. If it depends on the column’s position in a structure, you have something left to work with.

The number that should scare you

Everything above is measured within a benchmark. Here is what happens when you move.

A value-only model evaluated under distribution shift, applied to a benchmark it was not trained on, drops from 84.8% to 23.8%.

Sixty points. This is the result I would tattoo on anyone building an ingestion pipeline. Published column-typing accuracy is measured on web tables from public corpora. Your vendor exports are not web tables. They have merged header rows, units in the header, footnotes in the data region, and conventions specific to one company’s accounting software.

The model has not seen your distribution and the benchmark numbers do not transfer.

What about just asking an LLM

The obvious 2026 move is to skip the embeddings and prompt a model. This works better than nothing and worse than you expect.

On SOTAB, a realistic benchmark of web tables, prompt-based methods have struggled to exceed 66% accuracy even with tailored pre- and post-processing. Models are also notably sensitive to prompt phrasing, where minor wording changes produce significant performance swings, which is an unpleasant property for a pipeline component.

Part of this is the benchmark rather than the model. One analysis found that in SOTAB the types Integer, Number, and QuantitativeValue are effectively interchangeable and account for about 5% of test columns, so some of the measured error is label ambiguity rather than model failure. Fair. But if the ground truth is ambiguous between three numeric types, that ambiguity exists in your data too, and you inherit it.

The practical read: LLM annotation is good at proposing a mapping and bad at being trusted without review. Which is exactly how I use it.

Numbers are the hard part and nobody says so loudly enough

Every approach above works better on text columns than numeric ones, and recent work on generalist tabular embeddings names this directly: text embedding models frequently fail to capture both tabular structure and numerical semantics.

Think about what an embedding of 4821.50 is supposed to encode. Text embedding models tokenize numbers into pieces and place them in a space organized around linguistic similarity. 4821.50 and 4821.60 are near-identical semantically and may not be near each other in that space at all. Meanwhile 4821.50 and 4821.50 from a completely different concept in a different sheet are identical.

Numeric columns are where you most want semantic help, because a numeric column with a cryptic header is genuinely opaque, and they are where value-embedding helps least.

What I actually take from this

Do not embed individual cells and expect meaning. A cell is too small a unit. The literature moved from cell-level to column-level to context-aware column-level for a reason, and every step improved things.

Embed columns with their context, not in isolation. Whatever encoder you use, it should see neighboring column headers and sample values from the same table. The single-column versus multi-column ablation is the clearest result in this area.

Use embeddings for matching, not for classification. This is the distinction I would emphasize most. Column embeddings are strong at “is this column the same concept as that column,” which is retrieval. They are much weaker at “what is this column,” which is classification against a fixed label set. In an ingest pipeline, matching is what you actually need: you want to know that this vendor’s Inv Dt is the same thing as another vendor’s invoice_date, not to place either into a taxonomy of 78 types.

Keep a human in the loop on the mapping table, and make it cheap. Given the distribution shift result, treat any automated mapping as a proposal. Confirmed mappings are permanent and reusable. Model-proposed mappings should be re-runnable when the model improves. The value compounds because you are building an asset, not repeatedly running inference.

Do not send numbers through a text embedder and hope. Type them, store them typed, and query them with SQL. Embeddings should handle the prose and the headers. Arithmetic should not be a retrieval problem.

The overall shape: embeddings are excellent glue between heterogeneous sources and a poor substitute for knowing what your data means. In an ingestion system they belong in the header-matching layer, not the answer-producing layer.

References

• Hulsebos et al., Sherlock: A Deep Learning Approach to Semantic Data Type Detection, KDD 2019
• Zhang et al., Sato: Contextual Semantic Type Detection in Tables, VLDB 2020, arXiv:1911.06311
• Fan et al., Semantics-Aware Dataset Discovery from Data Lakes with Contextualized Column-Based Representation Learning (Starmie), VLDB 2023, arXiv:2210.01922
• Feuer et al., ArcheType: A Novel Framework for Open-Source Column Type Annotation using Large Language Models, VLDB 2024
• Korini and Bizer, Column Type Annotation using ChatGPT, TaDA 2023
• Pal et al., ALT-GEN: Benchmarking Table Union Search using Large Language Models, TaDA 2024
• Babamahmoudi et al., Evaluating Column Type Annotation Models and Benchmarks, WWW Companion 2025