What Embeddings Actually Learn

An embedding is often introduced as "just a list of numbers" — a vector that stands in for a word, an image, or an event. That description is true, and almost useless. The interesting part isn't the numbers themselves; it's the geometry that emerges once you have enough of them sitting next to each other.

Distance is a claim

Every embedding space makes an implicit claim: things that are close together are similar in some way that matters for the task. That claim is worth checking, not assuming. Two points can sit close together because the model believes they're semantically related — or because of an artifact in the training data that has nothing to do with meaning.

This is why probing an embedding space matters as much as training it. A few questions I ask by default:

Does distance in the space correlate with a property I actually care about?
Are there directions in the space that correspond to something interpretable?
Does the geometry hold up on data the model didn't see during training?

Physics makes this concrete

Working with detector data made this abstract idea very concrete. If an embedding space is going to inform whether an event looks like known physics or something new, "this looks kind of similar" isn't good enough — the notion of similarity needs to be calibrated and testable, the same way a physical instrument does.

That's pushed me toward treating embedding spaces less like a black-box output and more like a measurement: something with defined uncertainty, known failure modes, and a documented method, rather than a number that's simply trusted because a model produced it.

The practical takeaway

If you're building anything on top of embeddings — search, classification, anomaly detection — spend real time visualizing and probing the space before you trust it. The geometry will tell you things the loss curve never will.