Skip to content

Dense Vector

A vector in which most or all dimensions carry non-zero meaningful values, typical of neural network embedding outputs.

A dense vector is one in which most or all of the dimensions hold non-zero, meaningful values. This is the kind of vector produced by neural network embedding models, where information about the input is spread smoothly across every dimension rather than concentrated in a few positions.

Density contrasts with sparse vectors, which are mostly zeros. A sparse keyword vector might have one entry per vocabulary word, almost all of them zero; a dense embedding has a few hundred or thousand entries, essentially all active. This compactness is what allows smooth geometric relationships — items with similar meaning sit close together, and you can interpolate between concepts.

Dense vectors are what vector databases are primarily designed to store and search. Their fixed, consistent dimensionality makes memory usage predictable and fits the assumptions of approximate-nearest-neighbour index structures like HNSW and IVF, which expect every vector to have the same shape.