Skip to content

Add support for evaluating reduced-dimension (Matryoshka) embeddings - #7

Open
Andrian0s wants to merge 1 commit into
machinelearningZH:mainfrom
Andrian0s:add-embedding-truncation
Open

Add support for evaluating reduced-dimension (Matryoshka) embeddings#7
Andrian0s wants to merge 1 commit into
machinelearningZH:mainfrom
Andrian0s:add-embedding-truncation

Conversation

@Andrian0s

Copy link
Copy Markdown
Contributor

Closes #6

Summary

This PR adds the possibility to evaluate embedding models with reduced dimensions (Matryoshka), implemented as a truncation on the embeddings. It lets users easily check whether their domain would see a significant performance drop from using dimensionality-reduced embeddings.

What changed

  • Added an option to truncate embeddings to a target dimension before running the evaluation.
  • Kept the edits minimal and contained within the existing evaluation flow.

Notes

The dimensionality reduction is a plain truncation of the embedding vectors, as used in Matryoshka representation learning.

Not all models "guarantee" matryoshka representation learning dimensionality reduction results but other models won't break with this addition either, they will just underperform which the user should understand. No need to handle any form of "checks" before running with the truncation

Should work as expected but I recommend the maintainers to ensure that nothing broke from these minimal additions.

…sier examine efficiency performance tradeoffs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adding Possibility for Evaluating Embedding Models with Reduced Dimensions (Matryoshka)

1 participant