DRAFT This is as an update to an earlier post1, in which the goal is to increase the size of a golden dataset in order to help compare the query performance between two embedding models, all-MiniLM-L12-v2 and all-mpnet-base-v2 and the comparison is important because they are 384 and 768 dimensions respectively, meaning that the second one has twice the storage costs as the other and live postgresql storage is expensive not just for its storage but also for the storage of the hnsw indexes involved 😅....

Recently, I had been interested in locally reproducing the typesense huggingface models on my laptop. I want to experiment with the https://typesense.org nodes, but I also want to be able to use the same embedding models on my laptop for local development. I noticed that the models in the typesense section of hugging face are in the model.onnx format which I had not encountered before. I learned how to get them running locally and I was able to compare that the vectors on a typesense cluster I was running matched vectors I generated locally....

Recently, I got to the point in a project, of looking into TypeSense as an option for embedding hosting for search. Prior, I was working with one particular embedding model, all-mpnet-base-v2, which intuitively and anecdotally performed decently well for my retrieval task. But yea that was the problem, my information was anecdotal and cherry-picked. But when I started looking into TypeSense, I noticed my model of choice was not in the list, https://huggingface....

a bit tricky, not super intuitive in the docs, but I finally found how to builf a typesense query to do what I had previously done with postgres, that is, first constrain by postgis distance and then order by cosine distance. with a text_embedding column that embeds a text column, and lat lng available as well, flattened on the same collection, it is possible to query like so, { "q": query, "query_by": "text_embedding", "filter_by": f"location:({lat}, {lng}, {radius_km} km)", "sort_by": "_vector_distance:asc", "exclude_fields": "text_embedding", 'page': 1, 'per_page': 100 }

Have not covered yet, a few other cool topics. Can go into more detail later, but, for now some highlights. Runtime embeddings to save money At one point in a recent postgres pgvector retrieval project, a really cool epiphany was, w.r.t. indexing in pgvector both a concatenated blob of items and granular items, that it is not necessary to also embed the low level items because they can be embedded at runtime and it is not that time consuming....