Posts

Have not covered yet, a few other cool topics. Can go into more detail later, but, for now some highlights. Runtime embeddings to save money At one point in a recent postgres pgvector retrieval project, a really cool epiphany was, w.r.t. indexing in pgvector both a concatenated blob of items and granular items, that it is not necessary to also embed the low level items because they can be embedded at runtime and it is not that time consuming. The dataset is a corpus dishes grouped by menus. The initial design was to allow a two step embedding search, first across the concatenated menu embeddings to narrow down, and then on the menu level, to search the dish item level embeddings a second time. But disk space wise, it became clear that the dish items and their embedding hnsw index would take up an enormous amount. There are typically 10 dishes per menu so a proportionally ~10x larger space was needed. But the duhh moment was that maybe we don’t even need to store the dish item level index at all because maybe we can embed the dishes on the fly when we already have done a first pass, ending up with maybe 5 to 10 menus. Now we would only need to embed maybe 50 to 100 dishes and that would not be too slow. And indeed embedding and searching through 50 to 100 dishes did take time but it appeared to be worth the precious postgresql disk space saved. Of course depending on future query latency requirements it is always possible to reconsider this. ...

Wow, I found a very silly issue that was holding back this query. The data is a langchain embedding table of items associated with places, as well as a second table with those place addresses along with their longitude, latitude. Also, as I had a slower query, executed through sqlalchemy, like from sqlalchemy.sql import text with foo.engine.connect() as conn: data = {"vector": [0.06014461815357208, 0.07383278757333755, 0.010295705869793892, -0.058833882212638855,], "longitude": "", "latitude": ""} sql = """ """ statement = text(sql) # TODO also parameterize the `limit 10` ? fetchmany() results = conn.execute(statement, data).mappings().fetchall() I’m glad about three strategies I used to help speed up the debugging. Namely, (1) deconstructing the sqlalchemy code, into a raw SQL query I could play with, (2) using DBeaver to quickly throw my raw SQL query against postgresql, for super fast iteration, and (3) of course using explain analyze to look at query plans to understand where the bottle neck may be. ...

looked at someone’s fun project, https://www.npmjs.com/package/@rhettlunn/is-odd-ai That uses GPT 3.5 to see if a number is odd, for fun. My friend was asking has anyone evaluated this, so hmm i took a quick look. https://github.com/rhettlunn/is-odd-ai/blob/main/index.js Spotted two interesting things. Prompt injection attack not sure if i just made that up, but like a SQL injection , the code that places user input into the prompt asking about oddness, doesn’t type check if it is a number, and doesn’t therefore have any way of knowing will someone attempt to insert some text that will cause GPT to escape its sandbox. of course chat GPT is intentionally sandboxed for most purposes, but it can still typically do internet research and GPT analysis can execute code in a limited VM. ...

I had a issue where , using the langchain_pgvector library, with the vectorstore.add_documents function call, which has worked for me before for a while but somehow, for a new collection, I’m trying to add thousands of documents, but only a handful get added, and without errors. Very weird . I didn’t see any useful patterns in the 4 out of 1022 documents that did get added, and pdb tracing through the code did not reveal any silent errors. ...

This contains spoilers. This isn’t really a review, but maybe just free writing about a game I really enjoyed, recently. Maybe that is a review haha not sure. This is a story mode action adventure explorer type like Borderlands, walking dead, life is strange, and many others. Where, you Control a protagonist, in this case a cat,, 🐈, interacting with other characters in the game, as well as exploring your world to solve mostly 3 -space geometry puzzles, of , how to navigate a cat , with jumps and crawl spaces. The storyline feels original, reminiscent of one of the early post apocalyptic episodes of the Netflix original Love death and robots , but more intense. ...

First, an embedding table was created, using langchain pgvector. (TODO show that example). Initial query which was working, using, a chosen_embedding, of some uuid I randomly picked from the vector table, using the cosine similarity <=> as an ORDER BY, explain analyze with myblah(chosen_embedding, chosen_id) as ( values ( (SELECT embedding FROM langchain_pg_embedding WHERE id = '280aefd0-cb15-4a54-924d-aab37ee8a816' ), '280aefd0-cb15-4a54-924d-aab37ee8a816') ) SELECT substr(id, 0, 8) as id, substr(document, 0, 40) as doc, round( 1 - cast(embedding <=> chosen_embedding as numeric), 3 ) as score, cmetadata->'name' as name FROM langchain_pg_embedding, myblah WHERE id != chosen_id ORDER BY embedding <=> chosen_embedding LIMIT 100; with, output, we see below, the ix_embedding_hnsw index created last time does have an index scan. ...

I had this interesting situqtion, where I wanted to plot some numbers that were nested inside of struct columns. They were row counts in a delta table history output, but in any case, I tried to plot them, but my plot treated them as categories. Ok realizing they were strings, I cast them to integers, but then I got nulls. After a bit of trial and error I realized they were probably laerger than 32bit! Casting to big int, aka, long, did the trick. ...

placeholder. didnt get chance to write this up yet, but I had used langchain pgvector, to add embeddings to postgresql , I ran my queries, and noticed they were slow. I read about pgvector indexing, and on psql, noticed my embedding column was missing an index! I tried adding the HNSW index manually. Weird error about no dimensions on the vector column. ok learned need explicit dimension. Added it. Nice adding index worked. ...

motivation I know more about Databricks for distributed computing. But there is also the Kubernetes world. Let’s read some more on that. Was wondering hmm, I have used Tensorflow, pytorch on standalone VMs in the past and I know that Spark has its own distributed libraries, so do Tensorflow, pytorch natively support spark, hmm? lingering questions (1) What are the dominant technologies for distributed computing? (spark, and containerization by K8S) (2) specifically GPU vs cluster based, are these alternatives? (3) Are the typical deep learning options (Tensorflow, pytorch) abstracted by Horovod/ TFJobs or did Tensorflow , pytorch need to be modified to suport multiple clusters ? Wondering about the level of abtraction basically. (4) Back prop has some parallelization opportunities , but there are lot of dependencies too. wondering, hmm, what are some parallelization methods w.r.t. what is available? (5) are Horovod (and TFJobs) only really for deep learning or how about xgboost? the dominant architectures/frameworks? Somehow I got the impression that, since whenever I opened up a google colab notebook in the past and saw GPU set up for tensorflow or pytorch, and then later when I started working with Databricks, I got the impression that GPU and clusters were an either or thing. ...

My iphone has been overheating lately, 🤔 Read one good tip, forget where, just try to avoid using your phone while charging it if it is starting to overheat. But also, if it is really heating up, to stop charging it just in case. Makes sense. Going to try that next time. But what is the temp anyway? Nice, learned from here, https://www.guidingtech.com/how-to-check-iphone-temperature/, you can access your iphone temp through the Privacy Settings, Analytics data. Looks like I had this turned on so I downloaded two of my last files there, ...