as always it depends on your hardware, the tiny models for embedding / reranking typically have low latency
qmd is focussed on local to the point of designing around single machine setups and this creates a gap where one runs agents+qmd on their laptop and LLMs on their ai box
yup, one of the reasons I built gmd to replace qmd, also that I wanted to use Typesense and incorporate llm-wiki concepts. Updating memory does depend on the nature of the change and may impact multiple memories or indexed files, and why llm-wiki concepts are needed too.
Right now, imo, the main reason to build something like this or a harness is to understand the quirks son you can evaluate production grade implementations as the space matures. We are all still very early in the curve.
We use capn crunch (for cerealization)
For crypto/stock transfers we ofc use chips n dips
looks cool. what is latency like? I haven't used qmd before and it looks like it runs 3 local models.
as always it depends on your hardware, the tiny models for embedding / reranking typically have low latency
qmd is focussed on local to the point of designing around single machine setups and this creates a gap where one runs agents+qmd on their laptop and LLMs on their ai box
if this were localai, you could just figure out how to update the memore if the fill changes.
yup, one of the reasons I built gmd to replace qmd, also that I wanted to use Typesense and incorporate llm-wiki concepts. Updating memory does depend on the nature of the change and may impact multiple memories or indexed files, and why llm-wiki concepts are needed too.
Right now, imo, the main reason to build something like this or a harness is to understand the quirks son you can evaluate production grade implementations as the space matures. We are all still very early in the curve.
https://github.com/verdverm/gmd