01 Why
My grandmother had Alzheimer’s. By the end the hardest part was not the forgetting. It was watching the people who loved her become strangers to her.
Yaad is a voice companion that answers “who is this?” and “did I take my pills today?” from a real memory graph of one person’s life: their family, their routine, their places, their episodes. A family member adds a fact in a web app and it is usable in the next sentence the agent speaks.
The engineering constraint that shaped everything: a system that invents a life is worse than one that admits it does not know. Retrieval below a confidence threshold does not get paraphrased into a confident-sounding guess. It returns “I’m not sure, let me check with the family.”
02 How it works
03 Ask it something
The confidence gate, which is the whole design. Ask about Amma’s life. If retrieval scores above τ you get a grounded answer with its provenance; below τ you get a refusal, and no language model is given the chance to phrase a guess nicely.
Illustrative retrieval, the real engine scores on embeddings plus recency, salience and graph proximity; this uses token overlap so it can run offline. The gate behaviour is the real behaviour.
04 The parts that were actually hard
- Grounded-only generation. The system prompt forbids asserting anything not in the retrieved context. That is easy to write and hard to hold, the model very much wants to be helpful and fill gaps.
- Instant updates. A fact added in the caregiver web app has to be retrievable in under a second, so the write path indexes into Moss synchronously rather than waiting for a batch job.
- Temporal questions are a different query. “Did I take my pills today?” is not semantic search. It is a lookup against today’s medication log. Time-sensitive intents get routed away from the vector path entirely.
- Cross-lingual retrieval. A question asked in Hindi has to match memories stored in English, which the embeddings handle but which needed checking rather than assuming.
- A hard safety rule: if the user seems lost, the agent reassures them and alerts a human. It never gives turn-by-turn directions to a disoriented person.
Honest limits. This was a hackathon build and the demo path is the tested path. Reliable autonomous memory capture from open conversation is the weakest link. We shipped explicit “remember this” capture plus caregiver review, because passive extraction was not trustworthy enough to put in front of a vulnerable person. It has never been used by an actual patient, and it should not be until it is tested with clinicians.