Live knowledge augmentation cards for calls and videos, on your Mac.

Gloss is a macOS app that listens beside a video call (Zoom, Google Meet, Teams) or a YouTube video. When someone names a company, a founder, a fund, a statute or a term you may not know, Gloss puts a short card on screen a few seconds later: two or three lines, written from the source. Think of Prime Video's X-Ray, but for live conversations, and quiet about what you already know.

Private prototype, October 2026. Shared with a small group of testers by invite. This page stands in for a website and a repository, which are not public yet.

someone on the call says:  "... I remember meeting Parker Conrad when he was 
working on Zenefits..."

  ┌─ Gloss ─────────────────────────────────────────────────────────────┐
  │  TriNet Zenefits, cloud HR software company                          │
  │  · Cloud HR software focused on payroll and health insurance.        │
  │  · Founded 2013; formerly Zenefits, legally YourPeople, Inc.         │
  │  · Raised $500M in 2015 at $4.5B valuation; 2014-2015 service issues.│
  └──────────────────────────────────────────────────────────────────────┘

(The card Gloss drew in a real session, verbatim. Screenshots are attached separately.)


Why it exists

In a meeting or a talk, names fly by: a fund, a founder, a defunct startup, a regulation. Looking each one up breaks your attention, and most "AI meeting assistants" work after the call, from a transcript sent to a server. Gloss works during the call, keeps the transcript on the laptop, and treats silence as a valid output: most of the time the window is empty, because a card is only worth your attention when it tells you something you don't know.


How it works

  call / video audio
        │  Core Audio process tap (a small Swift helper): 
        │  the app's own audio, no virtual device
        ▼
  speech-to-text  (Deepgram Nova-3, streaming; told which names to listen for)
        │  words, a few hundred ms apart
        ▼
 ┌──────────────────────────── ON THE MAC (Rust) ─────────────────────────────┐
 │  1 DETECT    Aho-Corasick over ~18,500 entities and 84,667 aliases          │
 │              + Double Metaphone for misheard names                          │
 │  2 SCORE     A per-name "is this even an entity?" prior from word-frequency │
 │              lists; context from what the conversation already mentioned    │
 │  3 FILTER    How famous is it? and, if you wrote a paragraph about yourself,│
 │              how likely are YOU to know it?                                 │
 │  4 VERIFY    Qwen3-4B (4-bit, MLX) on the laptop answers one question:      │
 │              "does this sentence mean THIS entity?"   ~0.8 s, yes / no      │
 └─────────────────────────────────────────────────────────────────────────────┘
        │  yes → the card is drawn (No card? Unknown name? See below)
        ▼
 ┌──────────────────────────── THE SERVER (small) ────────────────────────────┐
 │  A name the Mac doesn't know, or knows without a card → Claude Haiku writes │
 │  the card ONLY from its Wikipedia article, once, cached for everyone        │
 └─────────────────────────────────────────────────────────────────────────────┘

The split is the design: the small model on the laptop judges (is this the right meaning, here?), the big model on the server writes (from a source, never from memory). Each does the job it measured best at.


What makes it technically interesting

1. Give each model the job it is good at. Our on-device 4B model is a good judge. Given 18 real moments, each with the right card and a wrong card for the same word ("Edo" which is the old name of Tokyo, against "EDO Corporation"), it turned down 17 of the 18 wrong cards and kept 13 of the 18 right ones. Asked to explain its answer instead, it said “no fit” to all 36. Cards which a model wrote from memory contradicted their source on 39 of 240 lines; written from the article, 230 of 231 lines were supported. Lesson: The laptop judges, and the server writes, from a source.

2. Classic before clever. Half our false positives were the words “we”, “he”, “so” and “if”. They are real company abbreviations in Wikidata (WE is Western Electric). Lesson: A per-name probability of being an entity at all, built from word-frequency lists, was one of our biggest precision wins, bigger than tuning the matcher.

3. Check the instrument before you fix the output. Our instruments were wrong more often than our product: (i) A regex that couldn’t see years (69 of 75 “broken” lines were correct sentences). (ii) An LLM judge grading cards without the source they were written from. Lesson: Read ten of the things a metric flags before you spend a week on its number.

4. The names a listener most needs are the ones every stage gets wrong. Of 130 names I labelled, 103 were transcribed as themselves; the misses were the obscure ones: “Webvan” became “web ban”, “Fairfax” became “Firefox”. A sound-alike retry I built to catch them drew zero cards over 18 recordings, because the verifier’s own prompt tells it to refuse a misheard word. And a startup with no Wikipedia page gets no card at all. The rule that keeps cards faithful is the same rule that leaves it out. Lesson: Your guardrails define your coverage, so measure what they refuse, not only what they let through.


Numbers (measured)