Smart gating

Deciding in milliseconds whether a new piece of speech is worth an LLM call.

staging version 2909a55

Benchmark for this version

Synthetic data and a mock LLM: this shows what the harness produces, not a result. The live demo is not built yet.

Scatter chart: LLM calls per minute against median time to field, one point per gate setting. The table below lists the same values.

Tail latency: 95th percentile chart · results.csv

Every gate, same test sessions

GateCalls / min× perfectPrecisionRecallGate delay p50Time to field p50p95Decision p95
always84.04.290.201.000103514300
turn_end14.20.720.770.431140222278671
every_2_words54.22.770.320.9001222204724
every_4_words30.11.540.540.504951647326423
every_8_words16.20.830.840.2915102577501324
pause_200ms79.64.070.221.00251133016621
pause_400ms52.52.680.320.51497170723011
pause_700ms31.01.580.490.001135223940391
pause_1000ms18.20.930.670.001775288578191
keyword+pause_700ms43.92.240.400.7601226265049
learned_p0.1042.62.170.420.9901038143297
learned_p0.2036.71.870.490.9801122143395
learned_p0.3535.51.810.510.9901124143593
learned_p0.5032.31.650.550.9501126163392
learned_p0.6531.31.600.570.94011281750101
learned_p0.8029.31.500.590.87011342530114
perfect (oracle)19.61.001.001.000103514301

Delays and time to field are in milliseconds; decision time is in microseconds. Recall counts changes the gate acted on within 500 ms.