[{"data":1,"prerenderedAt":541},["ShallowReactive",2],{"blog-post-en-014_data-integrity-nightmares-105-percent":3,"blog-translations-014_data-integrity-nightmares-105-percent":530},{"id":4,"title":5,"body":6,"date":514,"description":515,"draft":516,"extension":517,"image":518,"meta":519,"navigation":176,"path":520,"publishDate":514,"readingTime":97,"seo":521,"stem":522,"tags":523,"translationKey":528,"__hash__":529},"blog_en/blog/en/014_data-integrity-nightmares-105-percent.md","When Your Analytics Dashboard Shows 105%",{"type":7,"value":8,"toc":508},"minimark",[9,18,28,31,36,39,101,104,107,111,114,120,123,134,144,150,211,214,220,223,227,230,361,364,367,453,456,460,467,470,473,476,483,486,489,492,504],[10,11,12,13,17],"p",{},"I ran the analytics dashboard and it told me 105% of users had used dictation. The dashboard was bash and ",[14,15,16],"code",{},"jq",", four components, nothing fancy. The number was impossible.",[19,20,25],"pre",{"className":21,"code":23,"language":24},[22],"language-text","πŸ“ˆ Feature Distribution\n────────────────────────────────────────\nDictation: 105%\nVoice AI: 11%\n","text",[14,26,23],{"__ignoreMap":27},"",[10,29,30],{},"105%. Not 100%. Not \"approximately all.\" One hundred and five percent of users used dictation. More dictation sessions than total sessions. I stared at the terminal for a solid thirty seconds before my brain accepted what my eyes were seeing.",[32,33,35],"h2",{"id":34},"the-data-that-broke-math","The Data That Broke Math",[10,37,38],{},"The raw numbers:",[19,40,44],{"className":41,"code":42,"language":43,"meta":27,"style":27},"language-json shiki shiki-themes github-dark github-dark","{\n \"totalSessions\": 17,\n \"totalDictationSessions\": 18,\n \"totalVoiceAISessions\": 2\n}\n","json",[14,45,46,55,71,84,95],{"__ignoreMap":27},[47,48,51],"span",{"class":49,"line":50},"line",1,[47,52,54],{"class":53},"suv1-","{\n",[47,56,58,62,65,68],{"class":49,"line":57},2,[47,59,61],{"class":60},"s8ozJ"," \"totalSessions\"",[47,63,64],{"class":53},": ",[47,66,67],{"class":60},"17",[47,69,70],{"class":53},",\n",[47,72,74,77,79,82],{"class":49,"line":73},3,[47,75,76],{"class":60}," \"totalDictationSessions\"",[47,78,64],{"class":53},[47,80,81],{"class":60},"18",[47,83,70],{"class":53},[47,85,87,90,92],{"class":49,"line":86},4,[47,88,89],{"class":60}," \"totalVoiceAISessions\"",[47,91,64],{"class":53},[47,93,94],{"class":60},"2\n",[47,96,98],{"class":49,"line":97},5,[47,99,100],{"class":53},"}\n",[10,102,103],{},"Eighteen dictation sessions out of seventeen total sessions. This is the kind of number that makes you question your entire analytics pipeline. Is the denominator wrong? The numerator? Both? Am I dividing by the wrong thing? Did I accidentally deploy a counter that counts backwards?",[10,105,106],{},"The temptation was to cap it at 100% in the display code and move on. I have shipped that kind of fix before. But if the data was wrong here, it was wrong everywhere.",[32,108,110],{"id":109},"it-felt-like-a-script","It Felt Like a Script",[10,112,113],{},"The telemetry stack is four components. You could sketch it on a napkin:",[19,115,118],{"className":116,"code":117,"language":24},[22],"Yakki (macOS app)\n ↓ batched events\nCloudflare Worker\n ↓ aggregated counters\nKV Store\n ↓ API\nBash dashboard\n",[14,119,117],{"__ignoreMap":27},[10,121,122],{},"The dashboard code was straightforward division. If the output was wrong, the data was wrong. So I started at the source.",[10,124,125,129,130,133],{},[126,127,128],"strong",{},"Hypothesis 1: The dashboard math is broken.","\nI checked. ",[14,131,132],{},"18 / 17 * 100 = 105.88",". The math was fine. The data was the problem.",[10,135,136,139,140,143],{},[126,137,138],{},"Hypothesis 2: The client is sending duplicate events.","\nI added logging to the macOS app's telemetry batch sender. Each session generated exactly one ",[14,141,142],{},"session_start"," event with a unique session ID. No duplicates on the client side.",[10,145,146,149],{},[126,147,148],{},"Hypothesis 3: The Worker is counting things twice.","\nI opened the Cloudflare Worker code and found the event handler:",[19,151,155],{"className":152,"code":153,"language":154,"meta":27,"style":27},"language-javascript shiki shiki-themes github-dark github-dark","async function handleSessionEvent(event, env) {\n const stats = await getStats(env);\n stats.totalSessions++;\n\n if (event.feature === 'dictation') {\n stats.totalDictationSessions++;\n }\n\n await saveStats(env, stats);\n}\n","javascript",[14,156,157,162,167,172,178,183,189,195,200,206],{"__ignoreMap":27},[47,158,159],{"class":49,"line":50},[47,160,161],{},"async function handleSessionEvent(event, env) {\n",[47,163,164],{"class":49,"line":57},[47,165,166],{}," const stats = await getStats(env);\n",[47,168,169],{"class":49,"line":73},[47,170,171],{}," stats.totalSessions++;\n",[47,173,174],{"class":49,"line":86},[47,175,177],{"emptyLinePlaceholder":176},true,"\n",[47,179,180],{"class":49,"line":97},[47,181,182],{}," if (event.feature === 'dictation') {\n",[47,184,186],{"class":49,"line":185},6,[47,187,188],{}," stats.totalDictationSessions++;\n",[47,190,192],{"class":49,"line":191},7,[47,193,194],{}," }\n",[47,196,198],{"class":49,"line":197},8,[47,199,177],{"emptyLinePlaceholder":176},[47,201,203],{"class":49,"line":202},9,[47,204,205],{}," await saveStats(env, stats);\n",[47,207,209],{"class":49,"line":208},10,[47,210,100],{},[10,212,213],{},"Clean. Simple. And hiding a subtle distributed systems bug.",[10,215,216,219],{},[126,217,218],{},"The actual problem:"," Cloudflare Workers can receive the same request more than once. Network retries. Edge-location failovers. The client's exponential backoff after a timeout that wasn't actually a failure. Any of these can cause a single session event to be processed multiple times.",[10,221,222],{},"When that happens, the counters diverge. The feature counter increments. The total counter might not. Which duplicate hits which edge location, which KV write wins the eventual consistency race, that determines whether your math is possible or not.",[32,224,226],{"id":225},"the-fix-and-why-order-matters","The Fix (And Why Order Matters)",[10,228,229],{},"The solution was session-level deduplication. Before incrementing any counter, check if this session has already been counted:",[19,231,233],{"className":152,"code":232,"language":154,"meta":27,"style":27},"async function handleSessionEvent(event, env) {\n const sessionKey = `counted_session_${event.sessionId}`;\n const alreadyCounted = await env.TELEMETRY_KV.get(sessionKey);\n\n if (alreadyCounted) {\n // Already processed this session.\n // Return success so the client doesn't retry.\n return;\n }\n\n // Mark as counted with a 24-hour TTL\n await env.TELEMETRY_KV.put(sessionKey, 'true', {\n expirationTtl: 86400\n });\n\n // Now safe to increment\n const stats = await getStats(env);\n stats.totalSessions++;\n\n if (event.feature === 'dictation') {\n stats.totalDictationSessions++;\n }\n\n await saveStats(env, stats);\n}\n",[14,234,235,239,244,249,253,258,263,268,273,277,281,287,293,299,305,310,316,321,326,331,336,341,346,351,356],{"__ignoreMap":27},[47,236,237],{"class":49,"line":50},[47,238,161],{},[47,240,241],{"class":49,"line":57},[47,242,243],{}," const sessionKey = `counted_session_${event.sessionId}`;\n",[47,245,246],{"class":49,"line":73},[47,247,248],{}," const alreadyCounted = await env.TELEMETRY_KV.get(sessionKey);\n",[47,250,251],{"class":49,"line":86},[47,252,177],{"emptyLinePlaceholder":176},[47,254,255],{"class":49,"line":97},[47,256,257],{}," if (alreadyCounted) {\n",[47,259,260],{"class":49,"line":185},[47,261,262],{}," // Already processed this session.\n",[47,264,265],{"class":49,"line":191},[47,266,267],{}," // Return success so the client doesn't retry.\n",[47,269,270],{"class":49,"line":197},[47,271,272],{}," return;\n",[47,274,275],{"class":49,"line":202},[47,276,194],{},[47,278,279],{"class":49,"line":208},[47,280,177],{"emptyLinePlaceholder":176},[47,282,284],{"class":49,"line":283},11,[47,285,286],{}," // Mark as counted with a 24-hour TTL\n",[47,288,290],{"class":49,"line":289},12,[47,291,292],{}," await env.TELEMETRY_KV.put(sessionKey, 'true', {\n",[47,294,296],{"class":49,"line":295},13,[47,297,298],{}," expirationTtl: 86400\n",[47,300,302],{"class":49,"line":301},14,[47,303,304],{}," });\n",[47,306,308],{"class":49,"line":307},15,[47,309,177],{"emptyLinePlaceholder":176},[47,311,313],{"class":49,"line":312},16,[47,314,315],{}," // Now safe to increment\n",[47,317,319],{"class":49,"line":318},17,[47,320,166],{},[47,322,324],{"class":49,"line":323},18,[47,325,171],{},[47,327,329],{"class":49,"line":328},19,[47,330,177],{"emptyLinePlaceholder":176},[47,332,334],{"class":49,"line":333},20,[47,335,182],{},[47,337,339],{"class":49,"line":338},21,[47,340,188],{},[47,342,344],{"class":49,"line":343},22,[47,345,194],{},[47,347,349],{"class":49,"line":348},23,[47,350,177],{"emptyLinePlaceholder":176},[47,352,354],{"class":49,"line":353},24,[47,355,205],{},[47,357,359],{"class":49,"line":358},25,[47,360,100],{},[10,362,363],{},"The 24-hour TTL is important. Without it, the KV store accumulates session keys forever. With it, old keys expire naturally, and the storage stays bounded. Twenty-four hours is generous enough to catch any retry window.",[10,365,366],{},"But I also learned something about defensive analytics: never trust your own data pipeline completely. So I added validation to the dashboard itself:",[19,368,372],{"className":369,"code":370,"language":371,"meta":27,"style":27},"language-bash shiki shiki-themes github-dark github-dark","if [ \"$DICTATION_SESSIONS\" -gt \"$TOTAL_SESSIONS\" ]; then\n echo \"⚠️ Data Integrity Issue Detected\"\n echo \" Dictation sessions ($DICTATION_SESSIONS) > Total ($TOTAL_SESSIONS)\"\n echo \" Possible cause: duplicate event processing\"\n echo \" Run: ./scripts/repair-counters.sh\"\nfi\n","bash",[14,373,374,409,417,434,441,448],{"__ignoreMap":27},[47,375,376,380,383,387,390,392,395,398,401,403,406],{"class":49,"line":50},[47,377,379],{"class":378},"sOPea","if",[47,381,382],{"class":53}," [ ",[47,384,386],{"class":385},"s4wv1","\"",[47,388,389],{"class":53},"$DICTATION_SESSIONS",[47,391,386],{"class":385},[47,393,394],{"class":378}," -gt",[47,396,397],{"class":385}," \"",[47,399,400],{"class":53},"$TOTAL_SESSIONS",[47,402,386],{"class":385},[47,404,405],{"class":53}," ]; ",[47,407,408],{"class":378},"then\n",[47,410,411,414],{"class":49,"line":57},[47,412,413],{"class":60}," echo",[47,415,416],{"class":385}," \"⚠️ Data Integrity Issue Detected\"\n",[47,418,419,421,424,426,429,431],{"class":49,"line":73},[47,420,413],{"class":60},[47,422,423],{"class":385}," \" Dictation sessions (",[47,425,389],{"class":53},[47,427,428],{"class":385},") > Total (",[47,430,400],{"class":53},[47,432,433],{"class":385},")\"\n",[47,435,436,438],{"class":49,"line":86},[47,437,413],{"class":60},[47,439,440],{"class":385}," \" Possible cause: duplicate event processing\"\n",[47,442,443,445],{"class":49,"line":97},[47,444,413],{"class":60},[47,446,447],{"class":385}," \" Run: ./scripts/repair-counters.sh\"\n",[47,449,450],{"class":49,"line":185},[47,451,452],{"class":378},"fi\n",[10,454,455],{},"The dashboard now tells me when the data smells wrong, instead of silently rendering impossible numbers as though they were normal.",[32,457,459],{"id":458},"what-this-taught-me","What This Taught Me",[10,461,462,463,466],{},"Distributed counters are a solved problem, but only if you know you have one. When you are writing a Cloudflare Worker that increments a number in KV, it doesn't feel like a distributed system. It feels like ",[14,464,465],{},"counter++",". That's the trap.",[10,468,469],{},"I should have made the event handler idempotent from day one. A function that assumes it will be called exactly once is a function that hasn't met a network. Every handler should be safe to call twice with the same input. Not as good practice, but as a survival requirement.",[10,471,472],{},"I should have built validation into the dashboard itself. Before rendering a percentage, check if it's possible. Before displaying a count, check if it's positive. The dashboard should have been the last line of defense, not a dumb pipe from database to terminal.",[10,474,475],{},"I should have written a repair script before I needed one. When bad data gets in (and it will) you need a way to fix it without redeploying anything. A script that recalculates aggregates from raw events will save you at 2 AM when nothing else can.",[10,477,478,479,482],{},"And I should have had alerts on impossible states. If ",[14,480,481],{},"feature_sessions > total_sessions",", something is broken. I want to know immediately, not three weeks later when I happen to glance at a dashboard.",[10,484,485],{},"When you are a solo developer with 20 users, you might think these problems don't apply to you. The moment you put a Worker at the edge and a KV store behind it, you have a distributed system. It doesn't matter that your total traffic fits in a single HTTP request log. The failure modes are the same.",[10,487,488],{},"A counter incremented by 18 events across 3 edge locations over 2 hours has the same consistency challenges as one processing millions. The math doesn't care about scale. Neither do the decisions you made while the numbers were wrong.",[490,491],"hr",{},[10,493,494],{},[495,496,497,498,503],"em",{},"This is part of an ongoing series about building Yakki, a macOS dictation app. Want to see how the telemetry system itself was built? Read ",[499,500,502],"a",{"href":501},"007-privacy-first-telemetry","Telemetry That Respects Your Users",": the opt-in, privacy-first analytics pipeline that produced these (eventually accurate) numbers.",[505,506,507],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .suv1-, html code.shiki .suv1-{--shiki-default:#E1E4E8;--shiki-dark:#E1E4E8}html pre.shiki code .s8ozJ, html code.shiki .s8ozJ{--shiki-default:#79B8FF;--shiki-dark:#79B8FF}html pre.shiki code .sOPea, html code.shiki .sOPea{--shiki-default:#F97583;--shiki-dark:#F97583}html pre.shiki code .s4wv1, html code.shiki .s4wv1{--shiki-default:#9ECBFF;--shiki-dark:#9ECBFF}",{"title":27,"searchDepth":73,"depth":73,"links":509},[510,511,512,513],{"id":34,"depth":57,"text":35},{"id":109,"depth":57,"text":110},{"id":225,"depth":57,"text":226},{"id":458,"depth":57,"text":459},"2026-03-30","How a one-line counter increment hid a distributed systems bug. A debugging story about Cloudflare Workers, KV Store eventual consistency, and why idempotent event handlers aren't optionalβ€”even at small scale.",false,"md",null,{},"/blog/en/014_data-integrity-nightmares-105-percent",{"title":5,"description":515},"blog/en/014_data-integrity-nightmares-105-percent",[524,525,526,527],"engineering","debugging","cloudflare","telemetry","data-integrity-nightmares-105-percent","ZQg_SiDGuTMZKZLyDneBSZjEqQZHLlBvfNql_hu9lZg",[531,536],{"code":532,"name":533,"title":534,"path":535},"es","EspaΓ±ol","Cuando tu panel de analΓ­ticas marca un 105%","/es/blog/014_data-integrity-nightmares-105-percent",{"code":537,"name":538,"title":539,"path":540},"fr","FranΓ§ais","Quand Votre Dashboard Affiche 105%","/fr/blog/014_data-integrity-nightmares-105-percent",1775207564260]