Reliability, Reconciliation and Webhooks

we have seen missed deposits at other providers when webhooks fail how do you avoid this? we don’t rely on a single notification path think of webhooks as the doorbell and the blockchain as the security camera even if the doorbell doesn’t ring, we still watch the camera two signals, one source of truth we continuously read and reconcile directly from the blockchain with our own indexers, while treating external notifications (api/webhooks) as convenience signals if a notification is delayed or dropped, our indexers still capture the deposit for extra resilience, we also cross check with secondary rpc providers (e g, alchemy) to validate data and backfill if any node lags no double counting all inputs are sequenced and all outputs are hash chained if the same event arrives twice (or late), the sequence/chain checks make duplicates harmless and detect any gap, so the state stays correct finality/confirmations & reorg safety blockchains can occasionally re organize the latest blocks we mark deposits as pending until a chain specific confirmation threshold is met (i e , enough blocks are built on top) if a rare short term reorg happens before that point, the system re processes those blocks and corrects the event sequence automatically, so you don’t see a false “final” credit think of it like card payments “pending” first, then “settled” after the network is sure (this is exactly why we reconcile against the chain rather than trusting any single push event ) what we monitor (broken down) block progress (“height drift”) are our indexers keeping up with the network’s latest blocks? if not, we alert and catch up ingest health (“lag”) how long between an on chain event and our system recording it? if that exceeds our threshold, we alert service health we track the components that fetch chain data, evaluate policy, and publish events, with runbooks for quick rollback/redeploy when needed these controls sit alongside our audit trail, which makes selective drops or edits detectable our 24×7 engineering on call monitors these signals, follows runbooks (triage → verify → fix or rollback), and communicates through your dedicated channel if needed why this works the combination of (a) reading from the blockchain itself, (b) sequencing/chain linking all messages, and (c) immutable audit logs means a missed or duplicate notification can’t cause a wrong balance or a silent drop without being caught all in all, we treat webhooks as hints, not truth the chain is the truth we continuously reconcile against it and use sequence/chain checks so duplicates or delays can’t break accounting