logtapOpen Source
OverviewQuick StartIngest APISentry CompatibilitySDKsDeployment

Overview#

LogTap is an open-source (MIT) high-throughput observability data gateway (Go + Gin + NSQ + TimescaleDB/Postgres) that brings error monitoring, structured logs, event tracking, and multi-channel alerting together in a single service, with a React console for querying and analysis.

What you can do with it#

  • Error monitoring: natively compatible with the Sentry store / envelope reporting protocols — existing Sentry SDKs work unchanged; just point the DSN at LogTap
  • Structured logs: a unified HTTP endpoint with batch ingestion (single object or JSON array) and gzip; keyword search runs on pg_trgm trigram indexes, listings on time-descending indexes
  • Event analytics: report via /track/ and the console gives you DAU/MAU, top events, property breakdowns, retention cohorts, and custom funnels
  • Health probing & alerting: built-in active HTTP/TCP probes; alerts via WeCom, DingTalk, and Feishu webhooks, SMTP email, and Alibaba/Tencent Cloud SMS, with deduplication and noise suppression
  • GeoIP resolution: an offline-bundled MaxMind GeoLite2 database resolves reporting IPs to country, city, and ASN with no external lookups

Architecture at a glance#

Ingestion and storage are decoupled by NSQ: the gateway is stateless and horizontally scalable, and slow downstream queries never block reporting.

  1. Client reporting — a Sentry SDK or any HTTP client sends POSTs; single items and batches are accepted, batches may be gzipped
  2. Gateway validates and enqueues — the Go + Gin gateway only deserializes, authenticates, and publishes to NSQ; it never touches the database, keeping latency stable
  3. Queue absorbs spikes — events and logs use separate topics and channels, so database jitter does not affect the ingest path
  4. Batch writes — consumers batch-write to TimescaleDB every 500 rows or 20 ms, trading throughput for IO efficiency

Two storage engines, pick by scale#

DimensionOpen source (PostgreSQL / TimescaleDB)Enterprise / Cloud (ClickHouse engine)
Peak ingest EPS10,000 ~ 20,000100,000 ~ 300,000+
Write P95 latency1.7 ~ 1.9 ms (constant within 120 VU)< 1.5 ms (distributed parallel ingest)
Storage compression1 : 1.5 ~ 1 : 2 (row store, B-tree)1 : 5 ~ 1 : 10 (column store, LZ4/ZSTD)
100M+ event analytics3 ~ 10 s0.1 ~ 0.5 s (vectorized engine)
Operational complexityMinimal (Postgres + Redis + NSQ)Distributed (ClickHouse cluster)

Both editions share the same API and console, so you can migrate when scale demands it. Official single-node benchmarks (32-core/64 GB, co-located): P95 write latency steady at 1.7 ~ 1.9 ms across 10 ~ 120 concurrent users, 5,600+ EPS sustained, 11,200 EPS peak.

Next steps#