Last updated:
During a blocklist update at 08:14 UTC, the XDP program reload introduced a 340ms latency spike on affected interfaces. Root cause: oversized blocklist map causing map resize during reload. Fix: pre-allocated map at 2× capacity. Resolved within 6 minutes. No traffic dropped.
A subset of users reported TOTP codes rejected at the 30-second boundary window when device clock skew exceeded ±15s. Root cause: strict server-side validation window. Fix: extended acceptance window and added NTP sync check. Deployed without downtime.
A brief period of SMS delivery delays (15–40 minutes) affecting EU-region numbers between 14:22–15:47 UTC. Root cause: upstream carrier rate limiting. Mitigation: added secondary carrier failover. Fully resolved at 15:47 UTC.
Mission queue processing slowed by ~300ms per item during a Redis cluster rebalance between 09:11–09:43 UTC. No data loss. Cluster rebalance completed normally.
Prometheus alertmanager and synthetic probes fire within 60 seconds of degradation. On-call rotation is 24/7 with a 5-minute acknowledgement SLA.
This page is updated before any other channel. Incidents are created within 10 minutes of detection with an initial assessment. Updates follow every 30 minutes until resolution.
Every incident above Minor severity gets a written post-mortem: timeline, root cause, contributing factors, and specific remediation steps with owners and deadlines.
Primary infrastructure on Hetzner (DE/FI). All data within the EU. No hyperscaler lock-in. We own the hardware relationship — no surprise bills, no arbitrary account suspensions.
Prometheus + Alertmanager, Grafana, Loki for log aggregation, and external synthetic probes from three EU locations. Pagerduty on-call with 5-minute acknowledgement SLA for all P1 alerts.
Uptime calculated as percentage of 1-minute synthetic probe checks returning a successful response. A probe failure counts as 1 minute of downtime. No smoothing or rounding applied.
Critical: service unavailable. Major: significantly degraded. Minor: partial degradation or single-region impact. Informational: maintenance or advisory.
Get notified by email when incidents are created, updated, or resolved.
Planned maintenance is scheduled on Sundays between 02:00–06:00 EET. We post notices at least 48 hours in advance on this page and notify affected accounts by email. Maintenance rarely affects all services simultaneously — individual services may show degraded status while others remain fully operational.
Where 47Network services depend on third-party infrastructure (carrier networks for 47Comms, DNS resolvers, Let's Encrypt), we report upstream outages separately from our own. Upstream incidents outside our control are labelled as such — we don't claim credit for fixing something we didn't break.
Incident history is retained for 24 months. Post-mortem reports are published permanently and linked from the incident entry. We do not remove or alter historical incident records — if we got something wrong in an initial assessment, the update is appended, not substituted.
If you're experiencing an outage not reflected on this page, email status@the47network.com. Studio retainer clients have a dedicated on-call number. We acknowledge all reports within 30 minutes during business hours (09:00–19:00 EET, Monday–Friday).
The Grafana panels and PromQL queries behind our status monitoring — p95 latency, error rate, and the dashboard design that makes incidents readable under pressure.
Every 47Network service ships logs to Loki — label design, LogQL queries, and how log lines correlate with Prometheus metrics to complete the incident picture.
How we automate certificate renewal for all products — DNS-01 wildcard certs, auto-reload hooks, and the Prometheus alert that fires 30 days before expiry.