OmniScale Enterprise Cloud Gateway
High-Throughput Multi-Region API Router & Microservice Mesh
The Business & Technical Problem
Monolithic reverse proxies collapsed under flash-sale traffic surges exceeding 25,000 requests per second. Gateway CPU starvation drove P99 routing latency over 250ms, triggering cascading timeouts and service outages across 40+ downstream microservices.
Delivered Scope & Responsibilities
Architected multi-region API Gateway and service mesh on AWS EKS across us-east-1, eu-west-1, and ap-southeast-1.
Wrote custom Envoy Proxy filters in Go for dynamic header routing, rate limiting, and token validation.
Configured Cloudflare Anycast edge routing and DNS failover policies.
Designed global Redis cluster for distributed token bucket rate limiting and session blocklists.
Decentralized JWT validation to edge proxies using asymmetric RSA-256 keys.
Authored 100% infrastructure-as-code via Terraform modules with zero-downtime canary deployments.
Implemented Cilium eBPF network tracing with Prometheus and Grafana dashboards.
Architecture Decisions, Rejected Alternatives & Accepted Trade-Offs
Envoy Proxy with Custom Go Filter Extensions instead of NGINX / Kong
Envoy's non-blocking dynamic reloads, built-in circuit breaking, and sub-millisecond thread pool latency handled heavy concurrency without connection drops.
Kong was rejected due to Lua memory overhead; standard NGINX was rejected due to static configuration reload connection resets.
Steeper configuration learning curve and complex Envoy xDS control plane management.
Edge-Level Asymmetric JWT Validation
Validating cryptographic signatures directly at ingress proxies removed a 30,000 RPS authentication bottleneck on the internal auth microservice.
Synchronous HTTP auth-check calls for every incoming request.
Token revocation required maintaining a distributed Redis blocklist.
Measured Project Outcomes
Every metric specifies the measurement environment, method, and Abin's contribution. Unverified benchmarks are not presented as contractual SLAs.
| Metric Name | Baseline | Final Value | Environment | Measurement Method | Abin's Contribution |
|---|---|---|---|---|---|
| Sustained Peak Throughput | 25,000 RPS (system choked) | 50,000+ RPS | Benchmark | Distributed k6 load generators across 5 VPC worker nodes(Q4 2024) | Architected horizontal pod autoscaling and connection keep-alive pools. |
| P99 Gateway Routing Latency | 260ms | < 8ms | Benchmark | Envoy access logs and Prometheus latency histograms(Q4 2024) | Eliminated synchronous auth hops and tuned kernel TCP socket buffers. |
| Infrastructure Cloud Expense | $42,000 / month | 35% reduction (~$14,700/mo saved) | Staging | AWS Cost Explorer comparison before and after architecture rollout(Q1 2025) | Right-sized EKS node groups and eliminated over-provisioned standalone proxy instances. |
Evidence & Confidentiality Notice
Client details and proprietary network diagrams are omitted under confidentiality. The technical description has been generalized with client permission.
- The 50,000+ RPS benchmark was achieved on synthetic HTTP payloads with keep-alive connections; payload inspection of multi-megabyte multipart uploads introduces additional latency.
- Cost savings reflect compute right-sizing on AWS EKS and may vary on alternative cloud providers.
Discuss High-Throughput Cloud Architecture
Planning a multi-region API Gateway, microservice migration, or cloud infrastructure overhaul?
Technology Stack & Tools Used
Let's Build Your Custom Web Application
Whether you need a full-stack web app, a Node.js REST API, a Next.js SaaS MVP, an admin dashboard, or performance optimization for an existing application — I am available for freelance projects in Kerala, India, and worldwide.