# Designing a System for 1M Concurrent Users

Architecture deep dive for massive scale. Load balancing, caching, database sharding.


**Model:** claude-opus-4
**Created:** 7/30/2026

---

## Conversation

### **Human**

I need to design a system that handles 1 million concurrent users. What are the key things I need to think about?

---

### **Assistant**

This is where architecture gets really interesting! Let's design this systematically:

PHASE 1: Understand the load.

1M concurrent users does not equal 1M requests per second.
Active users might make 1-10 requests per minute each. But spikes, rush hour, and flash events matter.

Assumptions: Average 10 requests per minute per user. Peak 100 requests per minute per user (10x). Peak load equals 100M requests per minute equals 1.6M requests per second.

PHASE 2: Architecture layers.

User goes to CDN (static content), goes to Load Balancer (L7 LB), goes to API Servers (auto-scale), goes to Cache (Redis), goes to DB (sharded).

PHASE 3: Key design decisions.

DATABASE DESIGN: Sharding (split data across servers), Read replicas (scale reads independently), Connection pooling (don't exhaust connections).

CACHING STRATEGY: CDN for static content, Application cache (Redis) for hot data, Database cache (built-in query cache).

RATE LIMITING: Per-IP limits (prevent abuse), Per-user limits (fairness), Token bucket algorithm (smooth limiting).

Want me to design any specific component in detail?

---

### **Human**

Yes! How do I design the database sharding? What should I shard by?

---

### **Assistant**

Sharding is both art and science! Let's design it properly:

SHARDING STRATEGIES:

1. HASH-BASED SHARDING (recommended): Use consistent hashing. User ID gets hashed, then modulo by number of shards.

Why this is good: Even distribution, Deterministic (same user always goes to same shard), Easy to add more shards.

2. RANGE-BASED SHARDING: User IDs 1-1000000 go to Shard 1, 1000001-2000000 go to Shard 2.

Tradeoffs: Easy range queries, but Hotspots (new users go to same shard), Hard to rebalance.

3. GEOGRAPHICAL SHARDING: Route to closest data center. US users go to US shards, EU users go to EU shards.

RECOMMENDED SETUP:
Shard 0: US-West (users 0-2499999)
Shard 1: US-East (users 2500000-4999999)
Shard 2: EU-West (users 5000000-7499999)
Shard 3: EU-Central (users 7500000-9999999)

Critical: Shard by user_id, not by session or request.

Want me to design the cross-shard query strategy?

---


---

*Downloaded from BranchLM - GitHub for AI Sessions*