Week 7 — API Design & Blob Storage

API Paradigms

	REST	GraphQL	gRPC
Protocol	HTTP/1.1	HTTP/1.1	HTTP/2
Format	JSON	JSON	Protobuf (binary)
Fetching	Fixed endpoints → fixed response	Client specifies exact fields	Strongly typed contracts
Caching	Easy (HTTP cache headers)	Hard (POST requests)	Hard
Browser support	Yes	Yes	No (needs proxy)
Best for	Public APIs, external	Mobile, flexible frontends	Internal microservices

REST Design Principles

Resource naming

# Good — nouns, plural
GET    /api/v2/invoices
POST   /api/v2/invoices
GET    /api/v2/invoices/123
PATCH  /api/v2/invoices/123
DELETE /api/v2/invoices/123

# Bad — verbs
POST /api/getInvoice
POST /api/createInvoice

Versioning

/api/v1/invoices    ← old clients still work
/api/v2/invoices    ← new clients use this

Never break existing clients by changing v1. Add v2 for breaking changes.

Idempotency Keys

Critical for financial operations — prevent duplicate processing when client retries:

POST /api/v2/payments
Idempotency-Key: 550e8400-e29b-41d4-a716-446655440000

{ "amount": 5000, "currency": "AUD" }

Server stores the key + result. Same key → return cached result, don’t process again.

Pagination

Offset Pagination (simple, avoid at scale)

SELECT * FROM orders ORDER BY id LIMIT 20 OFFSET 10000;
-- PostgreSQL must scan and skip 10,000 rows — gets slower as offset grows

Keyset / Cursor Pagination (recommended)

-- First page
SELECT * FROM orders WHERE id > 0 ORDER BY id LIMIT 20;
-- Returns last id = 5234

-- Next page
SELECT * FROM orders WHERE id > 5234 ORDER BY id LIMIT 20;
-- Instant — uses the index

Return a cursor in the response:

{
  "data": [...],
  "next_cursor": "eyJpZCI6NTIzNH0=",
  "has_more": true
}

File / Blob Storage

Never Store Large Files in PostgreSQL

Problems:

• Bloats WAL logs — slows replication
• Bloats backups — uncompressible binary data
• Fills RAM (PostgreSQL buffer pool evicts useful data)
• Slow streaming through the app server

Object Storage (S3 Pattern)

Client → App server (get pre-signed URL) → Client uploads directly to S3
Client → App server (get pre-signed URL) → Client downloads directly from S3

App server never touches the file bytes. S3 handles:

• Durability (11 nines — effectively never loses data)
• Global replication
• CDN integration
• Lifecycle policies (archive old files automatically)

Pre-signed URL: a temporary URL that allows a specific operation (upload or download) without requiring credentials. Expires after N minutes.

Practice

Design the file attachment system for an ERP:

• Users upload invoices (PDF), contracts, product images
• 500k files/day, average 2MB each = 1TB/day
• Files must be accessible for 7 years (compliance)
• Some files are confidential (access control required)

Self-check

• Why does GraphQL break HTTP caching?
• What happens if a client retries a payment without an idempotency key?
• At what scale does offset pagination become a real performance problem?