Data & Databases — A Developer's Field Guide

layout: cover kicker: A developer's field guide title: Data & Databases subtitle: From ACID and indexes to the whole zoo — and how to choose.

layout: agenda kicker: The map title: What we'll cover items:

layout: section index: "01" kicker: Part one title: Foundations subtitle: The guarantees you take for granted.

layout: statement kicker: Why databases exist title: A plain file can't do concurrency, integrity, durability, or search. A database can.

layout: define kicker: The core promise term: ACID definition: The guarantee that a transaction is all-or-nothing and survives crashes. points:

layout: diagram kicker: The classic race title: Two buyers, one seat build: true note: With no isolation, both reads see 1 seat — so both bookings succeed and you oversell.

sequenceDiagram
  participant A as Buyer A
  participant DB
  participant B as Buyer B
  A->>DB: SELECT seats → 1
  B->>DB: SELECT seats → 1
  A->>DB: UPDATE seats = 0
  B->>DB: UPDATE seats = 0
  Note over DB: 2 bookings, 1 seat

layout: reference kicker: Isolation levels, in plain terms title: Each level prevents one more bug groups:

layout: section index: "02" kicker: Part two title: The relational core subtitle: Done right, it solves most problems.

layout: vs kicker: Schema design title: The two moves left: title: Normalize items: - Store each fact in exactly one place - No data going out of sync - The default when you're writing data right: title: Denormalize items: - Copy data so you can skip joins - Faster reads - On purpose — for analytics or scale label: vs

layout: quote quote: Normalize till it hurts; denormalize till it works. author: Database folk wisdom

layout: define kicker: The single biggest lever term: What is an index? definition: A sorted lookup structure — usually a B-tree — so the database can jump to a row instead of reading them all. points:

layout: diagram kicker: Index title: A B-tree lookup aside: under the hood build: true highlight: [Root, M, Hit] note: Find 42 in ~3 hops — follow the glowing path, skip everything else.

flowchart TD
  Root["50 · 100"] --> L["10 · 30"]
  Root --> M["60 · 80"]
  Root --> R["120 · 160"]
  M --> Hit["42 ✓"]

layout: code-explain kicker: How is this fast? title: Read the real plan notes:

EXPLAIN ANALYZE
SELECT * FROM orders
WHERE customer_id = 42;

Seq Scan on orders  (rows=1)
  actual time=0.014..12.706
  Rows Removed by Filter: 99999
Execution Time: 12.741 ms

layout: stats kicker: What an index buys title: The same query, indexed stats:

layout: panels kicker: Curiosity title: Under the hood — four building blocks aside: deep dive panels:

layout: section index: "03" kicker: Part three title: The data zoo subtitle: Not winners — different shapes for different jobs.

layout: statement kicker: The real question title: "Eight labels — but what really differs is the shape of your data and the one thing each makes fast."

layout: feature kicker: The landscape title: Eight kinds of database columns: 4 features:

layout: panels kicker: Why NoSQL exists title: Four families, each gives up one relational feature panels:

layout: two-cols kicker: Same data, two models title: One record, two shapes

Relational — split across two tables, joined on demand.

users table

orders table

::right::

Document — one nested object, stored whole.

{
  "id": 1,
  "name": "Ada",
  "orders": [
    { "item": "Book" },
    { "item": "Pen" }
  ]
}

layout: default kicker: At a glance · the general-purpose models title: How they actually differ

layout: default kicker: At a glance · the specialized engines title: Built for one job

These don't replace your database — they sit beside it for the one job it's bad at.

layout: diagram kicker: The vector engine, end to end title: How "find similar" works aside: deep dive note: Turn text into a vector, then find the nearest vectors. The pipeline behind RAG, semantic search & recommendations.

flowchart LR
  Q[Query text] --> E[Turn into a vector]
  E --> ANN[Similarity index]
  ANN --> K[Closest matches]
  K --> R[Re-rank & return]

layout: statement kicker: A different axis title: Shape is one axis. What you do with the data is the other — and it splits databases in two.

layout: vs kicker: The deepest split title: Your app vs analytics — two completely different jobs left: title: OLTP — your app items: - Many tiny transactions - Read & write single rows - Stored row-by-row - Postgres, MySQL right: title: OLAP — analytics items: - A few huge queries - Add up millions of rows - Stored column-by-column - Snowflake, DuckDB label: vs

layout: default kicker: Storage layout title: A column store reads just one column

layout: chart kicker: The payoff title: Adding up one column note: AVG(age) across 10M rows — the kind of gap columnar storage buys. chart: type: bar unit: ms categories: [Row store, Column store] series: - { name: scan time, data: [8100, 240] }

layout: metric kicker: One number value: "34" unit: "×" ghost: "×" label: faster on that sum — column store vs row store.

layout: default kicker: The honest default title: Postgres until it hurts

One boring relational database does most of these jobs — reach for a specialist only when you've measured the pain.

Every extra datastore is one more thing to run, back up, and keep in sync. Earn it.

layout: section index: "04" kicker: Part four title: Distributed & scale subtitle: Where the trade-offs get sharp.

layout: steps kicker: Scale in order title: Don't skip a rung steps:

layout: diagram kicker: Sharding title: One router, many shards build: true highlight: [C, RT, S2] note: hash(key) picks the shard — here it lands in 34–66 → Shard B. Queries that span shards are the cost.

flowchart TD
  C[Client] --> RT{"Router · hash(key)"}
  RT -->|0–33| S1[(Shard A)]
  RT -->|34–66| S2[(Shard B)]
  RT -->|67–99| S3[(Shard C)]

layout: define kicker: The famous one term: CAP theorem definition: If the network splits in two (a partition), you can keep Consistency or Availability — not both. points:

layout: statement kicker: The honest version title: Network splits are rare — so the everyday trade-off is speed vs consistency. That's PACELC.

layout: diagram kicker: Under the hood · consensus title: How nodes agree — Raft aside: under the hood note: One leader at a time; a write is saved once a majority of nodes agree.

stateDiagram-v2
  [*] --> Follower
  Follower --> Candidate: election timeout
  Candidate --> Leader: wins majority
  Candidate --> Follower: sees higher term
  Leader --> Follower: sees higher term

layout: section index: "05" kicker: Part five title: Choosing well subtitle: And not shooting your own foot.

layout: steps kicker: How to choose title: Four questions, in order steps:

layout: panels kicker: The usual suspects title: Four common ways to shoot your foot panels:

layout: bigtype kicker: The takeaway title: There's no best database — only trade-offs you chose on purpose.

layout: end title: Thanks subtitle: Questions welcome