Skip to content
Hi, Bot

DRAM: System Memory

The server's big, fast scratchpad. Forgets everything when the power goes off.

How big?A stick about the length of a pencil-case ruler, 13 cm.

Real: What it looks like.

Keys: arrows rotate · + / − zoom · 0 reset · 1–4 views · S signal · T tour · L labels · Space spin. Models are stylised and built from code: proportions are honest, details are simplified.

What it is

Each memory chip holds billions of tiny capacitors. A charged capacitor is a 1, an empty one is a 0. They leak, so the chip refreshes them thousands of times a second.

The stick (DIMM) plugs into a slot next to the CPU via the gold fingers along the bottom. The off-centre notch only lets it go in one way.

Why AI needs it

The CPU stages training data and checkpoints here before they move to the GPUs. An AI server often has a terabyte or more of it.

Every labeled part

  1. 1

    DRAM chip

    Billions of capacitor cells. Signal mode shows chips being read.

  2. 2

    Gold fingers

    The contacts that touch the slot. Gold doesn't corrode.

  3. 3

    Key notch

    Off-centre, so the stick can't go in backwards, and different generations can't be mixed up.

  4. 4

    Power chip

    Newer sticks regulate their own voltage on board.

  5. 5

    Heat spreader

    Aluminium plates. Explode to pop them off.

Try it · concept lab

How far away is your data?

Switch to human time to feel the gaps.

  • Register · inside the core · bytes1.0 s
  • On-chip cache (SRAM) · on the die · MBs10 s
  • HBM / DRAM · beside the chip · GBs–TBs6 min
  • NVMe SSD · in the server · TBs3 days
  • Hard drive · storage rack · PBs193 days
  • Data lake over network · across the building · PBs–EBs3.2 years

Rough, typical figures (log scale). Every step down is bigger and cheaper, and much slower.

Big idea: Fast memory is small and slow memory is big. AI performance is mostly about keeping the right numbers close to the cores.

Swap it: other ways to do the same job

  • HBM

    Several times faster, stacked next to the GPU, much smaller capacity and far more expensive.

  • CXL memory expansion

    Adds extra memory over a PCIe-style link, so servers can share or grow memory pools. A little slower.

  • SSD

    Keeps data with the power off and holds far more, but is thousands of times slower to reach.

Talk about it

  1. Q1

    This memory forgets everything when the power goes off. Why might forgetting be fine for a scratchpad?

  2. Q2

    Fast memory forgets and slow storage remembers. If a computer could only have one, which would you pick, and why?

For grown-ups: there are no right answers here. Ask a question, then ask "why do you think that?" The reasons matter more than the answer.

Printable question sheet (PDF)

Connected exhibits