Memory

Modern computer systems utilize several kinds of memory in a hierarchy of tradeoffs to balance speed, capacity, and cost.

Currently there isn’t a single memory technology that is simultaneously fast, large, and cheap. Generally, the closer the memory is to the CPU, the faster it is, but also the smaller and more expensive it is.

A basic rule of thumb is that the most expensive memory is tiny and fast, while the least-expensive is large and slow.

Register Memory: Where Computation Happens

Physically, they are tiny storage locations in the CPU (Central Processing Unit). They are expensive to manufacture and require a lot of physical space. Registers hold data, addresses, or instructions that the processor is currently working on.

• Registers are the fastest memory
• Very small size. Modern processors typically use 64-bit registers, allowing them to process larger values and memory addresses than older 32-bit systems.
• Holds the data the CPU actively uses

💡Analogy: Register memory is very fast. Similar to answering questions you know really well (your name, the city you currently live in, the name of a current pet), data stored in register memory can be accessed quickly.

Cache: So the CPU Doesn’t Have to Wait for RAM

Cache is inside the CPU. CPUs are much faster than RAM; cache stores frequently used data so the CPU doesn’t have to wait. It reduces the time needed to fetch commonly used data from RAM because it is a grid of transistors built directly into the CPU.

There are several types of cache. Some of the distinct forms:

• SRAM: Static RAM is extremely fast and holds data as long as it has power. Small amounts are used for cache because it is expensive.
  • In contrast, DRAM (Dynamic RAM) isn’t used as cache; it is used as a computer’s main system memory and is plugged in to the motherboard. Compared to SRAM, DRAM is cheap and compact.
• L1, L2, and L3
  • L1 (Level 1): The fastest and smallest, it is usually located inside the CPU core. Physically it is difficult to see as it microscopic and takes up small space on the chip.
  • L2 (Level 2): L2 is larger and slightly slower than L1, attached to a specific core.
  • L3 (Level 3): L3 is the largest type, shared between CPU cores.

💡Analogy: Cache memory is similar to notes you might have open on your laptop. Looking down at your laptop’s screen to answer a question is fairly fast, but not as fast as information that’s in your mind.

RAM: Random Access Memory

Random access memory (RAM) is the main working memory in a computer. It is larger than cache and volatile. Volatile data disappears when the power is turned off. Most main memory is DRAM; it’s cheap, dense, and requires constant refreshing. RAM stores programs and data that is currently in use. For example, if you are playing a game on your computer, the code and data is loaded into RAM. The more RAM a machine has, the more programs that can be used simultaneously.

When RAM fills up, the operating system uses virtual memory (sometimes called a page file or swap space). Virtual memory is useful, but is slower than RAM. Excessive use can slow a computer down so it feels less responsive.

• Memory Addresses: Each byte in RAM has an address that programs use to store and locate data. An address works much like a street address. It tells the CPU exactly where data is located in memory.
• Stack and Heap: When a function creates a local variable, it is usually stored on the stack. When a program creates an object whose lifetime extends beyond the current function, it is typically allocated on the heap. Heap memory can be manually managed in languages such as C and C++, while languages such as C# automatically reclaim unused heap memory through garbage collection.
• Memory Leaks: A memory leak occurs when a program allocates memory but never releases it. It increases RAM usage, reduces performance, and may cause crashes.

💡Analogy: RAM is similar to files you might have on your laptop that aren’t currently open, but that you could access quickly with a few clicks.

ROM: Read Only Memory

Read only memory (ROM) stores instructions that run before the operating system loads. ROM starts a computer, such as BIOS or UEFI firmware. Modern systems often use flash-based ROM.

Storage: SSD, HDD, and Network (Cloud)

Storage is often called memory, but there are significant differences between storage and memory such as cache and RAM.

SSD and HDD

• A solid state drive (SSD) is slower than RAM. It keeps data stored even when the power is off, and uses flash memory.
• A hard disk drive (HDD) is a non-volatile, electro-mechanical data storage device. It has magnetic discs and is slower than SSDs, but generally cheaper.

💡Analogy: Storage such as a SSD or a HDD is like information in a book on a bookshelf. It’s still fairly quick to retrieve, but not the fastest.

Network Storage (Cloud)

Network storage saves data on a separate machine, usually not in the same physical location. For example, when you save data to a service such as Google Drive, iCloud, Dropbox, or Microsoft’s OneDrive, you are saving data on remote servers accessible over the internet. As long as you have an internet connection and your login, you can access files – even if you aren’t on your own computer.

💡Analogy: Storage on a network (cloud storage) is like going to a neighbor’s house or a library to borrow a book.

Summary

Memory is a hierarchy of tradeoffs between speed, size, and cost.

Key Points

• Registers hold data, addresses, or instructions that the processor is currently working on. They are the fastest memory, and are inside the CPU.
• Cache speeds up access to frequently used data.
• RAM stores programs and data currently in use.
  • Every byte in RAM has an address.
  • Virtual memory uses storage to extend RAM (but is slower than RAM).
• ROM stores firmware (startup instructions) that runs before the operating system loads.
  • Memory allocation has two primary distinctions: the stack and the heap.
    • The stack is small and fast managed automatically by the CPU.
    • The heap is larger and used for dynamic allocations.
• The CPU fetches instructions from software and hardware, decodes, and then executes them. It performs basic arithmetic, logic, and I/O (input/output) operations.
  • Cores: processing units.
  • Threads: virtual versions of cores that let the operating system run tasks in parallel.
  • Clock speed: how many cycles the CPU can execute per second.
  • Cache: high-speed memory built into the CPU that stores frequently accessed data for fast retrieval.
• Volatile memory: loses contents without power. Examples: registers, cache, and RAM.
• Non-volatile memory: retains contents even without power. Examples: SSD, HDD, USB drives.