
Where caches live
- In the browser, for images, scripts and pages.
- At the edge, on servers close to the user that keep copies of responses.
- In the application, for the result of a slow calculation or a slow query.
- In the database itself, for data it has read recently.
The two questions
For anything you cache, answer two questions. How long may this copy be wrong? And what makes it wrong?
- Expiry by time. Keep the copy for a fixed period, then fetch again. Simple, and wrong for at most that period.
- Removal on change. Delete or update the copy when the underlying data changes. Fresh, and easy to get wrong, because every place that changes the data has to remember the cache.
What is safe to cache
| Data | Cache it? |
|---|---|
| A product description | Yes, for minutes or hours |
| A list of countries | Yes, for a long time |
| An account balance | Rarely, and never for a decision about money |
| A permission check | Carefully. A stale yes is a security problem |
| A page that differs per user | Only per user. A shared copy shows one person's data to another |
How it goes wrong
- Stale data. The price changed and the page still shows the old one.
- The wrong person's data. A response meant for one user is cached and served to the next. This is the serious one.
- The stampede. A popular entry expires and a hundred requests all rebuild it at the same moment.
- The cache hides a problem. The slow query is still slow. You only notice when the cache is empty, for example after a restart.
Questions you will hear in a meeting
- How long can this be out of date before someone notices?
- What clears it when the data changes?
- Is the cached copy different per user?
- What happens when the cache is empty?
In an interview
Adding a cache is the most common suggestion in a system design round, and the follow-up is always the same: how does it get invalidated? Have that answer ready before you say the word cache.