PostgreSQL and typeorm - Caching

With most web applications you can drastically increase performance by using caching for data that's frequently read across network boundaries. This lesson will explore some common caching techniques, you'll learn how some common tools and libraries provide caching for us.

While caching helps with performance it can also cause some surprises and bugs in applications and i'll discuss some of those too.

Database course index

This is part of a full course on persistence in postgres with typeorm and sql!

There is a github repo to go with this course. See part 2 for instructions.

You can see examples of typeorm in a real app on Use Miller on GitHub

Caching benefits

If you look at data flow in an app you can see how latency is reduced when a cache is used. The example below shows how a read might be cached.

Caching is used outside of networked applications also. Computer processors make heavy use of caching for example. Its orders of magnitude faster to read from RAM Vs a hard drive, same improvement reading from a processor cache Vs computer RAM.

Caching will usually make the cost of your application infrastructure cheaper. You can keep most data on a cheap storage system and only put current data on a faster storage system.

Caching helps to smooth out unpredictable access patterns. The effect of even very short caching can be significant.

If you have something that is read heavy but changes often and you cache it for just one second, you can reduce repeated downstream requests enormously. For one popular key on one cache instance that is roughly one refresh a second, but it isn't a global guarantee: different keys, cache instances and concurrent misses all add requests. A burst of concurrent misses is called a cache stampede and sometimes needs request coalescing or a lock.

In the diagram above there are 500k r/pm to some infrastructure without a cache. This would be pretty tough for a decent RDBMS to handle but Redis could handle that cheaply. In the second diagram you can see the effect of putting a short cache in front of the RDBMS - RDBMS requests are flat and predictable.

Common caching scenarios

Here are some tools you probably use every day that use caching

DNS

The domain name system that runs the internet is based on caches. Your browser and operating system check their local caches, then normally ask a recursive resolver run by your ISP, employer or a public DNS provider. If needed that resolver queries root, TLD and authoritative nameservers to find the record.

Having a local cache for common DNS records like google.com results in much faster browser experience.

TanStack Query

TanStack Query, previously called React Query, is an excellent data retrieval library for client applications. It stores successful query results in a local cache and can reuse them to quickly display data. Whether it refreshes in the background depends on settings such as staleTime, refetch triggers and whether the query is active.

Apps with data stores

Most applications store data somewhere, often a database. Most applications are read heavy so we can increase performance and reduce the cost of the data store by utilising a cache between the application and the data store.

Common caching patterns

There are a bunch of very common caching patterns. These three are the main ones I see when building web applications.

Cache aside

Cache aside is very common. The application is aware of the cache and acts as the coordinator.

Assuming a cache miss scenario:

1. The application checks the cache for data.
2. Cache returns empty response
3. Application gets the data from the database
4. Application stores the data for the next cache retrieval

There are issues with cache aside caching. The application has to be aware of the cache and it has to be aware of the data store. The developer has to be careful to keep the cache and data store in sync.

Read through, write-through and write-behind caching

In read-through, write-through and write-behind caching the application only interacts with the cache. The cache layer then communicates with the database layer directly. The application is never aware of the database layer.

Assuming a cache read miss:

1. The application queries the cache
2. The cache has no data so it queries the datastore

The cache stores the result and returns it to the client application

Assuming a write-through cache write

1. The application writes data to the cache
2. The cache writes the data to the database before acknowledging success

A write-behind cache acknowledges first and flushes the change to the database later. It is faster for the caller but you have to deal with durability, ordering and retry failures.

These read and write patterns are not always used together. e.g. you might always read through the cache but write directly to the database. It's just easier to talk about them in one section because the principle is so similar.

Slonik's interceptor API has been used to build read-through caching. The old slonik-redis-cache example for this is still online, but it has not kept pace with current Slonik and Redis APIs. The basic shape is more useful than copying that package today:

const cached = await cache.get(cacheKey)
if (cached) return JSON.parse(cached)

const result = await database.query(query)
await cache.set(cacheKey, JSON.stringify(result), { EX: 10 })
return result

An example of write-behind pattern can be described using Redis Enterprise. Redis provides functions (previously called Redis Gears) that allow you to run server-side code with hooks into the Redis runtime. The hook for write-behind caching will propagate Redis writes to your datastore.

Some GraphQL implementations provide write-behind and read-through caching. In particular Apollo provides these caching strategies. Read more about how they work on the Apollo site.

Cache Invalidation

If your datastore has data that isn't in the cache yet then you have a cache miss. Stale data is different: the cache has a value, but the source of truth has since changed. Whether this is an issue is highly dependent on the data and your application. At some stage you will want to invalidate or replace stale records.

There are excellent articles online describing how to solve this issue so i'll just give a brief overview here. Check out https://redis.com/blog/three-ways-to-maintain-cache-consistency/ for a deep dive into cache consistency and cache invalidation.

The following are common methods used to invalidate cached data. Write-through reduces the window where the cache and database disagree, but failures and racing writes mean it isn't magically always up to date.

Time to live

Time to live invalidation is extremely common. You ask the cache to automatically expire an item after a set time period, meaning your data layer will have to go to the source to refresh the item after some set time period.

DNS is a great example of using TTL in each layer. E.g. if you lookup Google.com it may be cached in your computer and recursive resolver. This is a great way to reduce the number of requests to the authoritative nameserver.

Once the TTL is reached your DNS client, typically your browser, will go to the next layer in the chain to get the latest record.

Cache on write

For data with a high read rate and that changes infrequently, you can simply update the cache each time you write to your datastore. This invalidation strategy becomes expensive as write frequency increases.

Cache Eviction

Least recently used is an eviction method. This focuses on freeing resources, rather than keeping data fresh, by deleting from the cache data that isn’t getting read very often.

Redis supports policies including:

• noeviction
• allkeys-lru and volatile-lru
• allkeys-lfu and volatile-lfu
• random and TTL-based variants

Their usage will be highly dependent on your application.

Issues to watch out for with caching

Caching is notorious in software development for causing difficult to debug issues. For example if you're debugging an issue and the data doesn't make sense, check for caching somewhere. Caching is a form of state in these scenarios.

These are some of the things I look for when caching is present in a system.

Inappropriate load for caching

If your database is not under significant load adding a cache can have a negative effect because the latency between app -> Cache -> DB can be more than a more traditional app -> DB not under load

Caching too eagerly

If you cache many things that are not read very often then you will fill up your cache. This makes it more expensive to run and creates memory pressure, more eviction work and a worse hit rate. The number of records by itself doesn't automatically make Redis slow.

Incorrect configuration

If you cache data for too long or you don't clear the cached data at the right time then you might return stale data to a customer.

Conclusion

You'll come across caching on most apps. Tweaking the caching strategies and invalidation strategies that are used will help you to improve user experience and cost of your infrastructure.

Caching introduces complexity that has to be managed carefully.

Database course index

This is part of a full course on persistence in postgres with typeorm and sql!

There is a github repo to go with this course. See part 2 for instructions.