Improving UX RESTfully with HTTP/1.1 cache injection

Improving UX RESTfully with HTTP/1.1 cache injection

The Motivation

In the end, the user experience will almost completely be converted from: “please wait, loading” to: “blazing fast, I am in the future and my connection has 0ms latency”.

Do you work with an application querying data via http? Then this article might be just for you. With the technique shown here you will be able to make your REST-based app literally fly, both in terms of speed and in offline operation. Though the words “http”, “cache” might make you raise your eyebrows (who never got frustrated by the browser using cached data when it shouldn’t during web development raise your hand), worry not: we’ll circumvent that issue elegantly.

AirplaneCouchRest

Requirements: Familiarity with Couchbase Lite and Sync gateway (if not, just read my previous articles), notion of HTTP protocol, and a need for speed 😉 

TL;DR: If you want to skip the explanation, go directly to the implementation section.

Auxiliary Material: I’ve made a demo project to prove the concept so you can try it for yourself. It is based on iOS but the concepts can be applied to other platforms. It is available for download in Github under the project named “HttpSmartCache“: link. You’ll as well see how Couchbase’s concept of channels can be applied to an application, which wasn’t so far covered by the previous articles.

 

Introduction: Getting ready for take-off

While building HTTP network-dependent apps (most cases), we rely on requests coming and going through the internet over and over as their resources are required.

While it is true that for certain actions (say, reserving a hotel room) we must communicate with the server in a centralised, “real time” manner (think consistent, atomic, non-idempotent methods), for many other operations, it is possible to drop a lot of the latency of the request-response model (mostly GET requests or idempotent methods) by making a small shift in the way we think or deal about distribution of information (the focus of this article).

Just to see how small ideas can cause big impacts, let’s take a look on something that was implemented in version 1.1 of the HTTP protocol itself. As an example, take the request-response done before (left), and a ‘smarter’ one on the right:

600px-HTTP_pipelining2.svg

Here we see how the total operation time is (at least theoretically) drastically reduced by removing the dependency of waiting for a response before asking for the next one. I say theoretically because although there are reports of significant improvements (sometimes twice as fast), in real life this is more easily accomplished simply by packing all data you need in one response (once you better understand your app data’s dependencies). Why not send all related data at once anyway if we know it will be needed?

The problem highlighted above is being better improved in the upcoming HTTP/2 by means of multiplexing, server push, server control and other techniques. But better than any transport layer improvement, let us remember that nothing beats the speed of local data. So, if our server could push data it knows that has high probability of being requested, depending for example on navigational context (see below), there’s nothing than can be faster/better than that. That’s one of the main reason users prefer apps than websites in the first place (its usually faster and to the point). We can identify data which is probably going to be needed in many ways, one of them is contextual navigation, data surrounding other data elements:

websiteImage

And when the user enters another screen, the same idea can be applied to it and on and on recursively. In fact, one could pre-load the most accessed pages of the app, depending on how often (statistically) the content is accessed, reducing the stateless requests to the server to the minimum possible, e.g., one or two tcp connection(s):

CachedRequests

 

Furthermore, when done correctly, by aggregating (coalescing) data to be transferred in bulks, we save energy, reduce bandwidth usage, and increase throughput:

Screen Shot 2015-03-02 at 11.14.06 PMScreen Shot 2015-02-27 at 4.20.49 PM

The trick is that by balancing this (too much pre-loaded data is wasteful, too little will cause too many request-resposes) the app becomes blazing fast while saving round trips (latency) to the server. Instead of firing multiple requests for each individual content, pre-loaded data can be downloaded from just one connection and served from local memory when needed.

The technique shown here, which surrounds the idea of “having data beforehand”, when combined with server-push, can improve a lot the interaction an user has with your REST based application, that is, still using your familiar REST (Representational State Transfer) API’s. We don’t need to wait for HTTP/2 or some other standard to get adopted, neither we need to completely re-write our apps or abandon REST: by combining technologies, we can code this effect ourselves all with existing open source tools and your existing infra-structure. Depending on your application, it might not be possible to optimize everything, but the good thing is you can apply it just to a small part and incrementally improve other parts the more you come to understand how your data behaves (instead of just querying for it *every time*).

In this article we will use a standard couchbase server+sync gateway installation connected to couchbase lite on the mobile side. With these components in place, only one or two additional functions will be called at the mobile to make it all work seamlessly for most http requests your app needs. Basically, any get request you make can become instantaneous while presenting current / relevant data (with no fear of http caches going out of control or stale).

Following we analyse the challenge, and later we implement a solution.

 

A quick analysis of caching in the most common web protocol of today (HTTP/1.1)

So how can we make an HTTP request be fast? We provide it cache data. We knew this for a long time. But, while its true that HTTP has cache support, we still depend on an initial and successful request-response cycle to have any caching system to run. That is, under normal circumstances, we require a network connection to bootstrap the caching system for each URL Request. Once the first request is made and a cached response exists, it continues with the normal flow:

 

The problem

HTTP v1.1’s caching per se has the following limitations:

1- Not always data is to be refreshed based on time expiration (generally the data changes as side-effect of another event) and

2- Although mechanisms such as eTags avoids transfer of unmodified data, it often still requires a round trip to the server (to make that HEAD request) to check whether data changed.

These two basic problems blocks us to do efficient caching, without compromising freshness of ‘dynamic data’: we don’t know whether a response for a request with certain parameters have actually changed until we run another request-response cycle: simply relying on time-stamps to know whether a response should be re-used or re-feched  doesn’t allow us to have some more dynamic/smart behaviour with our cache.

In other words, although HTTP caching protocols are well defined and implemented by the Foundation URL Loading System, caching won’t be performed if the app can’t establish a connection to the data source in the first place (the request must be successful, with a status code in the 200–299 range, along with the proper http caching response headers – see ref. documentation link). In fact, at least under my tests, e.g. when there are parameters in a GET request by default the URL Loading system will *always* try contact the server even if cached data exists for a same previous request which had a response with  Cache-Control header allowing it to do so (i.e., not stale, and no ‘no-cache’ field present). Also, simply overriding the cache policies won’t help either because then the app will then always be dealing with stale information. The end result is that we get a chicken-egg problem which blocks us to be fast already in the first place / request. Furthermore, if not carefully configured, even when there is already cached data to be applied, the system might erase its contents under certain situations (e.g. low memory) in a not easily predictable order, resulting many times in yet another request-response cycle with the server.

This incurs not only in wasted bandwidth, but also is a waste the of user’s time (insert your <please wait, loading…. > message here).

 

Solution: One Idea

As it should now be clear, there are situations where it is desirable to be able to pre-load server data.

Let’s take our sample application (the typical e-commerce or retail apps), where the app acts as an interface to navigate a catalog of products (scroll horizontally or vertically) at a certain page:

The video of app above was run using a fast internet connection, so why these loading messages? We know it well; it’s the latency caused by the TCP connection and requests overhead that first need to reach the server. So let’s finally solve this problem (i.e., make the wait time go away).

For that, we’ll use Couchbase Lite (CBL) as a supporting caching layer for HTTP requests to fill the ‘gaps’ of http v1.1 to do what we need. We’ll end with a hybrid solution of CBL / Request-response app, taking advantage of both data exchange methodologies, all without making any big changes to your existing request-response based project infra-structure (while still providing super fast user-responsiveness).

It might not be the exact thing you need for your application, but hopefully it will at least trigger some ideas on how you can improve your project, perhaps even in other creative ways (without huge up-front investments, and responsibly).

If you didn’t come to the same conclusion by yourself by now, the idea is basically to use the sync features of couchbase mobile to push/sync NSURLCache data between the server and the client. So NSURLResponses will be stored in couchbase “Documents”. This approach does not really interfere on the existing URL loading system except for overriding some cache policies in desirable and well controlled situations (in an attempt to avoid ‘reinventing the wheels’, especially related to memory management).

Once we have these Documents, we provide the data to NSURLCache using the storeCachedResponse: method so that it can be used by the standard URL Loading system. Every time a response changes to a certain request, we update the CBLDocument and repeat the injection for tight control of cached data.

Then, since we’ll have tight control of what the content of the cache is, we can safely use NSCachePolicyLocalDataElseLoad for such NSURLSession’s requests (this will make requests respond instantly).

So we’re going from this:

siteImage2

To this:

siteImage

Note that for data not present in the cache, the request proceeds automatically to fetch data with the server. Otherwise (when a response was already created for that request) the NSURLResponse will come immediately without ever reaching to the network.

Implementation – TL;DR

Breaking it down we need to achieve the following goals:

  1. Implement a cached first-run of any given http request;
  2. Implement Dynamic Cache Injection based for example, navigational context (more on that later);
  3. Update existing cache data in a ‘server push’ style, purging deprecated ones to free memory, taking into consideration its priority given a certain context.

Goal number one

For goal number one, we have two possibilities: create the response documents at the client or at the server. We’ll cover the first approach (client), and later we’ll see how it would work for the server side.

Since we’ll be using the standard NSURLCache, let’s prepare it in the app startup and set as shared object so we can use it to inject data to the URL Loading System:

 

Next, we can generate cache data documents by recording the responses in the app itself as we navigate, which is then synced with the server (and back to other devices later). For that I present NSURLSession+CBCache category. This class is at your disposal ready for use, and it works this way:

categories

All we have to do is import the NSURLSession+CBCache category (link) and then replace e.g. any dataTaskWithURL:completionHandler: calls you want to boost with the category cbcache_dataTaskWithURL:completionHandler:  counterpart:

What this method does is to intercept http requests and automatically generate the cache document for items not yet saved. If it doesn’t have a document for it, it will forward the request to the normal URL system, get the response, save it to Couchbase Lite and finally give it back to the originating method. Once you’ve done that, even if you delete the app, on the next run these documents will be reloaded on next sync and injected to NSURLCache due Couchbase mobile’s replication.

When recording cache document this way, no extra scripts in the server side are necessary, because a push replication can send the documents to the server.

Note: In practice (real life, production), if using this method, it will probably be necessary to setup some basic authentication so that only authorised devices can generate, update and push new response data to the server via the sync gateway.

By using the cbcache_dataTaskWithURL method, say, for an HTTP request like this:

GET http://myserver.com/mobile-api/clothing/?sort=asc

It will check in the local CBL database if an entry exists with this signature, before actually going for the external source. If not it loads it and stores it.

On the app startup or after a CBL sync completes, we call a method to iterate the documents and inject that data to our previously prepared NSURLCache:

Although you may be thinking that a simple NSCache or even NSDictionary should be sufficient in place of CBL, now we’ll see why CBL gives us benefits which would otherwise not be possible without it.

Goal number 2 (Implement Dynamic Cache Injection based on navigational context);

Now that we can inject data to NSURLCache, we should have a way of controlling which sets of data should be used when. For that, we can use the concept of channels in Couchbase Mobile. One channel could be replicated, for example, when the user is in the main menu, another when when he enters another screen, and so on. Search for ‘context’ in the demo app to see a sample implementation.

Goal 3

Finally, to achieve goal number 3 (Update existing cache data in a ‘server push’ style, purging deprecated ones to free memory):

Upon a cache Document deletion / update, a “purge” on such documents can be run at the mobile (client). NSURLCache can as well be cleared and reloaded only with the relevant data, thus completing the life-cycle of the cached data.

After the initial sync is completed, we add a listener to Couchbase Lite which fires a method to update cached content:

To make the system update the NSURLResponses automatically (that is, without the need to someone having to record / push data from the mobile side), we can setup a server job that from time to time iterates through each response document in the database, checks whether new data is available (that is, the response changed for a certain request), and if so update the document, causing it to be pushed back to the client.

Below is a short walk through video showing how this can be done (better seen in full screen) in the server and the reaction in the demo app.

 

Conclusion:

When thinking about exchanging data, we may tend to believe that either we use technology X (http API) or Y (data sync engine). But as shown, it is as well interesting to combine them in a way that one technology supports / improve the functionality of the other.

Once data is cached, this is the time difference (in seconds) for a request I obtained running the demo app with:

no HTTPSmartCache applied:

with HTTPSmarCache:

I’d say, quite a difference. Not a very fair comparison but hopefully you get the idea: Finally you can use NSURLCache effectively (avoiding the problem of stale information).

Now even network-dependent “UIWebView-based style” apps have an opportunity to feel more ‘native’ in terms of speed and reliability.

If you have any questions or suggestions, just let me know, or simply share your thoughts in the comment section below.

 

Further Notes:

1) I didn’t implement a similar category for NSURLConnection, but the same concepts apply except that one has to take note that it is better to have isolated configuration containers for each group of requests, as is possible in the newer NSURLSession API’s.

2) It’s not shown in the demo, but one may pre-load a database in the app’s bundle so that at startup there’s already content for the user to navigate through while the remaining data is synced.

3) You may notice that I’m storing image data in UIImage+CBCache category as NSData instead of CBLAttachments; The reason is that currently such data blobs are not propagated to the shadow bucket from the sync gateway resulting in that we have no other means of manipulating it from the server if not done otherwise.

4) Below is more detailed explanation in video format (with audio) regarding the code that causes the data-push effect in the cachePushDemoApp. The server script basically uses the Couchbase’s python SDK to query and modify the documents, which are synced / updated to the app side and injected to NSURLCache again (this is automatically handled by the provided UIImage+CBCache category, which is built on top of NSURLSession+CBCache). Of course instead of running the update manually as shown, on your server you’ll modify your documents upon another event, or inside another function etc.

Reference Links:

https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/URLLoadingSystem/Concepts/CachePolicies.html

Further research on the topic:

RFC 7234 – Hypertext Transfer Protocol (HTTP/1.1): Caching: https://tools.ietf.org/html/rfc7234

Also, I’ve searched the whole web but didn’t find anything solving the stale data problem when using NSURLCache. Below are some of related projects I found during that research.

Project SDURLCache (starred 664 times), last updated 3 years ago, no longer needed as apple enabled disk caching on NSURL loading system.

AFCache, also initially created due NSURLCache missing persistent disk cache.

RNCachingURLProtocol (starred 274 times): Simple offline caching for UIWebView and other NSURLConnection clients , last update 3 years ago

LocalSubstitutionCache: http://www.cocoawithlove.com/2010/09/substituting-local-data-for-remote.html

Explanation about http cache headers: https://devcenter.heroku.com/articles/increasing-application-performance-with-http-cache-headers

http://petersteinberger.com/blog/2012/nsurlcache-uses-a-disk-cache-as-of-ios5/ :  There doesn’t seem a way to force caching of certain requests; connection:willCacheResponse: is only called if the response contains a Cache-Control header

Research regarding caching in ios : http://www.hpique.com/2014/03/how-to-cache-server-responses-in-ios-apps/