iPhone caching

December 22nd, 2009. Tagged: performance

2010 update:
Lo, the Web Performance Advent Calendar hath moved

Dec 22 This post is part of the 2009 performance advent calendar experiment. Stay tuned for the articles to come - only 2 to go!

Some time ago there was a post on YUIBlog highlighting the findings of Wayne Shea and Tenni Theurer on the caching behavior of the iPhone. Curious if things have changed several iPhone OS updates later, I ran some experiments with OS3 and OS3.1.

Highlights

  • iPhone will not cache components bigger than 15K (was 25K in the previous experiment)
  • total cache is about 1.5MB (was 500K)
  • short-term memory cache will store components up to 1941K provided the component size does not divide by 4
  • powering off the device still clears all the cache, Expires headers still important
  • not sure since when, but closing all tabs clears the cache too
  • consider HTML5 application cache to improve cacheability and provide offline experience – components in the application cache can be bigger than 15K and stay cached even after you clear Safari's cache from the Settings app (or clear it any other way)

15 is the new 25

Previous experiment showed that the iPhone will not cache components bigger that 25K, but this number is now down to 15K. And this is ungzipped size, meaning if you have for example a 20K JavaScript and your server sends it gzipped down to 10K, it will not be cached by the iPhone.

So minification can definitely help you here. Minification may not be as beneficial as gzipping in desktop browsers, but it may be the difference between a cache hit or miss in the iPhone. Minify, then gzip. And, in general, try to keep your component sizes low - that always helps in any browser.

Total cache size

First off, how was the experiment run? I simply tailed the access log of a server, in order to monitor which components get requested:

$ tail -f ~/logs/http/access.log | grep "XXX.YYY.ZZZ.000"

Where XXX.YYY.ZZZ.000 is the IP address. You can also replace that with just "iPhone" if you think no one else is hitting this site with an iPhone.

Then I requested components with different sizes and look at what the log tail says.

Once it was clear that 15K is the maximum components size, the next question was how many of those 15K components can be cached before iPhone runs out of space allocated for caching? And the result was 105. Attempting to cache one more file resulted in removing existing ones from the cache. Then playing with the size of the very last component helped nail the exact number of bytes available in the cache – 105 * 15 + 7 = 1582K.

So the total cache is just a little over 1.5 MB (earlier research showed 0.5MB for earlier iPhone OS). Have in mind that this is the total cache shared between all pages, it's not per domain or per tab.

Memory cache

The iPhone has a little bit more cache space in the form of memory cache. The memory cache has no per-component size limit, so your components can be as big as the memory cache – 1941 bytes (don't ask how many attempts it took to nail this number).

There a little catch though, probably due to a bug, and it's that if the component size divides by 4, it won't be cached in memory. That's true for JavaScripts – they will be requested on every page load. CSS files with sizes over 15 that divide by 4 will not be requested in the same tab session, but if you close the tab and open the page in another tab, they will be requested again.

So components under 15K go to the disk cache, a 16K component is never cached (so is 20, 24, …, 36,.. 100,… 1024 and so on), 17K and up (up to almost 2 megs) components are cached in memory.

This memory cache is very unreliable as you can guess, because it gets cleared very often, it's probably useful only in the same user session on your web site.

Some observations

  • Closing all tabs, except for blank ones (as when you do "New Page") and then closing Safari is the same as clearing the cache from Settings. So it's a good idea in your normal daily use to leave at least one page open, before you close Safari in order not to cause the implicit cache clearing.
  • Tapping the reload icon in the address bar sends unconditional requests for all components, without the If-Modified-Since header and ignoring the Expires header. So to speed up your browsing, refresh the page by tapping the address bar and then tapping GO, don't use the refresh icon.
  • The tests I ran were using OS 3.0 and OS 3.1.2. I got my phone last Christmas which means that according to this page it came with OS 2.2. I don't remember if I ran any tests with OS2.2. too. I do remember though that I tried some tests back when I didn't have a phone and asked help from Ryan Grove and Nicole Sullivan. Chatting over IM they were loading pages while I was tail-ing the server log. Those tests must have been with OS 2 or OS2.1. Back then the results showed that the limit for a component that gets stored in the cache was 10Meg, which was also the total size of the cache. Now I have my doubts that back then we only tested memory cache, not disk cache. In any event, it's important to note that those restrictions are all software restrictions. No matter what model the phone, it's the OS that sets the limits. Different models with the same OS will behave the same when it comes to caching sizes.

HTML5 offline application cache

All in all we can safely summarize that the iPhone has no cache to speak of. 15K per component is nothing and the limit of 1.5Megs shared with all other pages will have your cached components kicked out pretty quickly.

So – what's an iPhone performance optimizer to do? Use HTML5 goodies.

One thing to consider is some form of client storage supported by the mobile webkit (either key-value or SQLite), but that will require some of your javascript to handle the caching, expiration and so on.

Another thing to do is use the offline application cache. It's meant to support applications to work offline, but it also ends up being useful to improve caching of online applications as well.

What you need is a simple text file called a manifest. In there you list all the components required by your application. E.g.

CACHE MANIFEST

/root/path/to/images/image.jpg
or/maybe/relative/paths/too/scripts.js

http://example.org/good-old/absolute/path/oojs-home.jpg

It's important to serve this file with content type text/cache-manifest.

Then in your html tag just point to that file. Let's say you called the manifest mycache.manifest (could be dynamic, php, or anything, as long as it's served with the proper content type). So your HTML should start like the following:

<!DOCTYPE html>
<html manifest="mycache.manifest">

And this is it. There's also JavaScript API available if you want to play with the items stored in the cache.

Now the browser will request the manifest file every time (Expires header won't help you here) and if its contents is changed it will quietly and unobtrusively download the updated components in the background and the next time the user will see the updated page. Needless to say it makes sense that all the pages of the site share the same manifest.

The best part is that when using offline cache none of the restrictions mentioned above apply – you can store files bigger than 15K and you don't share your total cache space with anyone.

I used this for a personal project – whomsy.com if you want to play with some requests and monitor the traffic (e.g. using the iPhone simulator and Charles proxy).

Manifest – not just for iPhone?

The manifest idea sounds so nice, it makes sense to apply it to desktop browsers as well. In a way it's similar to the idea of having web archives – a zip with all page components (sorry, can't find the URL for the proposal for web archives right now). You can include the manifest on your homepage and, after onload, the browser preloads all the other components used by internal pages (no need to do preloading yourself in JavaScript, no need to worry about spriting images and so on). This is absolutely doable and it also provides the side benefit that your app is available offline, which is the original purpose of the offline cache :) However the application of this technique is a little limited.

  • Browser support – currently Webkit browsers support manifests, and IE doesn't. Firefox does support offline cache, however it shows an ugly confirmation message at the top of the page and you normally don't want to stress out your users with warning messages (see screenshot below)
  • The "homepage" meaning the page that includes the cache manifest also gets cached implicitly. This could be undesired behavior for many sites, but could work just fine for Ajaxy one-page-sites and applications.

Overall, I'm pretty enthusiastic about using offline cache for the purposes of normal page caching too. The future is here, it's just not evenly distributed (nor supported in IE) yet :)

Tell your friends about this post: Facebook, Twitter, Google+

34 Responses

  1. Very interesting article !
    It would be nice with the growing number of Android devices to do the same tests with the Google Android browser.

    Anyway your advent calendar is very great ! I can’t wait for the december 25th :)

  2. (sorry, can’t find the URL for the proposal for web archives right now)

    Did you mean:
    http://limi.net/articles/resource-packages/

  3. @ExKwhanDo – thanks! BTW, there’s no Dec 25, the last one will be tomorrow Dec 24 :) Now sitting down to write Dec 23.

    @arnout – exactly, thanks, that’s the URL

  4. So given the size of cacheable components, maybe we need to relook at the rule for combining components. For the iPhone user agent, files should be combined in chunks of 15K or less. This will increase HTTP requests for the empty cache experience, but reduce them for the full cache experience.

  5. I’m very interested in your methods. I was able to get my iPhone (3G, OS 3.1) to cache an 800k Javascript file. I didn’t try bigger.

    My test was very simple: small HTML page with almost nothing on it but a SCRIPT tag. The script resource is a repetition of this line:
    whatever_random_variable = 5;
    until the file reaches 800k.

    I load the page, notice Safari making a request for the HTML page, (through my proxy) then making a request for the 800k javascript file. Then I close the tab, close Safari, open Safari, open a new tab, and load the same page. This time it only requests the HTML, not the javascript.

    Throughout this process I had gmail open in a separate tab – no other tabs.

    I am very interested to know why we had such a different experience and under what circumstances your iPhone will only cache a 15k file. Did you have lots of other tabs open?

  6. I forgot to mention perhaps the most important part: I sent this header along with my javascript file to ensure that it got cached:

    Cache-Control: max-age=660480

    What headers are you sending to indicate that your resources should be cached?

  7. Could you possibly post your code? I am having a VERY hard time getting caching to work at all on the iphone. Are you using any headers besides Cache-Control?

  8. The application cache feature is just weird. It makes perfect sense to cache large static components (CSS, JS, images) but the problem is, the HTML document that references the manifest file is cached implicitly as well (it is a master entry, in spec terms), so it can’t be dynamic. I’m totally at loss at why it was specced and implemented that way. Shouldn’t the manifest URL be the cache key? And why should any change in the manifest file result in re-download of all cached files?

  9. I am very excited about using the offline cache as a way to store all my js, css, and image files, this will speed up our Iphone app 10X.

    Problem is exactly what Leonya is saying … I think. I get all these files to cache just fine but now nothing on the site gets pulled down. If it isn’t in the manifest like say (..someImage.png) then it doesn’t show up at all. I think it’s because it is caching the page not just what is in the manifest. How do we get around this??? Thanks

  10. OK, I figured it out. I wasn’t setting the NETWORK: section in the manifest, which is required in order to determine what areas require online access.

  11. [...] 的 safari 最大只 cache 單檔大小 15~25kb 的檔案1 [...]

  12. Hi Friends

    I am using MANIFEST in my web application.I am using google chrome.By using manifest i can work offline.i want to play a audio file in offline is it possible.I can cache upto 5MB.Can i increase that size.

    Thank in advanced!!!!!!!!!!!!!!!!!!!!

  13. [...] capable browsers have appeared to challenge the iPhone. Stoyan Stefanov found, in late 2009, that the iPhone’s cache limits had changed (sadly, for the worse). But where do things stand now? And what about those non-iOS browsers? [...]

  14. [...] Stefanov wrote an excellent and detailed article on how the cache for Safari on the iPhone works, or rather, doesn’t work. You should read the [...]

  15. Thanks for this great article, I was looking for safari caching behavior since ages !

  16. Woah! iPhone caching / Stoyan’s phpied.com I’m really digging this design:idea for the blog. It is very simple, yet effective. Frequently it is challenging to get the balance concerning superb usability plus visual appearance. We should mention that you’ve done an amazing job. Also, your blog starts extremely fast to me with Chrome. Superb Site iPhone caching / Stoyan’s phpied.com By the way have you read Gaddafi amazing news… Kudos … Rob Rasner IMDB

  17. Most people thoroughly take pleasure in your blog post post. You’ll discover a lot of signifies we’re able to place it for you to quality make use of while wearing little or no work punctually and difficult gained hard cash. Thanks relating to serving contain the post respond to numerous problems we have now experienced previously.

  18. [...] that they’ll state them and we, developers, don’t have to hunt them down as we’ve done before and again. But I thought I should check these limits, namely the size of images limits. [...]

  19. yamaha r6…

    [...]iPhone caching / Stoyan’s phpied.com[...]…

  20. 7pm Tech Code Snippets…

    [...]iPhone caching / Stoyan’s phpied.com[...]…

  21. I am also having issues, would it be possible to post your code please?
    ty

  22. I just love the internet! Thank you very much for this webpage!

  23. great new deal,check this offer,iphone 5 newly released…

    [...]iPhone caching / Stoyan’s phpied.com[...]…

  24. [...] decrease load times for users on older hardware and for those with spotty network connectivity. As Stoyan Stefanov mentions: the current iPhone will not cache individual components if they are larger than 15K and [...]

  25. I do agree with all of the concepts you have introduced for your post. They are very convincing and will definitely work. Still, the posts are very short for novices. Could you please lengthen them a bit from subsequent time? Thanks for the post.

  26. Hello there, just became aware of your weblog via Google, and located that it’s really informative. I’m gonna be careful for brussels. I will appreciate should you proceed this in future. Many people will likely be benefited out of your writing. Cheers!

  27. Fantastic issues altogether, you just won a new reader. What could you recommend in regards to your submit that you made a few days ago? Any certain?

  28. [...] cache do iPhone, por exemplo, não guarda nenhum componente acima de 15 KB, e isso sem gzip (fonte). O Zepto.JS tem 24 KB na sua versão atual já minificada (o jQuery tem 95 KB). Se der, prefira [...]

  29. Are these findings still applicable to iOS 5 & iOS 6 iphones also?

  30. I enjoy you because of each of your labor on this web site. My mother really likes getting into internet research and it’s obvious why. We hear all relating to the powerful method you give both interesting and useful ideas on the web site and even cause participation from some other people on the issue and our princess is undoubtedly understanding a whole lot. Enjoy the remaining portion of the year. You’re the one performing a glorious job.

  31. You actually make it appear really easy with your presentation however I to find this matter to be actually one thing that I think I’d by no means understand. It kind of feels too complex and very huge for me. I am looking ahead for your subsequent post, I’ll attempt to get the grasp of it!

  32. […] cache do iPhone, por exemplo, não guarda nenhum componente acima de 15 KB, e isso sem gzip (fonte). O Zepto.JS tem 24 KB na sua versão atual já minificada (o jQuery tem 95 KB). Se der, prefira […]

  33. s Spoken Word and her website offers products to help Christians
    use Bible scriptures in every day living finding the
    secret power of god. Thats the main reasons people dont usually set out to catch
    the Blue cat fish. As I don’t buy Collector’s Edition games as, to me, they’re not worth the additional money, I had to wait an extra
    few weeks for BG to put Dark Parables: The Exiled Prince, the normal
    version, up on their site for sale.

  34. […] cache do iPhone, por exemplo, não guarda nenhum componente acima de 15 KB, e isso sem gzip (fonte). O Zepto.JS tem 24 KB na sua versão atual já minificada (o jQuery tem 95 KB). Se der, prefira […]

Leave a Reply