Brief Review Samsung Galaxy S4 vs. iPhone 5 : TecHead

Over the last year and a half, HTML5 performance has reached impressive new heights across the mobile device landscape, to the point where well-engineered HTML5 applications can easily compete with native applications. Nevertheless, some platforms have advanced farther and faster than others. For example, while Android has steadily progressed, it has always been the perennial also-ran vs. iOS in mobile HTML5. From the original iPhone through the iPhone 5, mobile Safari has led the market with the highest performance, best rendering, advanced features and general quality. The Winter 2011 release of Android 4.0 (Ice Cream Sandwich) improved the Android HTML5 experience substantially, but still fell notably short vs. the then flagship iPhone 4S. And about this time last year, we were encouraged by the performance and feature advances of the Chrome for Android beta. Nevertheless, more advanced features, better responsiveness and smoother GPU acceleration left iPhone firmly holding the winner’s cup for best mobile HTML5 platform.

With the release of the Samsung Galaxy S4, we thought it was time to revisit Android and once again put it through the wringer of our HTML5 Scorecard testing. In our testing, we chose the built-in Android browser vs. the Chrome for Android app because Chrome for Android has about a tenth of the browser usage on Android. Furthermore, when developers create a packaged web application, it runs in a WebView based on the Android browser, not Chrome for Android. To put the S4’s performance in context, we compared it against an iPhone 5 running the most up to date iOS, version 6.1.3. The iPhone 5 hardware is now a year old, which theoretically should create a Moore’s law 50% performance gap vs. new hardware. Going into the testing, the big question for us was whether the S4’s brand new hardware would over-power the iPhone’s historic hardware/software advantage in graphics and browser performance. Read on and see.

Summary of Results

Overall, the Galaxy S4 is a nicely upgraded HTML5 platform for Android users. It adds exciting new features like WebGL 3D graphics and IndexedDB local database storage. The Galaxy S4 overall checks more HTML5 feature boxes compared to iPhone 5, and has faster performance in some areas. However, we still have to give the overall advantage to the iPhone. Scrolling performance is still noticeably smoother on the iPhone. Android’s scroll stutter — micro-intervals of screen freeze-up during scrolls — is still visible on the S4 for styled content. We don’t know whether to hold Samsung or Android responsible for this continuing performance gap. At Mobile World Congress this year, we saw LG Optimus phones that did not have this problem running stock Android.

A few other areas to note where the mobile Safari shines. iPhone is a champ at handling touch inputs and content movement simultaneously, and performance tends to be very even under zoom. The S4 tends to like doing one thing at a time. Zooming while the S4 was running heavy benchmarks often greeted us with unpainted space for seconds at a time. Furthermore, while the S4 reported SunSpider benchmarks about 10% better than iPhone (as expected for newer hardware), the wall clock time for those benchmarks to run was 4x the iPhone – leading us to believe that these benchmark results should be taken with a pinch of salt.

As a side-note, in addition to the HTML5 experience, the general user experience of the S4 was a little like being attended by an over-eager waiter. The Galaxy S4’s headline new features: air gestures, eye tracking, orientation scrolling etc. were obtrusive and we often activated them inadvertently. Within 5 minutes of unboxing, we were attempting to turn them all off (although it took us another 5 minutes to find the settings that would do so.)

The Details

Our HTML5 scorecard consists of a series of tests that aim to help mobile HTML5 developers understand new devices and new form factors as they come to market. We test in a number of areas: JavaScript and DOM performance, HTML5/CSS3 features, rendering performance and accuracy. To assess, we use a variety of tests, including modernizer, SunSpider, dromaeo, Sencha Animator demos, our Sencha Touch Kitchen Sink and a variety of browser and community “in-the-wild” demos.

Device Basics

The Galaxy S4 comes with Android 4.2.2 – the nickname-less upgrade to the 4.1 release of Android JellyBean. If you’re looking for the device, its user agent string is:

Mozilla/5.0 (Linux; Android 4.2.2; en-use; SAMSUNG-SGH-1337 BUild/JDQ39) AppleWebKit/535.19 (KHTML, like Gecko) Version/1.0 Chrome/18.0.1025.308 Mobile Safari/535.19

The continued arms race to masquerade as a popular browser has led user agents to lay down geological layers of user agent strings from once-dominant browsers. We’re waiting for the day when a browser maker ships a “What Browser Would You Like Me To Be” user agent string.

First up, we like to test JavaScript timers using John Resig’s timer test. Although requestAnimationFrame has made the quality of a timer less important, there is much animation in the wild that does not use it. The timing implementation on iPhone has always been the best in class. The iPhone 5 has a 4ms timer with very high consistency at all levels of concurrency. In contrast, the S4 has a noisier JavaScript timer than we’ve seen on mobile in a while. The high variance of timer execution means that it was difficult to determine what the intended timer resolution was. And beyond the high variance, there were a number of “world-stops” of up to 100ms. This was true for all degrees of concurrency. (By contrast, Chrome Mobile on the S4 has a 4ms timer with very good consistency, so this is a clearly a software, not a hardware problem.)

HTML5 Features

The Galaxy S4 scores a 468 on thecurrent HTML5test.com test vs. the iPhone’s 386 points. (Both browsers now lag the market leader in HTML5 feature implementation, the Blackberry Z10, which scores 485). Beyond the baseline of HTML5 features that are now ubiquitous (Canvas, inline SVG, semantic elements etc.) both the iPhone and the S4 support more recent features like server-sent events and almost all HTML5 form inputs.

In addition to these baseline features, the S4 reported support for three new headline features: microdata, WebGL, and IndexedDB! We also saw support reported for getUserMedia, the Ogg Vorbis audio codec and the WebM video codec. On the other hand, the iPhone reported support for some notable features that the S4 lacks: such as shared workers and drag & drop. Neither phone reported support for the Ogg Theora or Ogg Opus codecs as well as features like seamless iFrame, and scoped styles.

Overall the S4 clearly checks more HTML5 feature boxes. But as we’ve come to know, just because a browser says it supports a feature doesn’t necessarily mean that it works. The truthiness of the Galaxy S4 was about average. Ogg Vorbis samples played successfully. The W3C test page for Microdata passed with about 90% of tests successful. WebGL worked nicely (more on performance below). IndexedDB also worked, although on Rodrigo Reyes’ storage benchmark, it was about 10x slower than WebSQL on bulk operations. We were unable to make the getUserMedia() demos on HTML5rocks work correctly.

We also tested HTML5 input types on the S4. Date, URL, email, and number inputs worked correctly, pulling up appropriate customized keyboards. But we have to give Samsung the booby prize for “least possible work to get a feature checkbox”: the HTML5 color input is simply a strip of 16 color squares! Also, someone should put in the 5 minutes required to get the range input to respond to touch! Finally, when we tested server sent events, the first few events were received successfully but then further events did not seem to be processed smoothly. Unlike the iPhone, on the S4, events seemed to be queued up and released to the mobile browser in batches.

All of the features that the iPhone reported worked correctly; which is to say that the iPhone is an Apple product — fewer features that are better implemented.

As an aside, we also tested a few other features in both phones. First was webfont loading. We found once again that Typekit did not recognize our novel user agent and Typekit fonts failed to load in the S4, although Google Fonts loaded correctly. We then tested CSS positioning with quirksmode.org tests. Although position:fixed is rock solid on both phones, and overflow-scroll is correctly implemented on the S4 (a rare feature gaffe for the iPhone), there is still no support for background-attachment: fixed on either platform. Not a huge omission.

Raw JavaScript Performance

We usually don’t put a lot of stock in SunSpider benchmarks because they’ve become decoupled from the user experience of performance, but we decided to give the Galaxy S4 a whirl anyway with the 0.91 version. Our testing uncovered something a little odd. Although the S4 *reported* the best performance on SunSpider (890ms vs. the iPhone 5’s 970ms), the wall clock time for the benchmark to complete on the S4 was 128 seconds — almost 4x higher than the 34 seconds it took on the iPhone 5. We thought this might have to do with the screen updating and javascript harness, so next we turned to dromaeo to see how general DOM interactions held up on both platforms.

Dromaeo’s Core DOM tests measure basic interaction speed with the page objects. Here the iPhone simply smoked the S4, with performance almost 90% higher than the S4. Test variance was also much lower on the iPhone. Since these are both webkit-derived browsers, it’s clear that Apple has added some special sauce into mobile Safari.

Graphics and Animation Performance

We then turned our attention to graphics and animation performance. Since CSS Animation was inspired by Apple’s CoreAnimation, the iPhone’s animation performance has always been near native performance, while other platforms have had to catch up. On the S4, we found CSS Animations worked well, although complex Animations felt like they were in the 20-30 fps range. We also found minor stutter when CSS transforms were combined with touch events. The famo.us demo of 3D transforms, for example, slowed to about 10-15fps when a touch was active.

The S4 has more GPU cycles and it shows in raw Canvas benchmarks. Benchmarks like this one from mindcat ran extraordinarily well on the S4, with a score of 1.72 (1.62 with alpha turned on). The visual smoothness and crispness of the transitions was extremely impressive. The iPhone managed a 1.57 (with and without alpha). Next we tried our favorite Canvas test: effectgames’Canvas color cycling. Happily, we saw very smooth performance with the S4. However, when we tried to pinch-zoom, we crashed the Android browser. When we reloaded and tried again, we crashed the entire Android OS. Our other favorite Canvas test is Microsoft’s fishbowl demo. Unfortunately, this does not work correctly on the iPhone, so we used Microsoft’s other swimming fish demo (someone at Microsoft must be an aquarium owner!). On the iPhone, 250 fish swam happily at 38 frames per second. On the S4, we got 30 fps. Both were visually smooth and artifact free. A platform tie.

SVG features and performance have been a traditionally weak area for both Android and iPhone (vs. Microsoft), so we wanted to see how the S4 would do. First, we ran David Dailey’s decahedra demo which combines rotating and overlapping shapes with SVG gradient animations. The S4 handled about 15 shapes before we started to see frame drops – similar to iPhone performance. A more dynamic pseudo 3D corridor navigation game, started at a reported 30fps but as we played, the frame rate dropped to approximately 10 fps on the S4. The iPhone ran at a more constant 20-25fps: usable, but less than perfect. SVG filters are now available on both platforms. Our test of some basic and complex filters worked great. There’s even support for SVG SMIL animation on each platform.

Finally, we were very excited to see how real the WebGL support in the S4 would be. We looked at the Khronos Group demos, and found very solid performance. The many planets demo ran at about 30 fps, and apart from a little shakiness under pinch/zoom, all the demos we tested worked properly. Game on iPhone – Apple has now been lapped by Blackberry and Android in WebGL implementation.

Other miscellanea

We did investigate the Galaxy S4’s “Air Gestures”. While we don’t anticipate many users leaving them active, it would be nice to know what events these fire on a page. Right now, Air gestures will scroll page content (air gesture up or down), or swap between tabs (air gesture from side to side). They do not seem to fire touch events, and we were unable to find a JavaScript event that they did fire.

Samsung Galaxy S4 vs. iPhone: Advantage iPhone

The G4 is essentially a Moore’s Law update of the same Android platform we’ve had. It packs a few exciting new HTML5 features like WebGL and IndexedDB; and fills out HTML5 support in other areas. But some of the new HTML5 features are a little uneven and the rough performance of Android’s core engine makes them feel in need of a patch update before we can really count on them. For the highest performing, highest quality HTML5 platform, we still have to give our continuing thumbs up to the year-old iPhone 5 vs. the Samsung Galaxy S4.

TecHead / @rrajowan