Okay, it’s not just us: New benchmark tests confirm Flash video oftentimes really does look bad on mobile phones. EffectiveUI Software Architect Sean Christmann published extensive benchmarks comparing Flash and HTML 5 performance on mobile devices this week, and the results clearly show Adobe still has to catch up in the mobile video space. The nine mobile devices tested by Christmann generally had no issues playing back 720p content via HTML 5, but less than half were able to do the same in Flash with a satisfying performance.
Christmann built a pretty elaborate setup to test devices for their video playback capabilities. He encoded a source video with four different encoding settings to mirror the quality of video served from YouTube. He also hard-coded the number of each and every frame in the upper left corner of his test videos and then recorded the playback performance of each device with a high-speed camera. Afterwards, he looked at the recordings and determined how many frames each device had dropped during playback. The whole process was “thoroughly painful,” Christmann wrote on his website, but the results are definitely interesting:
Percentage of frames played during video playback on various mobile devices via HTML5 and Flash. Says Christmann: “Subjectively, I would argue that video that stayed above 70% looked good during playback. Anything below that mark will have too much stutter and really starts looking like crap.”
For a more visual take on these results, compare the video Christmann did of a Droid X playing a 720p H.264 baseline profile via HTML 5…
Watch this video for free on Mobilize — Mobile Technology Reviews, News and Trends
… with the same device playing the same clip via Flash. Notice all the dropped frames?
Watch this video for free on Mobilize — Mobile Technology Reviews, News and Trends
There are a few things worth pointing out:
Christmann used two encoding qualities for HD video; 720p H.264 baseline profile and 720p H.264 high profile, which looks even better and is for example used for Blu-ray discs. Most devices had no problem playing back H.264 base profile video via HTML5, but some devices simply refused to play back 720p high profile videos altogether. Also, iPads and iPhones don’t support Flash, so you won’t find any comparison number for these Apple devices.
More importantly, Christmann writes that the most recent Gingerbread update brought the Galaxy Tab results close to 100 percent, but he wasn’t able to re-record the test and include those findings due to time constraints. The same was true for the Motorola Xoom: With Android 3.0, it seemed to drop about half of all frames during 720 base profile playback, but Android 3.1 brought its Flash video playback up to par with HTML5 (the table does include the 3.1 results).
The reason for these discrepancies? Hardware acceleration. Playback of HD video on a phone really requires some dedicated support from the device’s chipset, and Adobe needs help from hardware manufacturers, as well as mobile OS makers, to enable this kind of hardware acceleration. This is akin to Flash playback on Macs, which used to be a real pain until Apple finally provided API access to Adobe to make use of your Macbook’s hardware acceleration capabilities.
This means Adobe has a good chance to catch up with HTML5 video playback on devices sporting the newest Android systems, but the flip side of that is that many mobile consumers may be stuck with a bad Flash video experience for quite some time. Data from Google shows that 90 percent of Android users still have Android 2.2 or below. Add that Adobe still has to support a number of different devices and chipsets to actually make hardware acceleration work in mobile, and you’ll get a sense of how the challenges ahead for Adobe.
The good news for the company is that Flash actually outperformed HTML5 in a number of other benchmark categories, including a bitmap drawing test and a vector drawing test (one should point out that Adobe was involved in funding the tests). This means that mobile users will have more fun with Flash-based browser games than with HTML5-based browser games. But in video, at least for now, HTML5 has the lead.
No comments:
Post a Comment