This is a short note on some tests I ran yesterday during the Super Bowl with “2nd screen” devices – tablets and smartphones that feed auxilliary content – usually synced in some fashion – with the main tv content being watched “1st screen.”
An excellent longer review of a number of apps is located here (by Chuck Parker). My observations are more generic and cover the issues associated with the infrastructure required to adequately support a 2nd screen experience for millions of simultaneous viewers – as happened yesterday during the game.
First, a brief note on my test setup: A dedicated wireless router was attached to a 100Mb/s internet connection. 1st screen was HDTV fed from cable. 2nd screen devices were 2 iPhones and 2 iPads, connected to the router via WiFi. iPhone 4 and 4S, iPad 1 & 2. A laptop was also connected to the router, but was only used for checking connection speed and some network statistics.
Speedtest.net was used to verify internet connection speeds at the beginning of the game, and every 15 minutes thereafter. Actual download speeds averaged 87Mb/s over the game duration, upload averaged 4Mb/s. WiFi was first checked for local channel usage, then a static channel was selected on the router that had the least other local traffic. The SSID was not broadcast (to avoid anyone else attempting to log in and potentially affecting throughput – even though security was enabled). Pre-game testing ensured that all mobile devices were reliably connected to the router.
Speedtest.net- Mobile app was installed and used on all mobile devices to verify WiFi speed. Each device reported an average of 7Mb/s download and 4Mb/s upload.
Each iDevice was running IOS v5.01, internal memory was either 32 or 64GB.
I tested NFL2011 (even though it was supporting 2012 game the name is not updated…); NBC sports; CBS sports; Shazam; PrePlay; Tapcast. There were of course more, but that was as much multi-tasking as I felt my brain could handle!
I also tested ‘live streaming’ of the game from NBC and NFL websites. The NFL did not work at all: they streamed only Silverlight which is incompatible with iDevices… The NBC feed worked fine, but was delayed a full minute – which made for an odd comparison when looking at 1st screen. The timing was obvious when comparing the running clock in the scorebox at bottom of screen…
In general, the biggest issue was app instability and (assumed) network congestion. All of the apps experienced some degree of freezing, delays, crashes, etc. The NFL app was the most stable, Shazam and TapCast were the most unreliable. TapCast in particular crashed repeatedly, and even when running would lose it’s place often, returning the user to the main menu where one had to re-select the game and start over.
While I have no way of proving this, it felt like the scale of the communications may have affected the performance. It’s one thing to test an app in a lab, it’s another thing entirely to do projected load testing on your backoffice capacity to support 1,000,000 instances of your app trying to interact with your servers simultaneously during Super Bowl…
On one of the iPads I attempted to ‘multi-task’ the apps so I could switch back and forth… NOT. Even on iPad2 with more horsepower this just didn’t work well at all. Most of the apps either crashed outright, or got lost and basically it was as if I had just started the app – had to start over. Thread preservation just didn’t work. I don’t know enough about IOS and app development to understand why, but the bottom line is that the current state of a running app was not preserved.
I won’t comment here on the individual features of the apps, other than to say there was a wide range of styles, graphic excellence, etc. On the whole, I was impressed across the board with the NFL app: it was the most robust, had good, simple and intuitive graphics – I didn’t feel like I needed a legend to understand what was going on.
I must offer a disclaimer: I am not a big football fan, so certain subtleties of game description may have gone unnoticed, but in another sense I think this made for an objective review by a casual observer.
My summary is this new sector of content consumption (2nd screen) is here to stay, can be compelling and drive viewship, and has all sorts of advertising possibilities. An interesting commentary on the social aspects of 2nd screen, ads and such can be found here. The infrastructure and the apps themselves need to be more robust, or viewers will get frustrated. On the whole, a good start – and perhaps a bit of ‘trial by fire’ in terms of massive use of 2nd screen apps.
I can’t wait to see how good this gets by next year…
Parasam 