This post is intended for readers with advanced technical knowledge.
This article explains how you can mirror and show Web-based Flash videos on external displays connected to your iDevice. In order to answer the questions of several people at MacRumors (original thread), I've decided to thoroughly test the external monitor/Apple TV mirroring/playback capabilities of the currently available Flash browsers.
I've long been publishing compatibility, speed, efficiency comparisons and results of contemporary Flash browsers (see the UPDATE sections HERE). This article is entirely dedicated to TV output. Note that, here, I don't repeat the information (e.g., App Store links, lists of all Flash streamers etc.) I've already elaborated on in that article. Also note that, while I mainly discuss Flash players, some of the content of the article can also be made use of elsewhere. For example, using the excellent application DisplayOut applies to all third-party apps (and the entire iDevice itself) for excellent, high-speed screen mirroring, not only Flash players.
The need for outputting Flash videos to external monitors/projectors
You may need to present Flash Web videos directly on an external screen. If you use plain cabled or wireless AirPlay mirroring available in all iDevice starting with the iPhone 4S, iPod touch 5 or iPad 2, you won't be able to make use of the entire screen estate – even if you play back truly 16:9 Flash videos, only the center area of your screen will be utilized. Even on the iPhone 5 and the iPod touch 5G, which both are native 16:9 models and, therefore, one would expect their mirrored screen image to fill the entire screen estate. Nevertheless, it isn't the case, which can certainly be seen in the following shot of a mostly white Web browser screen filling the entire screen of my iPhone 5 mirrored to an external Full HD monitor via Apple's own Lightning-to-HDMI adapter:
(As with all photos in this article, click them for the original, high-quality, large originals)
In this case, the iPhone's screen is mirrored using a decidedly lower resolution than it otherwise would be possible with a Full HD monitor. The latter is pretty much the norm nowadays – at least when it comes to LCD monitors. (Most projectors still have lower native resolution.) This results in the thick black bars in the mirrored picture. I've even annotated the lower and the upper ones in a slightly edited version of the above shot:
Note that I dialed in strong exposure correction in the camera to make the active area of the screen in black still much brighter than the bezel so that the unused screen area can easily be spotted. This is why the center, white area of the screen is pretty much over-exposed.
Unfortunately, the case with the previous small-screened iDevices and the iPads are the same. The former have a screen with the aspect ratio of 3:2, the latter with 4:3. This means no one would expect their mirrored screen to entirely fill the screen of an external monitor. An example of the iPad 3's screen being mirrored over a wired connection (a VGA adapter):
As you can see, as opposed to the iPhone 5 / iPod touch 5 case, the full vertical screen estate is utilized. There, however, remain black bars on the left and right.
Note that this only applies to the wired connection (the VGA / HDMI adapters). Over AirPlay (to, say, the AppleTV), the vertical resolution is reduced in exactly the same way as with the wired (and wireless AirPlay) iPhone 5/iPt5 connection:
This isn't a problem when just mirroring the user interface itself. However, when playing back originally 16:9 videos in most Flash browsers, no attempt will be made to play it back making use of the entire screen estate. The reason for this is simple: these browsers aren't TV output-aware and the operating system itself can't decide whether the content they render is a video or not. It could only do this when the client plays back a natively playable video file – or, when the client is TV out aware and explicitly handles the TV output connection. Unfortunately, the latter is pretty uncommon with, otherwise, as Flash video player-recommended Web browsers.
1. New (2011+) iDevice Models
Let's start with current hardware; that is, all 2011 and later models: the iPhone 4S, 5; iPod touch 5; iPad 2, 3, 4 and Mini. These all are capable of mirroring the screen over both wired and wireless AirPlay connections, unlike previous models.
As most Flash Web browsers don't natively support TV output, using the, in 2011+ models, native mirroring capabilities is the only way to show their output on an external screen. However, this, unfortunately, means in no way will be the entire screen estate used in any case – there'll always be black bars on both the top/bottom and left/right with small-screen devices and on the left/right with iPads, both when used in Landscape.
The only exception is the Photon Flash browser (App Store links: iPhone/iPod touch version ($3.99); iPad version ($4.99)), which has native support for video output.
This means it does make use of the entire screen area on all kinds of iDevices with screen aspect ratios from 4:3 through 3:2 to the Full HD-native 16:9.
This also means it'll stretch the screen of non-natively 16:9 (that is, 3:2 and 4:3) devices. This is why the following iPad 1 shot shows a somewhat distorted (horizontally stretched) image:
Now, compare it to the following iPhone 5 shot, where the native 16:9 screen aspect ratio is retained (no horizontal stretching needed to be made):
1.1 VERY IMPORTANT:
No matter what HDMI/VGA/AirPlay-capable model (including the 2010 models) you run Photon on, make absolutely sure you enable full screen mode when playing back videos by tapping the arrow icon in the upper right corner. Otherwise, the active content will be badly cropped. An iPhone 5 shot of this:
(Note that, because of the restrictions when not playing back iOS native video files, this shot also shows black bars around all the edges – as with the previous one showing Photon running in the recommended full screen mode on the iPhone 5.)
An iPad 1 one:
And, finally, an iPod touch 4 one:
Note that the cropping problem is the worst on the last shot, a lot of the video's active area not being shown on the external screen. No such cropping would happen with enabled full screen mode. Also note that, as opposed to the iPhone 5 case, the full screen estate is used - as would be the case in the recommended full screen mode too.
Also note that, while both the iPad 1 and iPod touch 4 (iPt4 for short) belong to the group of iDevices released in 2010, explicit TV output from inside Photon has exactly the same problems as with 2011+ iDevices. This is why I've discussed them here and not in Chapter 2.
1.2 ALL IN ALL,
If you have a 2011+ model, you can safely use all Flash-capable players. However, Photon will present the best results as it explicitly utilizes the TV output. Just
- make sure you use it in full screen mode to avoid cropping
- use other Flash browsers (only capable of plain screen mirroring) when, in order to avoid horizontal stretching, you want to keep the original 3:2 or 4:3 screen aspect ratio on non-16:9 source devices connected to 16:9 monitors. This, again, won't be an issue on natively 16:9 models – that is, the iPhone 5 and the iPod touch 5.
2. 2010 iDevice models
2010 iDevice models (iPt4G, iPad1, iPhone 4) radically differ from their 2011+ successors, covered above, in a very important respect: they, while having support for both the HDMI and VGA adaptors, don't support simple mirroring, rendering most Flash browsers useless.
This means there are only two ways of making them work with online Flash videos:
- using Photon (see above)
- if you don't want to / can't use Photon, jailbreaking and using DisplayOut, which introduces display mirroring to all iDevices, even pre-2010 ones.
2.1 DisplayOut
DisplayOut (Cydia store description link; $3.99) has become an excellent app capable of screen mirroring speeds that come pretty close to Apple's own mirroring during the time.
(The main Cydia Purchases page with DisplayOut, shown on my iPhone 5.)
Basically, for Flash videos where the frame-per-second only exceeds 10-15 fps with SkyFire, the mirroring speed of this app will be more than sufficient. This means using DisplayOut to drive an external monitor over any video cable / adapter is indeed a viable alternative to using Photon, should you want to stick with another Flash browser.
2.1.1 Demo video
I've made a video of how DisplayOut runs on an iPod touch 4G running on iOS 6.1.2. It's as follows:
First, I quickly show the Settings interface of DisplayOut, and, at 0:22, start the Flash video browser SkyFire. (Which, again, doesn't explicitly support TV out and, therefore, cannot be used on any pre-2011 iDevice to play back Flash videos). After a lengthy loading process, the video playback starts at around 1:40. As you can see, the mirrored screen isn't cropped and fills the entire screen estate (by introducing horizontal sctretching – after all, the iPt4 has a screen of 3:2 aspect ratio) and is pretty fast – almost as fast as the playback on the iPt4's screen.
(Incidentally, as has been explained in my previous article, back on 06/22/2012, SkyFire entirely failed to recognize the test video. Newer versions play it back just fine, as you can also see in my video. Also note that, back then, Photon delivered the worst performance with this demo video and Puffin was significantly faster. Now, Photon was noticeably faster.)
At 2:26, I switch to another Flash browser, Puffin, which doesn't explicitly support TV out either. The playback on the iPt4 screen (and, of course, the mirrored external screen) is way slower than with SkyFire – or, as we'll soon see, with Photon.
At 3:20, I switch to Photon. After the original page is loaded, at 3:47, I tap the thunder icon in the lower right corner so that the browser switches to proxy-based streaming mode including Flash. At 4:00, DisplayOut states Photon (as we already know) explicitly supports TV out and that it's recommended to switch off DisplayOut altogether for better performance. However, in this case, I just stick with DisplayOut to show what it's capable of and, therefore, continue playback, which starts at around 4:20 and shows much better performance than that of Puffin – but still somewhat worse than that of SkyFire. As the video playback frame rate doesn't really exceed 10-15 fps in this case either, there isn't visible difference between the frame rate of the video rendered on the iPt4G's screen and on the external TV. That is, the performance hit introduced by using DisplayOut's explicit mirroring as opposed to Photon's own TV output isn't really evident.
Note that I didn't show iSwifter in the above video as it's iPad-only. It was capable of playing back the test video (as was the case back in June, 2012) – but doesn't explicitly support TV out.
3. Pre-2010 iDevice models
Unfortunately, with iDevices released before 2010 (all iPhones up to 3GS and iPod touch models up to the third generation) can't use the HDMI/VGA adapters either, let alone the 2011+-only wireless AirPlay.
The only way of using TV out with these devices is using either the component or the composite cables. Furthermore, programmers can't output anything to external monitors, except for hardware-decoded, local video files – which is in no way the case with streamed Flash content. This means not even the (only) Flash player explicitly supporting TV output, Photon, can mirror its screen to an external monitor.
With these old devices, your only choice is jailbreaking and using the above-mentioned (see Section 2.1) DisplayOut. Fortunately, as these old devices have low-resolution (pre-Retina) screens only, mirroring via DisplayOut isn't very slow, particularly not on the comparatively, even compared to today's models, fast 2009 models.
