This article is highly recommended for programmers and people interested in shooting video.
As of iOS 7 beta 2, the built-in Camera application can't make use of the extra pixels of the sensor when zooming. Only custom-written, third-party apps currently can do that. Hopefully Apple fixes Camera in the final version of iOS 7; before that, should you want to use zooming while shooting video, avoid using the Camera app and make sure you write or install an iOS 7-specific video recorder application for best results.
Many of my readers have asked for a complete elaboration on the brand new zooming features of iOS 7, both in the stock Camera application and the programmatic API support (AVCaptureDevice.videoZoomFactor and the related, highly useful properties AVCaptureDevice.activeFormat.videoZoomFactorUpscaleThreshold and AVCaptureDevice.activeFormat.videoMaxZoomFactor).My main aim was to find out whether iOS 7 has implemented anything like the venerable, stunning Nokia PureView 808, which can losslessly zoom up to the zoom factor of four when recording Full HD video (and even more when shooting lower-res footage).
I have some GREAT news for you all: while the stock Camera client (as of the just-released iOS 7 b2) doesn't make use of the extra pixels on the sensor when zooming, if you write your own video recorder and set the zoom yourself programmatically, the system will.
Let's start with my measurement results of the stock Camera after a quick intro to the question of lossless zooming. Then, we'll continue with those of my custom test app and, finally, let's take a look at a real iOS 7 app's source code with zoom.
As usual, I used the ISO 12233 test chart for measuring resolution. As the stock Camera client only records full HD videos, I used the same full HD output resolution in my programmatic tests too, so you can directly compare the results.
1. The Theoretical Maximum of Lossless Zoom
If you've followed tech press in the last 16 months, you've already heard of the Nokia PureView 808 and its excellent camera module. It allows for (almost) lossless zoom up to 4x when shooting Full HD video. (You can find some excellent, highly recommended comparative shots between the 1x and 4x zoom HERE, right at the bottom of the page.)
The iPhone 4S and 5 “only” have a 8 megapixel sensor (while the Nokia 808 has a 41 megapixel one), which means theoretically, they could only use lossless zoom up to 1.7x (3264 horizontal source pixels/1920 horizontal target pixels = 1.7). In practice, however, the upper limit is a bit lower, at 1.454545.
When properly implemented, video zooming should be able to produce almost as good of results at this zoom factor (1.454545x) than without zooming in at all. Let's see how iOS 7 fares in this respect!
2. The Stock Camera App
As has already been mentioned, the built-in zoom slider in the stock Camera client, in both iOS 7 b1 and b2, fails to make use of the extra pixels to deliver lossless zoom.
While creating a test video footage, I've set a value on the zoom slider close (but not higher than) to the above-mentioned maximum, at 1.454545x (note the slider at the bottom of the screen in Portrait orientation):
With the above setting, the reschart result is as follows:
The point where the lines are no longer distinguishable is around the 8 mark. Beyond that, there's absolutely no detail. (For comparison: without zooming, it's at 10.8 corresponding to 1080 pixel rows).
As expected, it has very bad resolution at the maximal (3x) zoom:
In this crop, you can't even see the point where the parallel lines are no longer distinguishable. Here, marks start at 5.5. By scrutinizing the full, original image (click the thumbnail to get it), you can easily see it's around the 5 mark.
Finally, as has always been the case with all iPhone and iPad models manufactured after 2009, at the default 1x zoom, the results are excellent and indeed almost makes full use of the resolution capabilities of the output file:
All this tested numerous times on the GSM version of the iPhone 5 (model A1429, manufacture date: week 22 of 2013), but as this is a software and non-hardware-specific issue, it must behave in the same way on other, iOS7-capable models.
3. Zooming In Your Own Apps
As has already been explained, the above problem does not apply to zooming in your own camera recorder client. My thoroughly tested results are as follows:
1. 454545x zoom (that is, the theoretical maximum of lossless zoom range):
3x zoom:
After comparing results from Section 2 above, you can see both are immensely better than those of the stock Camera app. Actually, the 1.4x zoom is only a little bit worse than the 1x one (see the last shot in Section 2), in which it's comparable to the pretty little quality loss on the Nokia PureView 808 when zoomed in to 4x on it (again, see the GSM Arena review's comparison).
4. Programmers Only: Utilizing Zoom in Yyour Own Apps
Making use of the zoom is really easy—basically, all you have to do is wire the videoZoomFactor property of your actual AVCaptureDevice instance to GUI controls; for example, a slider. In the code below, I just pre-programmed it to have the value of AVCaptureDevice.activeFormat.videoZoomFactorUpscaleThreshold (a read-only property specific to the current video mode); that is, the maximal zoom factor that can still be zoomed into without quality loss:
self.videoDevice.videoZoomFactor = videoDevice.activeFormat.videoZoomFactorUpscaleThreshold;
Note that there is another zoom-specific, read-only property, AVCaptureDevice.activeFormat.videoMaxZoomFactor. It's, generally, set up in a way to provide a 10x10-pixel area to zoom into. This is why it has the value of 108 for Full HD, which corresponds to a whopping 108x zoom. Of course, you won't want to use that large values.
Also note that, as low-res 4:3 modes are all binned (along with the 720p60 mode, see my next article), they essentially halve the maximal lossless zoom factor. For example, mode 0, which has the lowest resolution of 192x144, could use a lossless zoom up to 17x (3264p/192p = 17) without binning. However, as mode 0 is binned, this value is halved; this is why you can only use a lossless zoom factor of 8.5 (see its videoZoomFactorUpscaleThreshold) only. This (binning) is why the 720p60 modes can only use a lossless zoom factor of 1.05, while the (non-binned) 720p30 modes can zoom up to 2.18x losslessly.
An example (the full project is HERE) is as follows. It presents a Start button in the upper right corner. After tapping it, the method startVideoRecording is called. Only then will the zoom level be set. You can stop recording at tapping the button again (which will be renamed to “Stop” when recording). The recorded videos will be in the Documents directory of the app accessible via iTunes File Sharing (I've enabled UIFileSharingEnabled in the main plist file).
As I'm programmatically creating and displaying the start/stop button, I only modified (in addition to adding UIFileSharingEnabled to the main plist, of course) the View Controller of the project. The .h file:
#import <UIKit/UIKit.h>
#import <AVFoundation/AVFoundation.h>
@interface iOS760fpsRecorderViewController : UIViewController <AVCaptureFileOutputRecordingDelegate>
@property (retain) AVCaptureVideoPreviewLayer *previewLayer;
@property (retain) AVCaptureSession *captureSession;
@property (retain) AVCaptureDevice *videoDevice;
@property (retain) AVCaptureMovieFileOutput* fo;
@property (retain) UIButton* startStopButton;
@end
The .m file:
#import "iOS760fpsRecorderViewController.h"
@interface iOS760fpsRecorderViewController ()
@end
@implementation iOS760fpsRecorderViewController
@synthesize captureSession;
@synthesize previewLayer, fo, videoDevice, startStopButton;
- (void)viewDidLoad
{
[super viewDidLoad];
self.startStopButton = [[UIButton alloc] initWithFrame:CGRectMake(40, 40, 80, 60)];
[startStopButton setTitle:@"Start" forState:UIControlStateNormal];
[startStopButton addTarget:self action:@selector(buttonPressed) forControlEvents:UIControlEventTouchUpInside];
// 1. session
self.captureSession = [[AVCaptureSession alloc] init];
// 2. in
self.videoDevice = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
NSError *error;
AVCaptureDeviceInput *videoIn = [AVCaptureDeviceInput deviceInputWithDevice:videoDevice error:&error];
if (!error) {
if ([self.captureSession canAddInput:videoIn])
[self.captureSession addInput:videoIn];
else
NSLog(@"Video input add-to-session failed");
}
else
NSLog(@"Video input creation failed");
// 3. out
self.fo = [[AVCaptureMovieFileOutput alloc] init];
[self.captureSession addOutput:self.fo];
// 4. display preview
self.previewLayer = [[AVCaptureVideoPreviewLayer alloc] initWithSession:self.captureSession];
previewLayer.frame = CGRectMake(0, 0, self.view.frame.size.width,self.view.frame.size.height);
previewLayer.contentsGravity = kCAGravityResizeAspectFill;
[self.view.layer addSublayer:self.previewLayer];
[self.view addSubview:self.startStopButton];
[self.captureSession startRunning];
}
-(NSUInteger)supportedInterfaceOrientations{
NSLog(@"supportedInterfaceOrientations");
return UIInterfaceOrientationMaskPortrait;
}
-(void)startVideoRecording
{
int selectedAVCaptureDeviceFormatIdx = 15; // Full HD
[videoDevice lockForConfiguration:nil];
AVCaptureDeviceFormat* currdf = [videoDevice.formats objectAtIndex:selectedAVCaptureDeviceFormatIdx];
videoDevice.activeFormat = currdf;
if (selectedAVCaptureDeviceFormatIdx==12 || selectedAVCaptureDeviceFormatIdx==13)
videoDevice.activeVideoMaxFrameDuration = CMTimeMake(1,60);
NSLog(@"videoMaxZoomFactor: %f", videoDevice.activeFormat.videoMaxZoomFactor);
NSLog(@"videoZoomFactorUpscaleThreshold: %f", videoDevice.activeFormat.videoZoomFactorUpscaleThreshold);
self.videoDevice.videoZoomFactor = videoDevice.activeFormat.videoZoomFactorUpscaleThreshold;
// self.videoDevice.videoZoomFactor = 3;
[videoDevice unlockForConfiguration];
int fileNamePostfix = 0;
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
NSString *filePath = nil;
do
filePath =[NSString stringWithFormat:@"/%@/%i.mp4", documentsDirectory, fileNamePostfix++];
while ([[NSFileManager defaultManager] fileExistsAtPath:filePath]);
NSURL* fileURL = [NSURL URLWithString:[@"file://" stringByAppendingString:filePath]];
[self.fo startRecordingToOutputFileURL:fileURL recordingDelegate:self];
}
- (void)captureOutput:(AVCaptureFileOutput *)captureOutput didStartRecordingToOutputFileAtURL:(NSURL *)fileURL fromConnections:(NSArray *)connections
{}
- (void)buttonPressed
{
if ([self.startStopButton.titleLabel.text isEqualToString:@"Start"])
{
[startStopButton setTitle:@"Stop" forState:UIControlStateNormal];
[self startVideoRecording];
}
else
{
[startStopButton setTitle:@"Start" forState:UIControlStateNormal];
[self.fo stopRecording];
}
}
- (void)didReceiveMemoryWarning
{
[super didReceiveMemoryWarning];
}
@end
