×

Advanced photo capturing

During the last decade mobile phones with integrated cameras have been conquering land from the traditional pocket cameras. The saying "the best camera you have is the one that is with you" is more than true when speaking about taking photos daily wherever you are, when the opportunity to perpetuate the moment represents itself.

Developing applications that provide advanced photography possibilities is up to both the camera hardware of devices and the photo capture API provided by the platform. Nokia's new Lumia series devices come with the new Windows Phone 8 advanced photo capture API that is all about being able to develop photography applications that have extensive access to adjusting the device's camera hardware parameters, like exposure time, ISO, focus position and white balance.

Referring to an extensive example demo application, this article describes how to initialise the new camera into use, how to read the parameters that are supported by the device in question, how to set parameters, and how to capture and store photos.

Windows Phone 8 Advanced Photo Capture API

Windows Phone 8 comes with an all new Advanced Photo Capture API for advanced photography and high configurability of camera parameters. The main access point to new camera features is Windows.Phone.Media.Capture.PhotoCaptureDevice class that exposes methods to:

  • Query and adjust supported camera parameters.
    • static CameraCapturePropertyRange SupportedPropertyRange(CameraSensorLocation sensor, Guid propertyId)
    • static IReadOnlyList<object> GetSupportedPropertyValues(CameraSensorLocation sensor, Guid propertyId)
    • object GetProperty(Guid propertyId)
    • void SetProperty(Guid propertyId, object value)
  • Execute autofocus, auto white balance and auto exposure.
    • static bool IsFocusSupported(CameraSensorLocation sensor)
    • static bool IsFocusRegionSupported(CameraSensorLocation sensor)
    • IAsyncOperation<CameraFocusStatus> FocusAsync()
    • IAsyncOperation<CameraFocusStatus> ResetFocusAsync()
  • Check and set preview and capture resolutions.
    • static IReadOnlyList<Size> GetAvailablePreviewResolutions()
    • static IReadOnlyList<Size> GetAvailableCaptureResolutions()
    • IAsyncAction SetPreviewResolutionAsync(Size value)
    • IAsyncAction SetCaptureResolutionAsync(Size value)
  • Read preview buffer.
    • event TypedEventHandler<ICameraCaptureDevice, Object> PreviewFrameAvailable
    • void GetPreviewBufferArgb(out int[] pixels)
    • void GetPreviewBufferY(out byte[] pixels)
    • void GetPreviewBufferYCbCr(out byte[] pixels)
  • Create capture sequences that are then used to capture frames.
    • CameraCaptureSequence CreateCaptureSequence(uint numberOfFrames)
    • IAsyncAction PrepareCaptureSequenceAsync(CameraCaptureSequence sequence)

Accessing camera parameters happens basically in three different ways depending on the parameter. Preview and capture resolutions are read and set via the specialised asynchronous methods. Likewise the focusing operations have their own specialised methods in the API. Photography specific parameters, however, are adjusted via the use of a GUID that identifies a parameter with methods that report the supported range or set of values for a parameter in question, and a method to set one of the supported values active.

Photography specific parameters are either ranges or sets of values, arrays. Range type of parameters, for example ISO, define a numeric minimum and maximum values, having any value in between as a valid value for the parameter. Array type of parameters consist of a list of possible values, typically values from an enumeration, that are valid for the parameter in the device in question.

Also notice that ID_CAP_ISV_CAMERA application capability is needed in the WMAppManifest.xml for the above to work. In addition to this ID_CAP_MEDIALIB_PHOTO is needed for saving to Media Library to work.

Nokia Lumia camera properties

Here is a table of the most important supported image related camera properties on Nokia Lumia 820 and 920 devices.

Device Nokia Lumia 820 Nokia Lumia 920
Sensor Front Back Front Back
Autofocus range Infinity Auto, Macro, Normal, Full, Hyperfocal, Infinity Infinity Auto, Macro, Normal, Full, Hyperfocal, Infinity
Preview resolution 640x480 800x448, 640x480 1280x720, 1024x768 1280x720, 1024x768
Capture resolution 640x480 3264x2448, 3552x2000, 2592x1936, 2592x1456, 2048x1536, 640x480 1280x960, 1280x720, 640x480 3264x2448, 3552x2000, 2592x1936, 2592x1456, 2048x1536, 640x480
Exposure compensation (EV) -12...12 -12...12 -12...12 -12...12
Exposure time (microseconds) 1...33333 1...500000 1...33333 1...500000
Flash mode Off Auto, On, Off Off Auto, On, Off
Focus illumination mode Off Auto, On, Off Off Auto, On, Off
ISO 100...800 100...800 100...800 100...800
Manual focus position No Yes, 1000 positions No Yes, 1000 positions
Scene mode Auto, Sport, Night, Backlit Auto, Macro, Sport, Night, Night Portrait, Backlit Auto, Sport, Night, Backlit Auto, Macro, Sport, Night, Night Portrait, Backlit
White balance preset Cloudy, Daylight, Fluorescent, Tungsten Cloudy, Daylight, Fluorescent, Tungsten Cloudy, Daylight, Fluorescent, Tungsten Cloudy, Daylight, Fluorescent, Tungsten

Camera Explorer application

Camera Explorer example application is a practical example of how the new PhotoCaptureDevice can be used. The application contains a live viewfinder, a settings page with numerous adjustable parameters, and a preview page to display and provide means to save the captured photo.

In order to have an understanding of the whole picture before diving into example code, let's take a look at the Camera Explorer architecture.

Figure 1. Camera Explorer application architecture

In addition to the four main Windows Phone application pages, there is a singleton DataContext for owning the application widely used objects, like an instance of Settings and an instance of the PhotoCaptureDevice. The most interesting parts of the application for this article, however, are PhotoCaptureDevice initialisation code in DataContext, Parameters that have been created for this particular application to access the PhotoCaptureDevice properties, and the actual photo capture process that has been put in the MainPage which also contains the live viewfinder. Of all the parameters in Camera Explorer application, two parameters of different type have been selected to be covered in this article.

Using excerpts from the Camera Explorer code, let's take a look at how the PhotoCaptureDevice can be used in practice. Notice that the whole Camera Explorer application is not covered here and the code displayed here has been simplified for the purposes of this article. For the complete application source code please visit the project space.

Going forward, keep in mind the DataContext that holds the instances that are needed on multiple different pages, as it will be referenced in the following sections.

...

class DataContext : INotifyPropertyChanged
{
    ...

    public static DataContext Singleton { ... }

    public ObservableCollection<Parameter> Parameters { ... }

    public PhotoCaptureDevice Device { ... }

    public MemoryStream ImageStream { ... }
}

Capturing photos

Let's check how the PhotoCaptureDevice can be initialised with a live viewfinder and a possibility to snap pictures. In the MainPage XAML there is a Canvas element with a VideoBrush element rendering the background of the canvas.

<phone:PhoneApplicationPage x:Class="CameraExplorer.MainPage" ... >
    ...

    <Grid x:Name="LayoutRoot" ... >
        ...

        <Grid x:Name="ContentPanel" ... >
            <Canvas x:Name="VideoCanvas">
                <Canvas.Background>
                    <VideoBrush x:Name="videoBrush"/>
                </Canvas.Background>
            </Canvas>

            ...
        </Grid>
    </Grid>

    <phone:PhoneApplicationPage.ApplicationBar>
        <shell:ApplicationBar>
            <shell:ApplicationBarIconButton Text="capture" Click="captureButton_Click" ... />
     
            ...
        </shell:ApplicationBar>
    </phone:PhoneApplicationPage.ApplicationBar>

    ...
</phone:PhoneApplicationPage>

In the MainPage C# code the PhotoCaptureDevice is initialised in the InitializeCamera method, and the camera instance is stored to DataContext. In the event handler for the capture button that was declared in XAML we can see how a photo can be captured.

...

public partial class MainPage : PhoneApplicationPage
{
    private CameraExplorer.DataContext _dataContext = CameraExplorer.DataContext.Singleton;

    ...

    protected async override void OnNavigatedTo(NavigationEventArgs e)
    {
        if (_dataContext.Device == null)
        {
            ...

            await InitializeCamera(CameraSensorLocation.Back);

            ...
        }

        videoBrush.RelativeTransform = new CompositeTransform()
        {
            CenterX = 0.5,
            CenterY = 0.5,
            Rotation = _dataContext.Device.SensorLocation == CameraSensorLocation.Back ?
                       _dataContext.Device.SensorRotationInDegrees :
                     - _dataContext.Device.SensorRotationInDegrees
        };

        videoBrush.SetSource(_dataContext.Device);

        ...
    }

    private async Task InitializeCamera(CameraSensorLocation sensorLocation)
    {
        Windows.Foundation.Size initialResolution = new Windows.Foundation.Size(640, 480);

        PhotoCaptureDevice d = await PhotoCaptureDevice.OpenAsync(sensorLocation, initialResolution);

        d.SetProperty(KnownCameraGeneralProperties.EncodeWithOrientation,
                      d.SensorLocation == CameraSensorLocation.Back ?
                      d.SensorRotationInDegrees : -d.SensorRotationInDegrees);

        _dataContext.Device = d;
    }

    private async void captureButton_Click(object sender, EventArgs e)
    {
        if (!_manuallyFocused)
        {
            await AutoFocus();
        }
        await Capture();
    }

    private async Task AutoFocus()
    {
        if (!_capturing && PhotoCaptureDevice.IsFocusSupported(_dataContext.Device.SensorLocation))
        {
            ...
            await _dataContext.Device.FocusAsync();
            ...
            _capturing = false;
        }
    }
    
    private async Task Capture()
    {
        if (!_capturing)
        {
            _capturing = true;
            MemoryStream stream = new MemoryStream();
            CameraCaptureSequence sequence = _dataContext.Device.CreateCaptureSequence(1);
            sequence.Frames[0].CaptureStream = stream.AsOutputStream();
            await _dataContext.Device.PrepareCaptureSequenceAsync(sequence);
            await sequence.StartCaptureAsync();
            _dataContext.ImageStream = stream;
            ...
        }
        ...
    }

    ...
}

Displaying preview and saving to camera roll

Rendering a preview of the captured image is simple. Camera Explorer application does this using an Image XAML element with a BitmapImage object. Further, saving the captured image to camera roll is simple too, just remember that the image stream that goes as an argument to void MediaLibrary.SavePictureToCameraRoll(string name, Stream source) must be positioned to the beginning of the image data.

<phone:PhoneApplicationPage x:Class="CameraExplorer.PreviewPage" ... >
    ...

    <Grid x:Name="LayoutRoot" ... >
        ...

        <Grid x:Name="ContentPanel" ... >
            <Image x:Name="image" Stretch="UniformToFill"/>
        </Grid>
    </Grid>

    <phone:PhoneApplicationPage.ApplicationBar>
        <shell:ApplicationBar>
            <shell:ApplicationBarIconButton Text="save" Click="saveButton_Click" ... />
        </shell:ApplicationBar>
    </phone:PhoneApplicationPage.ApplicationBar>

    ...
</phone:PhoneApplicationPage>
...

public partial class PreviewPage : PhoneApplicationPage
{
    private CameraExplorer.DataContext _dataContext = CameraExplorer.DataContext.Singleton;
    private BitmapImage _bitmap = new BitmapImage();

    protected override void OnNavigatedTo(NavigationEventArgs e)
    {
        ...

        _bitmap.SetSource(_dataContext.ImageStream);

        image.Source = _bitmap;

        ...
    }

    private void saveButton_Click(object sender, EventArgs e)
    {
        try
        {
            _dataContext.ImageStream.Position = 0;

            MediaLibrary library = new MediaLibrary();

            library.SavePictureToCameraRoll("CameraExplorer_" + DateTime.Now.ToString() + ".jpg",
                                            _dataContext.ImageStream);
        }
        catch (Exception)
        {
            ...
        }

        ...
    }
}

Adjusting camera properties

Camera Explorer application architecture is such that the camera parameters are divided into ArrayParameters and RangeParameters, both derived from a base class called Parameter. The parameter architecture in the application provides easy and extensible way to handle parameters and render them to the screen by binding the parameter objects directly with UI controls like ComboBox, Slider and Text. Binding is not covered in this article, but you can find more about that from the applications project space.

Here's the simplified Parameter base class with just couple of elements that are common to both parameter types.

...

public abstract class Parameter : INotifyPropertyChanged
{    
    ...
   
    protected Parameter(PhotoCaptureDevice device, string name)
    {
        ...
    }
 
    public PhotoCaptureDevice Device { ... }

    public string Name { ... }

    public bool Supported { ... }

    public abstract void Refresh();

    ...
}

Now let's take a look at how the range type of properties can be handled. Reading the range minimum and maximum for a property identified by _propertyId is done by calling the static CameraCapturePropertyRange PhotoCaptureDevice.GetSupportedPropertyRange(CameraSensorLocation sensor, Guid propertyId) method and inspecting the return value. The current value of the property is read with a call to the instance method object PhotoCaptureDevice.GetProperty(Guid propertyId). Similarly, setting a new value for the property can be done by calling void PhotoCaptureDevice.SetProperty(Guid propertyId, object value).

...

public abstract class RangeParameter<T> : Parameter
{
    private Guid _propertyId;
    private T _value;

    ...

    protected RangeParameter(PhotoCaptureDevice device, Guid propertyId, string name)
        : base(device, name)  
    {
        ...
    }

    public override void Refresh()
    {
        try
        {
            CameraCapturePropertyRange range = PhotoCaptureDevice.GetSupportedPropertyRange(
                Device.SensorLocation, _propertyId);
            
            if (range == null)
            {
                Supported = false;
            }
            else
            {
                Minimum = (T)range.Min;
                Maximum = (T)range.Max;
                
                _value = (T)Device.GetProperty(_propertyId);
                
                Supported = true;
            }
        }
        catch (Exception)
        {
            Supported = false;
        }

        ... 
    }
    
    public T Minimum { ... }

    public T Maximum { ... }

    public T Value
    {
        ...

        set
        {
            if (!_value.Equals(value))
            {
                try
                {
                    Device.SetProperty(_propertyId, (T)value);
  
                    _value = value;

                    ...
                }
                catch (Exception)
                {
                    ...
                }
            }
        }
    }

    ...
}

As a concrete example of a range parameter, here is the definition of ExposureCompensationParameter, where the GUID of the RangeParameter is set to KnownCameraPhotoProperties.ExposureCompensation, a value defined in the platform. Also from the platform documentation we know that the actual type of exposure compensation property is Int32, so that is used as a template argument for our range parameter implementation.

...

public class ExposureCompensationParameter : RangeParameter<Int32>
{
    public ExposureCompensationParameter(PhotoCaptureDevice device)
        : base(device, KnownCameraPhotoProperties.ExposureCompensation, "Exposure compensation")
    {
    }

    ...
}

Camera Explorer application handles array type of properties with ArrayParameter. Here is the simplified definition of it for the sake of setting up the code context. The really interesting parts are the protected abstract methods void PopulateOptions() and void SetOption(ArrayParameterOption option) that we'will look into right after this code excerpt.

...

public class ArrayParameterOption
{
    ...
    
    public ArrayParameterOption(dynamic value, string name)
    {
        ...
    }

    public dynamic Value { ... }

    public string Name { ... }

    ...
}

public abstract class ArrayParameter : Parameter, IReadOnlyCollection<ArrayParameterOption>
{ 
    private List<ArrayParameterOption> _options = new List<ArrayParameterOption>();
    private ArrayParameterOption _selectedOption;

    ...

    public ArrayParameter(PhotoCaptureDevice device, string name)
        : base(device, name)
    {
        ...
    }

    public override void Refresh()
    {
        ...

        _options.Clear();

        _selectedOption = null;

        try
        {
            PopulateOptions();

            Supported = _options.Count > 0;
        }
        catch (Exception)
        {
            Supported = false;
        }

        ...
    }

    protected List<ArrayParameterOption> Options { ... }

    public ArrayParameterOption SelectedOption
    {
        ...
 
        set
        {
            if (_selectedOption != value)
            {
                ...

                if (_selectedOption != null))
                {
                    SetOption(value);
                }

                _selectedOption = value; 
            }
        }
    }

    protected abstract void PopulateOptions();

    protected abstract void SetOption(ArrayParameterOption option);

    ...
}

Now that we have the base for all the array type of parameters laid out, let's take a look at a concrete piece of code that reads out and sets the scene mode camera property. Reading the supported values and selecting the currently active value is in the void PopulateOptions() method, while setting a new value according to a change in the ArrayParameterOption SelectedOption property is in the void SetOption(ArrayParameterOption option) method. Notice how static IReadOnlyList<object> PhotoCaptureDevice.GetSupportedPropertyValues(CameraSensorLocation sensor, Guid propertyId) is used to read the KnownCameraPhotoProperties.SceneMode supported values and object PhotoCaptureDevice.GetProperty(Guid propertyId) is used to read the current value. Setting a new scene mode value is achieved by simply calling the void PhotoCaptureDevice.SetProperty(Guid propertyId, object value) method.

...

public class SceneModeParameter : ArrayParameter
{
    public SceneModeParameter(PhotoCaptureDevice device)
        : base(device, KnownCameraPhotoProperties.SceneMode, "Scene mode")
    {
    }

    protected override void PopulateOptions()
    {
        IReadOnlyList<object> supportedValues = PhotoCaptureDevice.GetSupportedPropertyValues(
            Device.SensorLocation, PropertyId);

        object value = Device.GetProperty(PropertyId);

        foreach (dynamic i in supportedValues)
        {
            CameraSceneMode csm = (CameraSceneMode)i;

            ArrayParameterOption option = new ArrayParameterOption(csm, csm.ToString());

            Options.Add(option);

            if (i.Equals(value))
            {
                SelectedOption = option;
            }
        }
    }

    protected override void SetOption(ArrayParameterOption option)
    {
        Device.SetProperty(PropertyId, option.Value);
    }

    ...
}

Tap to focus

By default, the app focuses and meters the scene when the shutter button is pressed. Tap-to-focus is the alternative behavior, where a finger tap sets the camera focus area, triggers focusing, and locks the focus, exposure, and white balance settings for the next shot. A focus indicator is displayed when tap-to-focus is active.

The focus indicator is a Rectangle element within the Canvas where the viewfinder image is displayed:

<Rectangle x:Name="FocusIndicator"
           Stroke="Red" Opacity="0.7" Width="80" Height="80" StrokeThickness="5"
           Visibility="Collapsed"/>

Collapsed visibility means that the element is initially hidden from view.

Tapping is detected by registering an event handler for the Canvas Tap event:

public MainPage()
{
    ...
    VideoCanvas.Tap += new EventHandler<GestureEventArgs>(videoCanvas_Tap);
    ...
}

The event handler has to do some extra work when determining the focus area. This is due to incompatible types used by the APIs and the fact that the VideoCanvas that is used to display the viewfinder image is not the same size as the viewfinder behind the scenes.

The handler compensates for the resolution difference by applying constant multipliers to coordinates. Then the focus region is clipped to the viewfinder preview image resolution, since trying to set a focus region that does not fall entirely within the area of the preview image would crash the app with an exception.

Finally the focus indicator is moved to the correct location and displayed, first in red while focusing is in progress, and in green when focus is locked.

private async void videoCanvas_Tap(object sender, GestureEventArgs e)
{
    System.Windows.Point uiTapPoint = e.GetPosition(VideoCanvas);
    if (PhotoCaptureDevice.IsFocusRegionSupported(_dataContext.Device.SensorLocation)
        && _focusSemaphore.WaitOne(0))
    {
        // Get tap coordinates as a foundation point
        Windows.Foundation.Point tapPoint = new Windows.Foundation.Point(uiTapPoint.X, uiTapPoint.Y);
        double xRatio = VideoCanvas.ActualWidth / _dataContext.Device.PreviewResolution.Width;
        double yRatio = VideoCanvas.ActualHeight / _dataContext.Device.PreviewResolution.Height;
        
        // adjust to center focus on the tap point
        Windows.Foundation.Point displayOrigin = new Windows.Foundation.Point(
            tapPoint.X - _focusRegionSize.Width / 2,
            tapPoint.Y - _focusRegionSize.Height / 2);
        
        // adjust for resolution difference between preview image and the canvas
        Windows.Foundation.Point viewFinderOrigin = new Windows.Foundation.Point(
            displayOrigin.X / xRatio, displayOrigin.Y / yRatio);
        Windows.Foundation.Rect focusrect = new Windows.Foundation.Rect(
            viewFinderOrigin, _focusRegionSize);
        
        // clip to preview resolution
        Windows.Foundation.Rect viewPortRect = new Windows.Foundation.Rect(
            0, 0,
            _dataContext.Device.PreviewResolution.Width,
            _dataContext.Device.PreviewResolution.Height);
        focusrect.Intersect(viewPortRect);
        _dataContext.Device.FocusRegion = focusrect;

        // show a focus indicator
        FocusIndicator.SetValue(Shape.StrokeProperty, _notFocusedBrush);
        FocusIndicator.SetValue(Canvas.LeftProperty, uiTapPoint.X - _focusRegionSize.Width / 2);
        FocusIndicator.SetValue(Canvas.TopProperty, uiTapPoint.Y - _focusRegionSize.Height / 2);
        FocusIndicator.SetValue(Canvas.VisibilityProperty, Visibility.Visible);

        CameraFocusStatus status = await _dataContext.Device.FocusAsync();
        if (status == CameraFocusStatus.Locked)
        {
            FocusIndicator.SetValue(Shape.StrokeProperty, _focusedBrush);
            _manuallyFocused = true;
            _dataContext.Device.SetProperty(
                KnownCameraPhotoProperties.LockedAutoFocusParameters,
                AutoFocusParameters.Exposure & AutoFocusParameters.Focus & AutoFocusParameters.WhiteBalance);
        }
        else
        {
            _manuallyFocused = false;
            _dataContext.Device.SetProperty(
                KnownCameraPhotoProperties.LockedAutoFocusParameters,
                AutoFocusParameters.None);
        }
        _focusSemaphore.Release();
    }
}

Focusing is also protected by a semaphore, so that autofocus is not triggered more than once at any given time. If we tried to call FocusAsync while the previous operation was still in progress, the app would crash with an exception. The zero given as a parameter to the WaitOne method of the semaphore means that if the semaphore is not available, the call will return false immediately instead of waiting for the semaphore to become available.

Lens picker integration

The default camera app on the device allows launching extensions called Lenses via the Lens Picker. Lenses are full camera apps in their own right, and usually provide image processing capabilities above and beyond those of the regular camera. To make the app available as a Lens in the Lens Picker, the following XML snippet is added to the application manifest, within the App element:

<Extensions>
    <Extension ExtensionName="Camera_Capture_App"
               ConsumerId="{5B04B775-356B-4AA0-AAF8-6491FFEA5631]"
               TaskId="_default" />
</Extensions>

This way when the app is selected within the Lens Picker, the app is launched and it shows the default page. In addition, icons for the Lens Picker are provided in the Assets folder. The icons have to have specific sizes and filenames which are documented in Lens design guidelines for Windows Phone (MSDN).

Further reading

In this article we have taken a brief look at the advanced photography on Nokia's Windows Phone 8 devices.

To learn more, check out for example the following sections in MSDN:

Windows Phone Multimedia portal on Nokia Developer Wiki also contains a number of photography related articles, resources and active discussion.


Last updated 11 July 2013

Back to top