×
Namespaces

Variants
Actions

FilterSquare: Using Nokia Imaging SDK to create an innovative Filter App for Windows Phone 8

From Nokia Developer Wiki
Jump to: navigation, search

This article provides a detailed description of the steps required to apply one or more Nokia Imaging SDK filters to user selected region(s) in a photo.

SignpostIcon XAML 40.png
WP Metro Icon WP8.png
Article Metadata
Code ExampleTested with
SDK: Windows Phone 8.0 SDK, Nokia Imaging SDK Beta 1
Compatibility
Platform(s):
Windows Phone 8
Platform Security
Capabilities: ID_CAP_ISV_CAMERA, ID_CAP_MEDIALIB_PHOTO
Article
Created: RatishPhilip (31 Jul 2013)
Last edited: hamishwillee (14 Oct 2013)


Contents

Introduction

Nokia has been a pioneer in the field of Imaging and it brought lots of excitement in the development community when it planned to make some of the imaging technologies, that powers their own imaging application, available to developers in the form of The Nokia Imaging SDK. Through this SDK, developers (including me) have found an easy solution to perform various image manipulations on images captured by mobile devices. This efficient library includes features such as decoding and encoding JPEG images, applying filters and effects, cropping, rotating and resizing. In the current mobile era, where speed matters, this SDK comes out with flying colors, thanks to meticulous memory and code optimization.

The biggest advantage of the SDK in my opinion is its ease of use. By adding just a few lines of code, this SDK can be integrated with any Windows Phone 8 project and with just a few API calls it is possible to add filters to images in a jiffy. Moreover, this SDK comes pre-loaded with more than 50 filters which can help a developer make an imaging app which can give serious competition to other popular mobile imaging apps out in the market.

In this article, I will explain how, using the Nokia Imaging SDK, I went from a single idea and evolved it into a complete working application.

The Idea

The idea which popped up in my head a few weeks ago is: Normally a filter is applied to an entire image. How do you apply a filter to a specific region within the image? And, once you accomplish that, how do you apply several filters in several regions within the same image? Before I give you the long detailed answer to these questions, I will give you a short and concise answer to it: Nokia Imaging SDK!

Prerequisites

  • Windows 8 (or Windows 8.1 Preview)
  • Visual Studio 2012 (or Visual Studio 2013 preview)
  • Windows Phone 8.0 SDK
  • Nokia Imaging SDK
  • Windows Phone Toolkit - This library provides the developer community with new components, functionality, and an efficient way to help shape product development. It includes open source code, samples & docs, plus design-time support for the Windows Phone platform.
  • WriteableBitmapEx - A very useful library. It is a collection of extension methods for the WriteableBitmap class. The library extends the WriteableBitmap class with elementary and fast (2D drawing) functionality, conversion methods and functions to combine (blit) WriteableBitmaps.

Installing the libraries

Adding the libraries as NuGet packages is the simplest method. Just right click on the References folder in the solution and select Manage NuGet Packages"

Adding NuGet Packages

This will bring up the Manage NuGet Packages dialog, wherein you can search for the required packages and install them.

Search for NuGet packages

Note.pngNote: After installing the NuGet package for Nokia Imaging SDK, a few more steps need to be performed to ensure that it works correctly. Refer adding libraries to the project for more details.

The App

FilterSquare

The App is called FilterSquare. One of the meanings of the word Square is an open area or plaza in a city or town. Similarly the app is the place where the user would find a lot of filters for manipulating mobile based images.Also in the logo, there is a circle and a square depicting the two selection frames available in the app which are used for selecting a region within a photo.

Capabilities

The app must have the following capabilities defined in the WMAppManifest.xml

  • ID_CAP_ISV_CAMERA - For access to the camera
  • ID_CAP_MEDIALIB_PHOTO - For access to the camera roll and albums

The App Architecture

The app has a MVVM architecture. The components are depicted in the diagram below

FilterSquare components

The main classes within these components are depicted in the below diagram

FilterSquare classes

The Nokia Developer Example Projects have proved to be very helpful in gaining an insight into the features of the Nokia Imaging SDK and this project is also influenced by the FilterExplorer project as I am reusing the FilterModel and FiltersModel classes from that project. More details ahead.

View

The View mainly consists of three Pages:

  • The HomePage: where the FilterSquare logo and two buttons (one to browse the photos and the other to capture a photo using a camera) are displayed.
public partial class HomePage : StateAwarePage
{
...
 
private void SelectPhotoFromMediaLibrary(object sender, RoutedEventArgs e)
{
PhotoChooserTask task = new PhotoChooserTask();
EventHandler<PhotoResult> handler = null;
handler = (s, ea) =>
{
task.Completed -= handler;
if (ea.TaskResult == TaskResult.OK)
{
BitmapImage img = new BitmapImage();
img.SetSource(ea.ChosenPhoto);
PhotoManager.Instance.ViewModel.ChosenPhoto = new WriteableBitmap(img);
 
NavigationService.Navigate(new Uri("/Views/PhotoPage.xaml", UriKind.Relative));
}
 
InnerGrid.Visibility = Visibility.Visible;
};
 
InnerGrid.Visibility = Visibility.Collapsed;
task.Completed += handler;
task.ShowCamera = false;
task.Show();
}
 
private void CapturePhotoFromCamera(object sender, RoutedEventArgs e)
{
CameraCaptureTask task = new CameraCaptureTask();
EventHandler<PhotoResult> handler = null;
handler = (s, ea) =>
{
task.Completed -= handler;
if (ea.TaskResult == TaskResult.OK)
{
BitmapImage img = new BitmapImage();
img.SetSource(ea.ChosenPhoto);
PhotoManager.Instance.ViewModel.ChosenPhoto = new WriteableBitmap(img);
 
NavigationService.Navigate(new Uri("/Views/PhotoPage.xaml", UriKind.Relative));
}
 
InnerGrid.Visibility = Visibility.Visible;
};
 
InnerGrid.Visibility = Visibility.Collapsed;
task.Completed += handler;
task.Show();
}
}
  • The PhotoPage: where the main action happens. This shows the selected photo and provides a few options (such as add filter to whole image, select a rectangular region or oval region, undo, save, reset) which the user can use to select and apply multiple filters on the image.
public partial class PhotoPage : StateAwarePage
{
...
 
void OnAddFilter(object sender, EventArgs e)
{
Rect region = photoFrame.GetFullImageBounds();
 
PhotoManager.Instance.ViewModel.SetRegion(region, false);
 
NavigationService.Navigate(new Uri("/Views/FilterGallery.xaml", UriKind.Relative));
}
 
void OnRectFrameSelected(object sender, EventArgs e)
{
UpdateAppBarIcons(true);
 
RectFilterFrame frame = new RectFilterFrame
{
Width = 200,
Height = 200,
ThumbWidth = 60,
ThumbHeight = 60,
MinWidth = 20,
MinHeight = 20,
ThumbMargin = new Thickness(-25)
};
Canvas.SetLeft(frame, (photoFrame.canvasFrame.ActualWidth - 200) / 2);
Canvas.SetTop(frame, (photoFrame.canvasFrame.ActualHeight - 200) / 2);
photoFrame.AddFilterFrame(frame);
}
 
void OnOvalFrameSelected(object sender, EventArgs e)
{
UpdateAppBarIcons(true);
 
OvalFilterFrame frame = new OvalFilterFrame
{
Width = 200,
Height = 200,
ThumbWidth = 60,
ThumbHeight = 60,
MinWidth = 20,
MinHeight = 20,
ThumbMargin = new Thickness(-25)
};
Canvas.SetLeft(frame, (photoFrame.canvasFrame.ActualWidth - 200) / 2);
Canvas.SetTop(frame, (photoFrame.canvasFrame.ActualHeight - 200) / 2);
photoFrame.AddFilterFrame(frame);
}
 
async void OnUndo(object sender, EventArgs e)
{
await PhotoManager.Instance.ViewModel.UndoAsync();
}
 
void OnDoneFraming(object sender, EventArgs e)
{
UpdateAppBarIcons(false);
Rect region = photoFrame.GetFilterFrameBounds();
 
PhotoManager.Instance.ViewModel.SetRegion(region, photoFrame.IsFilterFrameOval());
 
photoFrame.RemoveFilterFrame();
 
NavigationService.Navigate(new Uri("/Views/FilterGallery.xaml", UriKind.Relative));
}
 
void OnCanceFraming(object sender, EventArgs e)
{
UpdateAppBarIcons(false);
photoFrame.RemoveFilterFrame();
}
 
void OnReset(object sender, EventArgs e)
{
PhotoManager.Instance.ViewModel.Reset();
}
 
async void OnSavePhoto(object sender, EventArgs e)
{
await PhotoManager.Instance.ViewModel.SavePhotoAsync();
}
 
protected override void OnNavigatedTo(NavigationEventArgs e)
{
// This is to prevent the user from going to the SplashScreen
// when he presses back in this Page. This way the
// application will exit when the user presses back key at the main menu.
if (e.NavigationMode == NavigationMode.New)
{
NavigationService.RemoveBackEntry();
}
 
//JournalEntry entry = NavigationService.BackStack.ElementAtOrDefault(0) as JournalEntry;
//if (entry != null)
//{
// var eStr = entry.Source.ToString();
//}
 
base.OnNavigatedTo(e);
}
 
protected override void LoadState(IObservableMap<string, object> pageState)
{
Binding b = new Binding();
b.Path = new PropertyPath("CanUndo");
BindingOperations.SetBinding(this, PhotoPage.CanUndoProperty, b);
 
this.DataContext = PhotoManager.Instance.ViewModel;
}
}
  • The FilterGallery Page: where several images corresponding to the selected region (each shown with a different filter applied) is displayed. The user can tap on any one of them to apply the appropriate filter. I have reused the FilterPage from the FilterExplorer project to show the list of filters.


These pages derive from StateAwarePage which provides the basic framework for tombstoning and recovery of the app.

Controls

The following custom controls are being used:

  • PhotoFrame: It is a user control which encapsulates a ViewportControl which contains a canvas containing the image. This control is used in the PhotoPage to display the photo chosen by the user. It allows pinch/zoom manipulation. The ViewportControl has very little documentation online. However, I found this link to be very useful in getting the knowhow about how to implement the pinch/zoom manipulation.
  • RectFilterFrame: This control is used to display a rectangular selection area on top of the image in the PhotoFrame. This allows the user to select a rectangular area within the photo where the filter can be applied. It can be resized using the handles in the corners and middle.
  • OvalFilterFrame: This control is used to display a circular selection area on top of the image in the PhotoFrame. This allows the user to select a circular area within the photo where the filter can be applied.It can be resized using the handles in the corners and middle.
  • PhotoThumbnail: This control is taken from the FilterExplorer example and is used to display the thumbnails of the images with filters applied in the FilterGallery page.

View Model

The PhotoViewModel class is the main View Model class to which all the pages in the View bind to. It provides APIs to the View using which a filter can be applied to a region within the photo. In order to apply a filter to a region, first a WriteableBitmap of that region is obtained and then an Editable session is created with that region and the required filter(s) applied. Once the filters are applied, this writeablebitmap is blitted on top of the original image and the result is shown. The Blit() method is an extension method available in the WriteableBitmapEx library.

public class PhotoViewModel : BindableBase
{
...
 
#region APIs
 
public void SetRegion(Rect region, bool isOval)
{
if (!region.IsEmpty)
{
_currentRegionRect = region;
CurrentRegion = PhotoEngine.CreateRegion(DisplayPhoto, region, isOval);
}
}
 
public void ClearRegion()
{
CurrentRegion = null;
}
 
async public Task ApplyFilterToRegionAsync(FilterModel filter)
{
EditingSession session = new EditingSession(CurrentRegion.AsBitmap());
 
foreach (IFilter f in filter.Components)
{
session.AddFilter(f);
}
 
WriteableBitmap regionBmp = BitmapFactory.New((int)_currentRegionRect.Width, (int)_currentRegionRect.Height);
await session.RenderToWriteableBitmapAsync(regionBmp);
DisplayPhoto.Blit(_currentRegionRect, regionBmp, new Rect(0, 0, _currentRegionRect.Width, _currentRegionRect.Height));
 
_filterOpStack.Push(new FilterOp { Session = session, Frame = _currentRegionRect, FilterName = filter.Name });
 
CurrentRegion = null;
_currentRegionRect = Rect.Empty;
 
CheckUndoStatus();
}
 
async public Task UndoAsync()
{
if (_filterOpStack.Count > 0)
{
FilterOp op = _filterOpStack.Peek();
if ((op.Session != null) && (op.Session.CanUndo()))
{
op.Session.Undo();
WriteableBitmap sourceBmp = BitmapFactory.New((int)op.Frame.Width, (int)op.Frame.Height);
await op.Session.RenderToWriteableBitmapAsync(sourceBmp);
DisplayPhoto.Blit(op.Frame, sourceBmp, new Rect(0, 0, (int)op.Frame.Width, (int)op.Frame.Height));
 
DisplayPhoto.Invalidate();
 
if (!op.Session.CanUndo())
{
op.Session.Dispose();
_filterOpStack.Pop();
}
 
CheckUndoStatus();
}
}
}
 
async public Task<SerializedViewModel> SerializeAsync()
{
SerializedViewModel svm = new SerializedViewModel();
 
if (this.ChosenPhoto != null)
{
WriteableBitmap sourceBmp = new WriteableBitmap(this.ChosenPhoto);
EditingSession session = new EditingSession(sourceBmp.AsBitmap());
 
svm.BufferData = await session.RenderToJpegAsync();
svm.FilterOps = new List<SerializedFilterOp>();
foreach (FilterOp op in _filterOpStack.ToArray())
{
SerializedFilterOp sfo = new SerializedFilterOp
{
FilterName = op.FilterName,
FrameRect = op.Frame
};
svm.FilterOps.Add(sfo);
}
 
// Reverse the order so that during deserialization,
// they are pushed into the stack in the correct order
svm.FilterOps.Reverse();
}
 
return svm;
}
 
async public void DeserializeAsync(SerializedViewModel svm)
{
if (svm != null)
{
EditingSession session = new EditingSession(svm.BufferData);
WriteableBitmap destBmp = BitmapFactory.New((int)session.Dimensions.Width, (int)session.Dimensions.Height);
await session.RenderToWriteableBitmapAsync(destBmp);
 
this.ChosenPhoto = destBmp;
 
foreach (SerializedFilterOp sfo in svm.FilterOps)
{
FilterModel filter = GetFilter(sfo.FilterName);
if (filter != null)
{
// TODO: Not always false
SetRegion(sfo.FrameRect, false);
await ApplyFilterToRegionAsync(filter);
}
}
}
}
 
#endregion
}

Besides the PhotoViewModel class, there are two other classes: SerializedViewModel and SerializedFilterOp which are used for serializing the PhotoViewModel while tombstoning.

Model

I have reused the FilterModel and FiltersModel classes from the FilterExplorer example. They encapsulate the various types of Filters available in the Nokia Imaging SDK.
Apart from these two classes, the FilterOp class encapsulates the filter operation that was done by the user for a particular region. It stores the EditingSession, the region dimension, the name of the Filter that was applied to the region and whether the region is rectangular or oval.

internal class FilterOp
{
public EditingSession Session { get; set; }
public Rect Frame { get; set; }
public string FilterName { get; set; }
public bool IsOval { get; set; }
}

The PhotoEngine class is a utility class which contains the API which is used to create rectangular or oval regions from a larger WriteableBitmap.

internal static class PhotoEngine
{
internal static WriteableBitmap CreateRegion(BitmapSource photo, Rect region, bool isOval)
{
WriteableBitmap source = new WriteableBitmap(photo);
WriteableBitmap croppedImage = null;
if (isOval)
{
croppedImage = source.CropEllipse(region);
}
else
{
croppedImage = source.Crop(region);
}
 
return croppedImage;
}
}

The PhotoManager class acts as the main interface of the ViewModel and Model with the other components. It is a singleton class. It holds instances of the ViewModel and the Model.

internal class PhotoManager
{
static PhotoManager _instance = new PhotoManager();
 
public static PhotoManager Instance
{
get { return _instance; }
}
 
public PhotoViewModel ViewModel { get; set; }
 
public FiltersModel Filters { get; private set; }
 
PhotoManager()
{
ViewModel = new PhotoViewModel();
Filters = new FiltersModel();
}
}

Helpers

The WriteableBitmapEx library provides a lot of useful extension methods for performing manipulations on the WriteableBitmap. This library contains a Crop method which is used to crop a large image into a smaller image. However, the smaller image is rectangular in shape. I needed a method which could crop in an elliptical manner. The resulting image is still rectangular in shape, but only the pixels within the ellipse have opaque values. The pixels falling outside the Ellipse are transparent. So using the methods provided in this library I created my own extension method for this purpose.

public static class WriteableBitmapExtensions
{
public static WriteableBitmap CropEllipse(this WriteableBitmap bmp, int x, int y, int width, int height)
{
WriteableBitmap cropResult = bmp.Crop(x, y, width, height);
WriteableBitmap cropMask = BitmapFactory.New(width, height);
cropMask.FillEllipse(0, 0, width, height, Colors.Red);
cropResult.Blit(new Rect(0, 0, width, height), cropMask, new Rect(0, 0, width, height), System.Windows.Media.Imaging.WriteableBitmapExtensions.BlendMode.Mask);
 
return cropResult;
}
 
public static WriteableBitmap CropEllipse(this WriteableBitmap bmp, Rect region)
{
return CropEllipse(bmp, (int)region.Left, (int)region.Top, (int)region.Width, (int)region.Height);
}
}

Rich Media Extensibility

In order to extend the built-in photo viewer experience, Rich Media Extensibility has been added to this app. Extensions are specified in the WMAppManifest.xml file. Just after the Tokens element, inside the Extensions element, the rich media extension is specified with the following Extension element.

<Extension ExtensionName="Photos_Rich_Media_Edit"
ConsumerID="{5B04B775-356B-4AA0-AAF8-6491FFEA5632}"
TaskID="_default" />

More details on Rich Media Extensibility can be found in this MSDN article.

Source Code

Download FilterSquare Source Code.

Summary

In a nutshell, Nokia has provided a robust library for image manipulation which not only is powerful and less memory hungry but also wins the hearts of developers due to its ease of use. The ability to apply a dazzling filter to an image in just a few lines of code will appeal to many.

The whole journey from the idea for this app till finishing this article has been a great learning experience for me. I have attached the source code for this application. I intend to add more features in the future and release this as a free app in the Windows MarketPlace.

I am looking forward to the release of the first version of the SDK and I am hoping that Nokia will incorporate more powerful features in this SDK.

This page was last modified on 14 October 2013, at 04:22.
413 page views in the last 30 days.
×