×
Namespaces

Variants
Actions

How to apply filters to specific image region using the Nokia Imaging SDK

From Nokia Developer Wiki
Jump to: navigation, search

This article provides details about applying filters, provided by the Nokia Imaging SDK, to specfic regions of an image.

SignpostIcon XAML 40.png
WP Metro Icon WP8.png
Article Metadata
Code ExampleTested with
SDK: Windows Phone 8.0 SDK, Nokia Imaging SDK Beta 1
Compatibility
Platform(s):
Windows Phone 8
Article
Created: RatishPhilip (27 Aug 2013)
Last edited: hamishwillee (14 Oct 2013)

Contents

Introduction

In my previous article, I had described about my Windows Phone app, FilterSquare, which uses Nokia Imaging SDK to apply filters, provided by the SDK, to specific regions in an image. I spent the last two week optimizing my app further and adding more features. While doing this, I got the chance to delve deeper into the SDK and I encountered several challenges. In this article I am going to share my solutions of a few of these challenges.

WriteableBitmapEx

In my previous article, I had used the WriteableBitmapEx library along with the Nokia Imaging SDK to apply filters to specific regions of an image. These regions could either be rectangular or elliptical in shape. Here is the code to achieve that:

public static class FilterHelper
{
async public static Task ApplyFilterToRegion(WriteableBitmap sourceImage, Rect region, IFilter filter, bool isOval, bool invert = false)
{
WriteableBitmap regionBmp = GetRegion(sourceImage, region, isOval);
WriteableBitmap srcBmp = sourceImage;
 
if (!invert)
{
srcBmp = regionBmp;
}
 
using (EditingSession session = new EditingSession(srcBmp.AsBitmap()))
{
session.AddFilter(filter);
 
WriteableBitmap resultBmp = BitmapFactory.New((int)srcBmp.PixelWidth, (int)srcBmp.PixelHeight);
await session.RenderToWriteableBitmapAsync(resultBmp);
resultBmp.Pixels.CopyTo(srcBmp.Pixels, 0);
}
 
sourceImage.Blit(region, regionBmp, new Rect(0, 0, region.Width, region.Height));
}
 
private static WriteableBitmap GetRegion(WriteableBitmap sourceImage, Rect region, bool isOval)
{
WriteableBitmap croppedImage = null;
if (isOval)
{
croppedImage = sourceImage.CropEllipse(region);
}
else
{
croppedImage = sourceImage.Crop(region);
}
 
return croppedImage;
}
}
 
public static class WriteableBitmapExtensions
{
public static WriteableBitmap CropEllipse(this WriteableBitmap bmp, int x, int y, int width, int height)
{
WriteableBitmap cropResult = bmp.Crop(x, y, width, height);
WriteableBitmap cropMask = BitmapFactory.New(width, height);
cropMask.FillEllipse(0, 0, width, height, Colors.Red);
cropResult.Blit(new Rect(0, 0, width, height), cropMask, new Rect(0, 0, width, height), System.Windows.Media.Imaging.WriteableBitmapExtensions.BlendMode.Mask);
 
return cropResult;
}
 
public static WriteableBitmap CropEllipse(this WriteableBitmap bmp, Rect region)
{
return CropEllipse(bmp, (int)region.Left, (int)region.Top, (int)region.Width, (int)region.Height);
}
}

In this process, first the region where the filter needs to be applied is obtained by cropping the original image. The WriteableBitmapEx library provides the Crop() extension method which provides a rectangular cropped image. Since there was no method which would provide a elliptical cropped image I wrote my own extension method for this purpose - CropEllipse(). The argument invert indicates whether the filter should be applied inside the region of interest or outside. Once the region is obtained in the form of a WriteableBitmap, based on the invert flag, a new EditingSession is created either for that region or for the source image and the required filter is applied. Then that region is blitted on top of the source image by using the Blit method of the WriteableBitmapEx library.

The following diagram shows the various steps involved when invert is false

WriteableBitmapEx - ApplyFilterToRegion non-inverse

The following diagram shows the various steps involved when invert is true

WriteableBitmapEx - ApplyFilterToRegion inverse

This method of applying filters has the following disadvantages:

  • For each region a separate EditingSession needs to be created and they must be kept track of, in case we need to provide the Undo feature in our app.

Nokia Imaging SDK

While optimizing my app, I came across the ImageFusion filter in Nokia Imaging SDK which allows us to fuse two images. This filter also requires a mask (basically a back and white image) which governs the way the fusion between the two images occurs. The areas in black will be replaced with the source image and the areas filled in white will be replaced with the fusion image. After playing around with this filter for sometime, I came up with the following extension method.

public static class SessionExtensions
{
public enum RegionType
{
Rectangle = 0,
Ellipse
}
 
public async static Task AddFilterToRegionAsync(this EditingSession session, IFilter filter, Rect region, RegionType regionType, bool invert = false)
{
if ((session != null) && (filter != null))
{
// Cache the dimensions
Windows.Foundation.Size dimensions = session.Dimensions;
 
// Create the Fusion Image
WriteableBitmap fusionImage = new WriteableBitmap((int)dimensions.Width, (int)dimensions.Height);
session.AddFilter(filter);
// Initialize the MaskImage
WriteableBitmap maskImage = new WriteableBitmap((int)dimensions.Width, (int)dimensions.Height);
 
// Create the Mask Image
Task maskTask = Task.Run(new Action(() =>
{
// The region filled in Black will be replaced with the source image
maskImage.FillRectangle(0, 0, (int)dimensions.Width, (int)dimensions.Height, Colors.Black);
 
// The region filled in White will be replaced with the Fusion Image
// Based on the region type, fill a rectangular or an elliptical area
switch (regionType)
{
case RegionType.Rectangle:
// Fill a rectangle region
maskImage.FillRectangle((int)region.Left, (int)region.Top, (int)region.Right, (int)region.Bottom, Colors.White);
break;
case RegionType.Ellipse:
// Fill an elliptical region
maskImage.FillEllipse((int)region.Left, (int)region.Top, (int)region.Right, (int)region.Bottom, Colors.White);
break;
default:
break;
}
}));
 
Task renderTask = session.RenderToWriteableBitmapAsync(fusionImage);
//await session.RenderToWriteableBitmapAsync(fusionImage);
 
// Await for all the tasks to complete
await Task.WhenAll(maskTask, renderTask);
 
// Undo the applied filter
session.Undo();
 
// Apply the fusion filter
session.AddFilter(FilterFactory.CreateImageFusionFilter(fusionImage.AsBitmap(), maskImage.AsBitmap(), invert));
}
}
}

In this extension method too, the argument invert specifies whether the filter should be applied within the region of interest or outside it.This value is passed as the invertAlpha argument to the CreateImageFusionFilter of the FilterFactory.

First the filter (which is to be applied only to a region in the final image) is applied to the whole source image and rendered to a new bitmap - fusionImage.

A (new) mask bitmap the same size as the source image is then created for use as the ImageFusion filter. This is filled with the colour black and then the region where the filter needs to be applied is filled with white.

Finally, the Undo() method is invoked to clear the filter on the EditingSession and then the ImageFusion filter is added with the mask instead. When this filter is applied, the source image and filtered image are merged, with with the white area of the mask defining the area where the original filter appears(if invert is false, else vice versa).

The following image displays the steps involved when invert is false

Nokia Imaging SDK- ApplyFilterToRegion non-inverse

The following image displays the steps involved when invert is true

Nokia Imaging SDK - ApplyFilterToRegion inverse

The advantages of this method are:

  • Since there is only one EditingSession involved, providing an "undo" feature becomes easy as it is managed by the EditingSession itself.

One disadvantage of this method is that regardless of whether invert is false or invert is true, the first filter is applied to the entire source image which can be time consuming in comparison to applying the filter only to the region of interest in the WriteableBitmapEx extension method (when invert is false).


Applying Filter to a region

The attached source code runs the two methods for 100 iterations each and displays the average time taken for both the processes. I tested these two methods on my Nokia Lumia 720 on an image of resolution 1024x768 and I found out that different filters took different amount of time. In some cases the WriteableBitmapEx extension methods were faster while in the other cases the Nokia Imaging SDK extension methods were faster. Hoever, the time taken by the Nokia Imaging SDK extension methods were almost the same regardless of the value of the invert argument, but there was a vast difference in the time taken by the WriteableBitmapEx extension methods when invert was false or true.

Here are the average times I obtained for few filters (values are in milliseconds)

Time taken to apply various filters

Source Code

Download Source Code

Summary

Here I have displayed two extension methods of applying filters to specific regions in an image. However both have their pros and cons and it cannot be said that one is faster than the other. I do expect Nokia to provide their own version of Blit method which is faster and more optimized that the WriteableBitmapEx version.

Version Hint

Windows Phone: [[Category:Windows Phone]]
[[Category:Windows Phone 7.5]]
[[Category:Windows Phone 8]]

Nokia Asha: [[Category:Nokia Asha]]
[[Category:Nokia Asha Platform 1.0]]

Series 40: [[Category:Series 40]]
[[Category:Series 40 1st Edition]] [[Category:Series 40 2nd Edition]]
[[Category:Series 40 3rd Edition (initial release)]] [[Category:Series 40 3rd Edition FP1]] [[Category:Series 40 3rd Edition FP2]]
[[Category:Series 40 5th Edition (initial release)]] [[Category:Series 40 5th Edition FP1]]
[[Category:Series 40 6th Edition (initial release)]] [[Category:Series 40 6th Edition FP1]] [[Category:Series 40 Developer Platform 1.0]] [[Category:Series 40 Developer Platform 1.1]] [[Category:Series 40 Developer Platform 2.0]]

Symbian: [[Category:Symbian]]
[[Category:S60 1st Edition]] [[Category:S60 2nd Edition (initial release)]] [[Category:S60 2nd Edition FP1]] [[Category:S60 2nd Edition FP2]] [[Category:S60 2nd Edition FP3]]
[[Category:S60 3rd Edition (initial release)]] [[Category:S60 3rd Edition FP1]] [[Category:S60 3rd Edition FP2]]
[[Category:S60 5th Edition]]
[[Category:Symbian^3]] [[Category:Symbian Anna]] [[Category:Nokia Belle]]

This page was last modified on 14 October 2013, at 01:33.
115 page views in the last 30 days.

Was this page helpful?

Your feedback about this content is important. Let us know what you think.

 

Thank you!

We appreciate your feedback.

×