Please note that as of October 24, 2014, the Nokia Developer Wiki will no longer be accepting user contributions, including new entries, edits and comments, as we begin transitioning to our new home, in the Windows Phone Development Wiki. We plan to move over the majority of the existing entries. Thanks for all your past and future contributions.

Revision as of 10:12, 31 August 2013 by to_pe (Talk | contribs)

Partial filter application and blending with Nokia Imaging SDK

From Wiki
Jump to: navigation, search

Note.pngNote: This is an entry in the Nokia Imaging Wiki Competition 2013Q3

This article explains how to use Nokia Imaging SDK to create powerful image editing applications. In this particular application, a user can select part of the picture by 'painting' over it and apply filter only to painted region. Then the filtered region can be blended back onto the image to create interesting images.

SignpostIcon XAML 40.png
WP Metro Icon WP8.png
Article Metadata
Code ExampleTested withCompatibility
Windows Phone 8
Dependencies: MVVM Light, Windows Phone Toolkit, Coding4Fun
Created: to_pe (30 Aug 2013)
Last edited: to_pe (31 Aug 2013)



Nokia Imaging SDK brings a set of powerful filters which, either alone or combined, can create wonderful images. Applications that rely on filtering technology (e.g. Instagram) apply one or several filters to entire images in order to make them more appealing. By using a different approach, a user can create interesting images by applying filters to arbitrary regions and then applying filters to just that region. Afterwards, that part is blended back into image and images can now feature one or more differently filtered regions. In this article we will show the technical details behind building such application and several examples of interesting effects will be shown.

Since we will talk a lot about blending in this article, a definition is needed. The process of merging two images or, in case of advanced photo manipulation applications, merging two layers is called blending. Since both images or layers are made of pixels, the function that takes two pixels that occupy the same position and returns a resulting pixel is called a "blending function". There are several well known and defined blending modes, you can read up on the wikipedia article. There is also something called "alpha blending" and is used when dealing with transparent images and that is not covered in this wiki article.

Note: when testing the application in the emulator, keep in mind that there is a bug when choosing photos from the library. Until one manually opens the "photos" application, calling the PhotoChooserTask will give appearance that the library is empty. It is not, but you need to start the "photos" application once prior to choosing an image from the library.


Besides using Nokia Imaging SDK, which is available as a NuGet package, several other packages have been used.

Metro Studio 2 by Syncfusion was used to create icons.

Creating the application

For this particular application, the default Windows Phone 8 template was used. After adding the necessary NuGet packages and adjusting build configuration (specific to using Nokia Imaging SDK, for more information click here we are ready to start using the SDK. The idea behind this application is to leverage filters that come with Nokia Imaging SDK to create stunning and surreal images similar to other applications in the marketplace but with a twist - let user pick which parts of the image are filtered and with what filter.

Here is one example:

Unlike traditional filtering applications, the user can choose arbitrary regions for application filtering. By applying filers to specific regions, those regions can be highlighted. Here are some more examples:

Application architecture and functionality

This is the basic layout of the application. Application architecture

Add caption here

The application consists of just one main editing page (name MainPage.xaml) which contains the entire application's functionality. The logic is implemented in the backing view model (named MainViewModel). The main functionalities are:

  • Pick an image from the library or take a photo.
  • Choose one of the selection tools (the default is "brush selection tool".
  • Discard all changes.
  • Choose a filter to apply to the selected region.
  • Choose a blending mode.
  • Change the overall shade of gray for the replacement pixels (more on that further down).
  • Save resulting photo to the phone or share it via built-in sharing capabilities.

Picking an image is done through the standard PhotoChooserTask and the resulting stream is cached in the application's isolated storage. The next time user opens the application, the last image from previous session will be used as a starting image. This choosing part can be extended to allow picking files from URL, SkyDrive or any other cloud storage provider.

Once the image is loaded, user can start defining the region using the built-in selection tools. There are three that are currently implemented:

  • Brush selection tool - user can touch screen to select anything.
  • Magic wand selection tool - by touching particular area, pixels that are similar to the chosen pixel are selected
  • Rectangular selection tool - user draws a rectangle over the image which is then used as a brush mask

Here are the differences between the selection tools:

By default, the brush selection tool is used. User can select another tool by pulling down a "selection tools" overlay. This can be done by swiping down on the top edge of the phone (the edge above the image). This brings down the "selection tools" overlay which can be seen on the image to the right. Once a tool is selected, the user can start applying it to the selected image (if any). Application is designed with extensibility in mind and allows for easy creation of additional selection tools. Zooming in or inverting selection can also be easily implemented using the same interfaces. All tools inherit from the ITools interface, while

Once user picks an image, a region is defined by applying selection tools. Even though in this early iteration undo feature is missing, it is relatively easy to add it by separating all selections into "layers". Once the user is satisfied with the selection, the currently selected filter is applied on the defined region and blended back onto the image by pressing the "apply" icon (the one with the check). As mentioned before, blending is a process of merging two images to produce a third one. In this case, one image is the original image with the part of the image "carved out " and replaced with white pixels. The second image contains those pixels that have been carved out with the rest of the pixels turned to pure white. The replacement pixels can be adjusted using the two sliders on the main page which will be described in more detail below.

Even after the filter and blending are applied, user can change both the filter used for the filtered region and the blending parameters. This allows for "fine tuning" and experimentation. Once the user is satisfied with the results, the image can be either shared using the built-in sharing capabilities or saved to the media library.

Selection tools

As mentioned before, there are three different selection tools each implemented with specific algorithms. They will be described here in further detail. Each tool implements the ITool interface:

public interface ITool
double ImageWidth { get; set; }
double ImageHeight { get; set; }
WriteableBitmap TargetImage { get; set; }
void Started(ManipulationStartedEventArgs e);
void Delta(ManipulationDeltaEventArgs e);
void Ended(ManipulationCompletedEventArgs e);

First two properties (ImageWidth and ImageHeight) are used for transformation from screen space to image space. In most cases the underlying image will be significantly larger than the UI control used to represent it. The last property is necessary since a tool might want to inspect image for its own purposes. The three methods in the interface are used for analyzing user manipulation.

Using this interface you might implement crop tool, color picker tool and in general read-only tools. To implement a selection tool that generates a mask, you need several additional parameters. To accommodate that, another interface is added on top of the ITool interface:

public interface ISelectionTool : ITool
byte[] MaskBuffer { get; set; }
int[] Source { get; set; }
int[] Target { get; set; }

Tools that inherit from this interface have input pixels array which should be left unmodified and output pixels array which they use for building the result. MaskBuffer is used to remember which pixels have been manipulated. Undo stack can easily be created by chaining tools and putting them on the stack but due to the lack of time has not been implemented at this point. We can now see how individual tools have been implemented.

Brush selection tool

Add caption here
The brush selection tool is the default tool when starting the application. The brush is this case is a circular shape filling a 50x50 area. This can be further extended in three major directions:
  1. The size could be customizable.
  2. Wider selection of brush shapes: square, triangle, star, custom shape, shape combinations, etc.
  3. "Solidness" of the brush - right now it is a filled circle but it can be sparsely filled, hollow or something different (think graffiti brush from MS Paint).

When user touches the image, the center of the brush is positioned to match the center of the touch surface. User can drag around and "paint" the area. Unfortunately, due to the missing zoom capabilities and the inability to customize brushes size, it may be hard to precisely paint the desired region. This brush gives the natural feel of picking the region.

The brush is defined as a mask: a byte[50*50 array filled with values from 0 to 255. The value of 0 means that this pixel is to be left alone, other values signify the "replacement strength". Brush is generated via the following algorithm:

private const int BrushRadius = 50;
private const int Tolerance = 6;
private void BuildBrush()
_brushMask = new byte[BrushRadius * 2 + 1, BrushRadius * 2 + 1];
const double threshold = (BrushRadius - Tolerance) * (BrushRadius - Tolerance);
const double fallout = BrushRadius * BrushRadius;
const double delta = fallout - threshold;
for (var i = 0; i < BrushRadius * 2 + 1; ++i)
for (var j = 0; j < BrushRadius * 2 + 1; ++j)
// distance squared
var d2 = (i - BrushRadius) * (i - BrushRadius) + (j - BrushRadius) * (j - BrushRadius);
byte a;
if (d2 <= threshold)
a = 0xFF;
else if (d2 > fallout)
a = 0;
var t = fallout - d2;
a = (byte)(255.0 * t / delta);
_brushMask[i, j] = a;

Note that the algorithm generates a soft fade-out which could be used for alpha blending, but is unused in the current implementation. The brush is applied on the separate thread. Since the code is rather large and is provided as an attachment to this article, we will just take a look at the underlying algorithm. When user interacts using the tool, all points and movements are added to the queue which is processed in another thread. This makes the application responsive when user swipes across the screen since the brush applying algorithm is faster when compared with generating the final image and refreshing the UI.

public void Started(ManipulationStartedEventArgs e)
lock (_syncLock)
_touchQueue.Enqueue(Tuple.Create(e.ManipulationOrigin, new Point()));
_lastPoint = e.ManipulationOrigin;
public void Delta(ManipulationDeltaEventArgs e)
lock (_syncLock)
_touchQueue.Enqueue(Tuple.Create(e.ManipulationOrigin, new Point(e.ManipulationOrigin.X - _lastPoint.X, e.ManipulationOrigin.Y - _lastPoint.Y)));
_lastPoint = e.ManipulationOrigin;
public void Ended(ManipulationCompletedEventArgs e) { }

In the separate thread, the _touchQueue is inspected and values are pulled out. Once there are no values on the queue (which means that the user is no longer moving the finger across the screen), brush is applied on the user selected points. Brush is clipped to the image and every pixel of the brush is processed (all 2500 of them in this case). If the brush has a nonzero value at the given pixel, a pixel is moved from the Source to the Target and TargetImage is updated to reflect this process. Here is the sketch of the algorithm:

for each point to apply
clip brush to the image
for each pixel that is inside the image
if brushmask != 0
copy pixel from source to target
make display image pixel white

After the algorithm completes, the Target will contain only those pixels where the brush has nonzero value and the TargetImage will have white pixels instead of the old ones. The remaining pixels (those where the brush has zero value) are left intact in the TargetImage and are equal to 0 (black) on Target.

There are several other optimizations that could be used to speed up the algorithm, but then again, this is not a professional application for image manipulation and the performance issues should be negligible on high-end phones.

Magic wand selection tool

Add caption here
This selection tool is designed to select an area of "similar looking pixels". For example, you might want to select a part of the image which has same color, but different lighting e.g. door, table, field, sky, water, etc. Determining how similar two pixels are is done by calculating the distance between their UV components. If the distance is below some threshold value, they are considered "similar", otherwise they are not considered similar. UV components are part of the YUV color space and you have to convert RGB values using the transformation formulas. Distance is the standard two-dimensional Euclidean distance.

The algorithm is queue based flood fill algorithm with fixed threshold value. Ideally you would want to be able to adjust the threshold value, but it is not possible in the current implementation. Here is the algorithm:

private void Apply(Point p)
// convert from screen space to image space
// prefix o = original
var ox = (int)(p.X * TargetImage.PixelWidth / ImageWidth);
var oy = (int)(p.Y * TargetImage.PixelHeight / ImageHeight);
if (ox < 0 || ox >= TargetImage.PixelWidth ||
oy < 0 || oy >= TargetImage.PixelHeight)
var originalPixel = TargetImage.Pixels[ox + oy * TargetImage.PixelWidth];
var ouv = MathEx.GetUV(originalPixel);
// apply flood fill algorithm
var pointsToCheck = new Stack<Tuple<int, int>>();
pointsToCheck.Push(Tuple.Create(ox, oy));
while (pointsToCheck.Any())
var pointToCheck = pointsToCheck.Pop();
var x = pointToCheck.Item1;
var y = pointToCheck.Item2;
var index = x + y * TargetImage.PixelWidth;
// already processed
if (MaskBuffer[index] == 255)
// test similarity between colors
var pixel = TargetImage.Pixels[x + y * TargetImage.PixelWidth];
var uv = MathEx.GetUV(pixel);
if (MathEx.Distance2(ouv, uv) > 150)
// apply for this pixel
MaskBuffer[index] = 255;
Target[index] = Source[index];
TargetImage.Pixels[index] = (Constants.White & 0xFFFFFF) | (0xFF << 24);
// push neighbors to the stack (if reachable)
if (x > 0)
pointsToCheck.Push(Tuple.Create(x - 1, y));
if (x < TargetImage.PixelWidth - 1)
pointsToCheck.Push(Tuple.Create(x + 1, y));
if (y > 0)
pointsToCheck.Push(Tuple.Create(x, y - 1));
if (y < TargetImage.PixelHeight - 1)
pointsToCheck.Push(Tuple.Create(x, y + 1));

This tool is useful when picking out an area of similar looking colors which would be hard to paint using the regular brush selection tool.

Rectangle selection tool

Add caption here

This is probably the classic selection tool which is familiar to anyone who ever used image processing applications. The idea is simple: user touches the screen at one point and drags the finger around to draw the desired rectangle. Once the user is satisfied, finger is lifted and the area under the rectangle is used for selection. To enable the region preview during the dragging process, another interface is used:

public interface IVisualTool
FrameworkElement Element { get; set; }
Point Position { get; set; }
event EventHandler ElementChanged;
event EventHandler PositionChanged;

The idea is that only after the user finishes manipulation the actual selection is done. During the dragging period a UI element is used to convey the information about the resulting selection process. Event handlers are used to signal the parent UI component that the underlying visual part has changed. Final algorithm is straightforward (_startPoint is defined when the manipulation started):

private void Apply(Point finalPoint)
// burn onto image
var startx = (int)(Math.Min(_startPoint.X, finalPoint.X) * TargetImage.PixelWidth / ImageWidth);
var starty = (int)(Math.Min(_startPoint.Y, finalPoint.Y) * TargetImage.PixelHeight / ImageHeight);
var endx = startx + (int)(Math.Abs(_startPoint.X - finalPoint.X) * TargetImage.PixelWidth / ImageWidth);
var endy = starty + (int)(Math.Abs(_startPoint.Y - finalPoint.Y) * TargetImage.PixelHeight / ImageHeight);
for (var x = startx; x < endx; ++x)
for (var y = starty; y < endy; ++y)
var index = x + y * TargetImage.PixelWidth;
Target[index] = Source[index];
TargetImage.Pixels[index] = Constants.White;

Further enhancements of this tool might include positioning, scaling and rotating the rectangle before applying it.

Source Code

Download Source Code


Nokia Imaging SDK is a powerful tool that can be harnessed in different ways. I hope that this interesting approach to building complex compositions yields interesting applications that go beyond the simple "filter and share" applications. Since the application described here applies filters to just part certain part of the image (or consider applying filter everywhere but on some special part), resulting images can have surreal or hyper-real feel.

Version Hint

Windows Phone: [[Category:Windows Phone]]
[[Category:Windows Phone 7.5]]
[[Category:Windows Phone 8]]

Nokia Asha: [[Category:Nokia Asha]]
[[Category:Nokia Asha Platform 1.0]]

Series 40: [[Category:Series 40]]
[[Category:Series 40 1st Edition]] [[Category:Series 40 2nd Edition]]
[[Category:Series 40 3rd Edition (initial release)]] [[Category:Series 40 3rd Edition FP1]] [[Category:Series 40 3rd Edition FP2]]
[[Category:Series 40 5th Edition (initial release)]] [[Category:Series 40 5th Edition FP1]]
[[Category:Series 40 6th Edition (initial release)]] [[Category:Series 40 6th Edition FP1]] [[Category:Series 40 Developer Platform 1.0]] [[Category:Series 40 Developer Platform 1.1]] [[Category:Series 40 Developer Platform 2.0]]

Symbian: [[Category:Symbian]]
[[Category:S60 1st Edition]] [[Category:S60 2nd Edition (initial release)]] [[Category:S60 2nd Edition FP1]] [[Category:S60 2nd Edition FP2]] [[Category:S60 2nd Edition FP3]]
[[Category:S60 3rd Edition (initial release)]] [[Category:S60 3rd Edition FP1]] [[Category:S60 3rd Edition FP2]]
[[Category:S60 5th Edition]]
[[Category:Symbian^3]] [[Category:Symbian Anna]] [[Category:Nokia Belle]]

149 page views in the last 30 days.