×
Namespaces

Variants
Actions

Custom Effects in Nokia Imaging SDK: Pixelate and Color Quantizer

From Nokia Developer Wiki
Jump to: navigation, search
Featured Article
15 Jun
2014

This article explains how to create Custom Filter Effects in Nokia Imaging SDK 1.0 and provides the example of two custom Filter effects: PixelateEffect and ColorQuantizerEffect.

Note.pngNote: This is an entry in the Nokia Imaging and Big UI Wiki Competition 2013Q4.


SignpostIcon XAML 40.png
WP Metro Icon WP8.png
Article Metadata
Code ExampleTested with
SDK: Windows Phone SDK 8.0, Nokia Imaging SDK 1.0
Compatibility
Platform(s):
Windows Phone 8
Article
Created: RatishPhilip (16 Dec 2012)
Last edited: kiran10182 (15 Jun 2014)

Contents

Introduction

Nokia Image SDK 1.0 has a lot of improvements over the beta version of the SDK and apart from the code optimizations, the features most noticeable are the ability to change Effect parameters dynamically and creation of Custom Effects. Custom Effects provide a nice way to extend the SDK by providing our own effects which can be used by anyone in the developer community. Since there is only basic documentation available for Custom Effects, I thought of writing this article which would be helpful for beginners in grasping the details about Custom Effect.

Prerequisites

I would recommend going through the following links first to get a good grasp of the changes incorporated in the latest version of the SDK:

The Basic Imaging Pipeline

The basic imaging pipeline employed in the SDK is depicted in the diagram below.

Basic Imaging Pipeline

The pipeline mainly uses three basic elements : An Image source, an Effect and a Renderer. Using these elements it is possible to create complex filter operations in the SDK.

  1. Image Source - The image source implements the IImageProvider interface. This element represents the image upon which the processing needs to be done. This image can be either loaded from storage (like Camera Roll or Albums) or generated dynamically (like capturing an image through the camera). This element usually forms the beginning of the pipeline. The SDK provides several concrete implementations like BitmapImageSource, StreamImageSource, BufferImageSource which can be used as an image source.
  1. Effect - The Effect implements the IImageConsumer and the IImageProvider interfaces. Implementing the IImageConsumer interface allows it to consume or take as an input an object implementing the IImageProvider interface (like the image source). Implementing the IImageProvider interface allows the output of the processing done by the Effect to be used as an input to another effect or the renderer. The SDK provides the FilterEffect class as the concrete implementation of the Effect element. The Filters property of this class is able to accommodate a sequence of 50+ filters provided by the SDK.
  1. Renderer - The Renderer forms the end of the pipeline and implements the IImageConsumer which allows it to consume the output of an Effect or the image source directly and render it to an image of specific format. The SDK provides several renderers which can be used to render the output image. They are WriteableBitmapRender and JpegRenderer .

Chaining of Effects

Since the Effect implements both IImageProvider and IImageConsumer interfaces, it facilitates in chaining multiple effects together to create an overall complex effect chain or graph.

Effect Chaining

Imaging Pipeline Example

Lets take an example of the Effect Chaining. We shall create a StreamImageSource from an image in the albums (using a PhotoChooserTask) and we will apply a FlipFilter (with mode as Horizontal) and to that we will then apply the Sepia filter. Finally we shall use a WriteableBitmapRenderer to render the output of the processing on a WriteableBitmap.

The chaining of effects is shown below.

Chaining of Effects Example

Here is the sample code

var filters = new IFilter[]
{
new FlipFilter() { FlipMode = FlipMode.Horizontal },
new SepiaFilter()
};
 
using (var source = new StreamImageSource(stream))
using (var filterEffect = new FilterEffect(source) { Filters = filters })
using (var renderer = new WriteableBitmapRenderer(filterEffect, writeableBitmap))
{
await renderer.RenderAsync();
}

Pipeline Example

Custom Effect

In managed code, such as C#, in order to create a custom Effect you must define a class which derives from Nokia.Graphics.Imaging.CustomEffectBase. The CustomEffectBase class is an abstract class which provides the basic functionalities which we can use in our custom effect.

public abstract class CustomEffectBase : IImageProvider, IImageConsumer, IDisposable, ICustomEffect
{
protected CustomEffectBase(IImageProvider source, bool isInplace = false);
 
public IImageProvider Source { get; set; }
 
public void Dispose();
 
protected virtual void Dispose(bool disposing);
 
public static uint FromColor(Color color);
 
public IAsyncOperation<Bitmap> GetBitmapAsync(Bitmap bitmap, OutputOption outputOption);
 
public IAsyncOperation<ImageProviderInfo> GetInfoAsync();
 
public bool Lock(RenderRequest renderRequest);
 
protected virtual IAsyncAction OnLoadAsync();
 
protected abstract void OnProcess(PixelRegion sourcePixelRegion, PixelRegion targetPixelRegion);
 
public IAsyncAction PreloadAsync();
 
public static Color ToColor(uint uintColor);
}

From the above code, the most noticeable functions are:

  1. OnProcess: This method contains the core logic for the custom effect and needs to be implemented. It provides two PixelRegion objects, (one sourcePixelRegion and the other targetPixelRegion). They provide access to the pixels of the source image and provide various properties and methods which facilitate in iterating through the pixels. The color values of the pixels are represented as a 32-bit unsigned integer which encapsulates the Red, Green, Blue and Alpha values.
  2. ToColor: This method converts the uint value to Windows.UI.Color object.
  3. FromColor : This method converts the Windows.UI.Color object to uint value.
  4. The Constructor: Along with the imageSource the constructor takes in a boolean parameter isInPlace. According to the code documentation, if isInPlace is true, the sourcePixels and targetPixels parameters to OnProcess will refer to the same array. This can be more efficient, but may restrict the effect (writing a pixel means the original source pixel is discarded). If false, different buffers are used. The default value is false.
  5. GetBitmapAsync: Creates a bitmap with the contents of the source Image provided.

OnProcess method

The OnProcess method in CustomEffectBase is the most important method of all wherein we will be providing the main processing logic. Lets assume we have provided an Image of size 1024x768 as the input to our customEffect. The PixelRegion object provides the following properties/methods:

  1. Bounds : A Windows.Foundation.Rect object i.e. Rect(0, 0, width, height) where width and height depict the width and height of image source. (In the case of above example Rect(0, 0, 1024, 768))
  2. ImagePixels : A uint array (of size width x height) representing the unsigned int values of the pixels of the imagesource.
  3. ImageSize : A Windows.Foundation.Size object representing the size of the image source.
  4. Pitch : The width of the image source.
  5. StartIndex : The index within the ImagePixels array representing the first pixel to process.
  6. ForEachRow() : This method runs a user-defined Action for each of the pixels in the row of pixels within the Bounds.

Note.pngNote: If you want to provide additional data to the OnProcess method, implement those data as properties in your Custom Effect and pass them via the constructor.

Sample Effects

In the attached sample code, I have provided two custom Effects - PixelateEffect and ColorQuantizerEffect.

Sample Application

Pixelate Effect

The PixelateEffect pixelates the input image in such a way that the output image seems to be constructed with larger blocks of a particular color. The entire set of blocks seems similar to the input image. The size of the blocks is determined by the BlockSize property.

PixelateEffect

public class PixelateEffect : CustomEffectBase
{
public const int MIN_BLOCK_SIZE = 1;
public const int MAX_BLOCK_SIZE = 128;
 
private int blockSize;
 
public int BlockSize
{
get { return blockSize; }
set
{
// Clamp the value between the MIN and MAX Block sizes
blockSize = Math.Min(Math.Max(MIN_BLOCK_SIZE, value), MAX_BLOCK_SIZE);
}
}
 
public PixelateEffect(IImageProvider source, int blockSize = MIN_BLOCK_SIZE)
: base(source)
{
BlockSize = blockSize;
}
 
protected override void OnProcess(PixelRegion sourcePixelRegion, PixelRegion targetPixelRegion)
{
for (int srcY = 0; srcY < sourcePixelRegion.ImageSize.Height; srcY += BlockSize)
for (int srcX = 0; srcX < sourcePixelRegion.ImageSize.Width; srcX += BlockSize)
{
int offsetX = BlockSize / 2;
int offsetY = BlockSize / 2;
// Ensure that the center pixel remains within the image bounds
while (srcX + offsetX >= sourcePixelRegion.ImageSize.Width)
offsetX--;
while (srcY + offsetY >= sourcePixelRegion.ImageSize.Height)
offsetY--;
 
// Get the index of the center most pixel in the Block
int srcIndex = (srcY + offsetY) * ((int)sourcePixelRegion.ImageSize.Width) + (srcX + offsetX);
uint srcColorValue = sourcePixelRegion.ImagePixels[srcIndex];
Color srcColor = ToColor(srcColorValue);
 
// Apply the same color to the pixels in the entire block
for (int targX = srcX; (targX < (srcX + BlockSize)) && (targX < targetPixelRegion.ImageSize.Width); targX++)
for (int targY = srcY; (targY < (srcY + BlockSize)) && (targY < targetPixelRegion.ImageSize.Height); targY++)
{
int targetIndex = targY * ((int)targetPixelRegion.ImageSize.Width) + targX;
targetPixelRegion.ImagePixels[targetIndex] = FromColor(srcColor);
}
}
}
}

ColorQuantizer Effect

The ColorQuantizerEffect reduces the number of distinct colors used in an image. The number of distinct colors to be used can be specified by selecting the color mode. Based on this, the Euclidean distance of each pixel in the source image is calculated against each of the colors in the color palette and the the color with the least distance is used to color the pixels in the entire block. The size of the blocks is determined by the BlockSize property.

class ColorQuantizerEffect: CustomEffectBase
{
public enum QuantizerMode
{
None = 0,
Mode16 = 1,
Mode64 = 2
}
 
public const int MIN_BLOCK_SIZE = 1;
public const int MAX_BLOCK_SIZE = 128;
List<Color> _colorPalette = null;
 
private int blockSize;
 
public int BlockSize
{
get { return blockSize; }
set
{
// Clamp the value between the MIN and MAX Block sizes
blockSize = Math.Min(Math.Max(MIN_BLOCK_SIZE, value), MAX_BLOCK_SIZE);
}
}
 
private QuantizerMode _colorMode;
 
public QuantizerMode ColorMode
{
get { return _colorMode; }
set { _colorMode = value; }
}
 
 
public ColorQuantizerEffect(IImageProvider source, QuantizerMode mode, int blockSize = MIN_BLOCK_SIZE)
: base(source)
{
BlockSize = blockSize;
_colorMode = mode;
CreateColorPalette();
}
 
private void CreateColorPalette()
{
_colorPalette = new List<Color>();
switch (_colorMode)
{
case QuantizerMode.None:
case QuantizerMode.Mode16:
_colorPalette.Add(Color.FromArgb((byte)255, (byte)0, (byte)0, (byte)0));
_colorPalette.Add(Color.FromArgb((byte)255, (byte)128, (byte)0, (byte)0));
_colorPalette.Add(Color.FromArgb((byte)255, (byte)255, (byte)0, (byte)0));
_colorPalette.Add(Color.FromArgb((byte)255, (byte)255, (byte)0, (byte)255));
_colorPalette.Add(Color.FromArgb((byte)255, (byte)0, (byte)128, (byte)128));
_colorPalette.Add(Color.FromArgb((byte)255, (byte)0, (byte)128, (byte)0));
_colorPalette.Add(Color.FromArgb((byte)255, (byte)0, (byte)255, (byte)0));
_colorPalette.Add(Color.FromArgb((byte)255, (byte)0, (byte)255, (byte)255));
_colorPalette.Add(Color.FromArgb((byte)255, (byte)0, (byte)0, (byte)128));
_colorPalette.Add(Color.FromArgb((byte)255, (byte)128, (byte)0, (byte)128));
_colorPalette.Add(Color.FromArgb((byte)255, (byte)0, (byte)0, (byte)255));
_colorPalette.Add(Color.FromArgb((byte)255, (byte)192, (byte)192, (byte)192));
_colorPalette.Add(Color.FromArgb((byte)255, (byte)128, (byte)128, (byte)128));
_colorPalette.Add(Color.FromArgb((byte)255, (byte)128, (byte)128, (byte)0));
_colorPalette.Add(Color.FromArgb((byte)255, (byte)255, (byte)255, (byte)0));
_colorPalette.Add(Color.FromArgb((byte)255, (byte)255, (byte)255, (byte)255));
break;
case QuantizerMode.Mode64:
List<byte> colorbits = new List<byte> { 0, 85, 170, 255 };
for (int i = 0; i < 4; i++)
for (int j = 0; j < 4; j++)
for (int k = 0; k < 4; k++)
{
_colorPalette.Add(Color.FromArgb((byte)255, (byte)colorbits[i], (byte)colorbits[j], (byte)colorbits[k]));
}
break;
default:
break;
}
}
 
protected Color QuantizeColor(Color srcColor)
{
double minDistance = Double.MaxValue;
Color result = srcColor;
 
foreach (Color c in _colorPalette)
{
double distance = CalculateEuclideanDistance(c, srcColor);
 
if (distance < minDistance)
{
minDistance = distance;
result = c;
}
}
 
return result;
}
 
private double CalculateEuclideanDistance(Color a, Color b)
{
return Math.Sqrt(Math.Pow(a.R - b.R, 2) + Math.Pow(a.G - b.G, 2) + Math.Pow(a.B - b.B, 2));
}
 
protected override void OnProcess(PixelRegion sourcePixelRegion, PixelRegion targetPixelRegion)
{
for (int srcY = 0; srcY < sourcePixelRegion.ImageSize.Height; srcY += BlockSize)
for (int srcX = 0; srcX < sourcePixelRegion.ImageSize.Width; srcX += BlockSize)
{
int offsetX = BlockSize / 2;
int offsetY = BlockSize / 2;
// Ensure that the center pixel remains within the image bounds
while (srcX + offsetX >= sourcePixelRegion.ImageSize.Width)
offsetX--;
while (srcY + offsetY >= sourcePixelRegion.ImageSize.Height)
offsetY--;
 
// Get the index of the center most pixel in the Block
int srcIndex = (srcY + offsetY) * ((int)sourcePixelRegion.ImageSize.Width) + (srcX + offsetX);
uint srcColorValue = sourcePixelRegion.ImagePixels[srcIndex];
Color srcColor = ToColor(srcColorValue);
Color destColor = QuantizeColor(srcColor);
 
// Apply the same color to the pixels in the entire block
for (int targX = srcX; (targX < (srcX + BlockSize)) && (targX < targetPixelRegion.ImageSize.Width); targX++)
for (int targY = srcY; (targY < (srcY + BlockSize)) && (targY < targetPixelRegion.ImageSize.Height); targY++)
{
int targetIndex = targY * ((int)targetPixelRegion.ImageSize.Width) + targX;
targetPixelRegion.ImagePixels[targetIndex] = FromColor(destColor);
}
}
}
}

Note.pngNote: These two Effects are provided for sample purpose only. They are not fully optimized.

Source Code

Source Code

Summary

Here I have showed the basics of creating custom effects for the Nokia Imaging SDK. By providing robust, optimized processing code in the OnProcess method on the CustomEffectBase, we can create truly awe-inspiring effects for our images.

References

Version Hint

Windows Phone: [[Category:Windows Phone]]
[[Category:Windows Phone 7.5]]
[[Category:Windows Phone 8]]

Nokia Asha: [[Category:Nokia Asha]]
[[Category:Nokia Asha Platform 1.0]]

Series 40: [[Category:Series 40]]
[[Category:Series 40 1st Edition]] [[Category:Series 40 2nd Edition]]
[[Category:Series 40 3rd Edition (initial release)]] [[Category:Series 40 3rd Edition FP1]] [[Category:Series 40 3rd Edition FP2]]
[[Category:Series 40 5th Edition (initial release)]] [[Category:Series 40 5th Edition FP1]]
[[Category:Series 40 6th Edition (initial release)]] [[Category:Series 40 6th Edition FP1]] [[Category:Series 40 Developer Platform 1.0]] [[Category:Series 40 Developer Platform 1.1]] [[Category:Series 40 Developer Platform 2.0]]

Symbian: [[Category:Symbian]]
[[Category:S60 1st Edition]] [[Category:S60 2nd Edition (initial release)]] [[Category:S60 2nd Edition FP1]] [[Category:S60 2nd Edition FP2]] [[Category:S60 2nd Edition FP3]]
[[Category:S60 3rd Edition (initial release)]] [[Category:S60 3rd Edition FP1]] [[Category:S60 3rd Edition FP2]]
[[Category:S60 5th Edition]]
[[Category:Symbian^3]] [[Category:Symbian Anna]] [[Category:Nokia Belle]]

This page was last modified on 15 June 2014, at 22:46.
431 page views in the last 30 days.
×