×
Namespaces

Variants
Actions
Revision as of 05:59, 10 December 2013 by hamishwillee (Talk | contribs)

Filter and Custom Filter Management Framework for the Nokia Imaging SDK

From Nokia Developer Wiki
Jump to: navigation, search

This article explains Imaging Framework that implement for to use custom filters and built-in filters.

Note.pngNote: This is an entry in the Nokia Imaging and Big UI Wiki Competition 2013Q4.

SignpostIcon XAML 40.png
WP Metro Icon WP8.png
Article Metadata
Code Example
Installation file: File:WP Imaging.zip (ARM and x86)
Tested with
SDK: Windows Phone 8.0 SDK, Nokia Imaging SDK v1.0
Devices(s): Windows Phone 8 Emulator, Nokia Lumia 920
Compatibility
Platform(s):
Windows Phone 8
Dependencies: Nokia Imaging SDK
Article
Created: Engin Kırmacı (31 Aug 2013)
Last edited: hamishwillee (10 Dec 2013)

Contents

Introduction

The Nokia Imaging SDK provides more than 50 filters and effects for developers to use and combine them to create new filters. Moreover with SDK 1.0, It allows us to implement custom filters and image providers.

I implemented a framework which helps easier use of custom filters and built-in filters with extended capabilities (Preprocess and Postprocess filter for custom filter, historize filters etc...) Also I try to demonstrate how to implement different filters which can be used for image processing.

Please read all documents about Nokia Imaging SDK to understand this article.

Useful information about custom filters in Nokia imaging SDK

  • You can't use custom filters as built-in filters. You use custom filters as you use FilterEffect. It's ImageConsumer and ImageProvider. So you can pipeline with filter effect. More details about pipe lining explained in Core concepts
  • ForEachRow method in PixelRegion class:
    Use this method to iterate over the pixels. If you want to iterate every pixels, don't use for loop, it's the most efficient way. Unless you want to iterate some pixels, not all of them or multiple iteration for each pixel, use your own for loop.
  • IsInPlace property of CustomEffectBase constructor:
    • Default value is false. Which means sourcePixel, targetPixel is different array and you have to get pixel from sourcePixel, process it and set to targetPixel. Also note that If you don't set targetPixel, you'll see black image.
    • If you set IsInPlace() to true, sourcePixel and targetPixel is same array. Documentation says that it's more efficient. But you have to be careful if you want to process pixel and don't want to change source pixel.


  • ToColor and FromColor methods:
    converts between a Windows.UI.Color and the color format expected in the pixel arrays.


Imaging Framework

Interfaces

  • IHistoryable

It used for factory classes, when factory able to have more than one filters. Example is NokiaFilterFactory which stores every built-in filters. This interface provides following methods;

bool CanUndo();
void Undo();
void UndoAll();
  • IPostProcess

Use to extend custom filters. It provides custom filter to post process factory. So after custom filter processed, post process factory's filters processed. For example, QuadTransform filter implementation.

public class QuadTransformation : CustomEffectBase, IPostProcess
{
.
.
protected override void OnProcess(PixelRegion sourcePixelRegion, PixelRegion targetPixelRegion)
{
.
.
}
 
public FactoryBase PostProcessFactory()
{
var factory = new NokiaFilterFactory(null);
factory.Filters.Add(new ReframingFilter(new Rect(0, 0, Size.Width, Size.Height), 0));
 
return factory;
}
}
  • IPreProcess

Use to extend custom filters. It provides custom filter to pre process factory. So before custom filter processed, pre process factory's filters processed.

Classes

  • ImagingManager

This class is the main core, everything happens here.

  • FactoryBase

Factories derives from this class. It has Input and Output as ImageProvider so that it builds pipeline with another factory. There is already two factories; CustomFilterFactory for customfilter and NokiaFilterFactory for built-in filters. In future, there'll be more for same type of custom filters like convulation filters used for smoothing, blurring, etc.

  • CustomFilterFactory

This factory stores only one custom filter. If custom filter has IPreProcess or/and IPostProcess interface, with ApplyFilter method, it builds pipeline inside it.

  • FilterHistory

This class used in ImagingManager. It historize built-in filters and custom filters added. It provides Undo and UndoAll method for filters.

How to implement custom filters?

I'm categorizing my sample custom filters in 3 category; calculation purpose, simple ones and complex ones. I'll try to explained over sample filters.

  • Implementation of custom filter for Calculation Purpose

HistogramFilter is the only sample for this type of filter. This filters calculates Red, Green, Blue and Luminance of image. These parameters can be used as filter parameters then.

Trick is here, IsInPlace property in base constructor is set to true, so that no need to set targetPixelRegion.ImagePixels.

public class HistogramFilter : CustomEffectBase
{
public int[] Red { get; set; }
 
public int[] Green { get; set; }
 
public int[] Blue { get; set; }
 
public int[] Luminance { get; set; }
 
public HistogramFilter(IImageProvider source, out int[] red, out int[] green, out int[] blue, out int[] luminance)
: base(source, true)
{
Red = new int[byte.MaxValue + 1];
Green = new int[byte.MaxValue + 1];
Blue = new int[byte.MaxValue + 1];
Luminance = new int[byte.MaxValue + 1];
 
red = Red;
green = Green;
blue = Blue;
luminance = Luminance;
}
 
protected override void OnProcess(PixelRegion sourcePixelRegion, PixelRegion targetPixelRegion)
{
sourcePixelRegion.ForEachRow((index, width, pos) =>
{
for (int x = 0; x < width; ++x, ++index)
{
Color c = ToColor(sourcePixelRegion.ImagePixels[index]);
 
Red[c.R]++;
Green[c.G]++;
Blue[c.B]++;
 
double luminance = 0.299 * c.R + 0.587 * c.G + 0.114 * c.B;
if (luminance < byte.MinValue)
luminance = byte.MinValue;
else if (luminance > byte.MaxValue)
luminance = byte.MaxValue;
 
Luminance[(byte)luminance]++;
}
});
}
  • Simple custom filter

ThresholdFilter is one of the sample for this type of filter. It's the same implementation as described here

public class ThresholdFilter : CustomEffectBase
{
private byte _thresholdR = 0;
private byte _thresholdG = 0;
private byte _thresholdB = 0;
 
public ThresholdFilter(IImageProvider source, byte R, byte G, byte B)
: base(source)
{
_thresholdR = R;
_thresholdG = G;
_thresholdB = B;
}
 
protected override void OnProcess(PixelRegion sourcePixelRegion, PixelRegion targetPixelRegion)
{
sourcePixelRegion.ForEachRow((index, width, pos) =>
{
for (int x = 0; x < width; ++x, ++index)
{
Color c = ToColor(sourcePixelRegion.ImagePixels[index]);
 
if (c.R < _thresholdR)
c.R = 0;
 
if (c.G < _thresholdG)
c.G = 0;
 
if (c.B < _thresholdB)
c.B = 0;
targetPixelRegion.ImagePixels[index] = FromColor(c);
}
});
}
}
  • Complex custom filter

PixelInterpolation is one of the sample for this type of filter. This filter pixelates image without changing the image size. It divides to image to rectangles like table and get top left pixel of each cell. Then changes every pixels in cell to corner.
For that purpose, iteration every pixels with ForEachRow is not efficient. First of all, use following to get width and height of an image.

int width = (int)sourcePixelRegion.Bounds.Width;
int height = (int)sourcePixelRegion.Bounds.Height;

Then, it iterates only corner pixels.

int offsetX = _pixelSize / 2;
int offsetY = _pixelSize / 2;
 
for (int x = 0; x < width; x += _pixelSize)
{
for (int y = 0; y < height; y += _pixelSize)
{
//calculation here
}
}

Final code looks as below

public class PixelInterpolation : CustomEffectBase
{
private int _pixelSize = 0;
 
public PixelInterpolation(IImageProvider source, int PixelSize)
: base(source)
{
_pixelSize = PixelSize;
}
 
protected override void OnProcess(PixelRegion sourcePixelRegion, PixelRegion targetPixelRegion)
{
int offsetX = _pixelSize / 2;
int offsetY = _pixelSize / 2;
 
int width = (int)sourcePixelRegion.Bounds.Width;
int height = (int)sourcePixelRegion.Bounds.Height;
 
for (int x = 0; x < width; x += _pixelSize)
{
for (int y = 0; y < height; y += _pixelSize)
{
// make sure that the offset is within the boundry of the image
while (x + offsetX >= width) offsetX--;
while (y + offsetY >= height) offsetY--;
 
// get the pixel color in the center of the soon to be pixelated area
var pixel = sourcePixelRegion.ImagePixels[DimensionConverter.To1D(x + offsetX, y + offsetY, width)];
 
// for each pixel in the pixelate size, set it to the center color
for (int i = x; i < x + _pixelSize && i < width; i++)
for (int j = y; j < y + _pixelSize && j < height; j++)
targetPixelRegion.ImagePixels[DimensionConverter.To1D(i, j, width)] = pixel;
}
}
}
}

How to use?

As far as now, I described confusing things. The most easy part of this Imaging Framework is how to use. It's really simple.

First of all, add ImagingManager to page;

ImagingManager ImagingManager = new ImagingManager();

Then, start a new session;

await ImagingManager.StartSession(imageStream, imageControl);
  • ImageStream is whatever you want to use from them.
  • imageControl for viewing purpose. Use image control which is in page.


After that, add filters to be applied. The only restriction here, when initializing custom filters, you have to pass ImageProvider. But you don't have to worry about provider, use DummySource. ImagingManager handles and changes it.

var effects2 = new FilterEffect();
effects2.Filters = new IFilter[]
{
new BlurFilter(32)
};
 
ImagingManager.AddFilter(effects2);
 
ImagingManager.AddFilter(new ThresholdFilter(ImagingManager.DummySource, 150, 100, 0));
ImagingManager.AddFilter(new ColorPaletteFilter(ImagingManager.DummySource, AccentColorsPalette()));
ImagingManager.AddFilter(new PixelInterpolation(ImagingManager.DummySource, 64));

Finally, render it.

await ImagingManager.Render();

Note that in sample project, you can see more detailed use of framework.

Sample Filters

  • IPixelManipulation
  • HistogramFilter (collects Red, Green, Blue and Luminance data for histogram of image)
  • CornerDetection (detects corners of quadratical shape in black&white image)
  • QuadTransformation (gets quadratic shape using corner points, then transforms shape to rectangular form)

Contribution

If you want to contribute the project, please contact me over github page.

Conclusion

It's my approach for custom filter and built-in filters for Nokia Imaging SDK. I'm not an expert in imaging process, I tried my best and continue to learn new things. I believe Imaging Framework will speed up development of filter based application. I'm also using it with my applications and constantly updated and added new things to it. I'm planning to implement more filters that used for image processing.

391 page views in the last 30 days.