×
Namespaces

Variants
Actions

Pixelation Custom Effect (Nokia Imaging SDK)

From Nokia Developer Wiki
Jump to: navigation, search

This article explains how to create a Pixelation Custom Effect.

SignpostIcon XAML 40.png
WP Metro Icon WP8.png
Article Metadata
Tested with
Devices(s): Nokia Lumia 925
Compatibility
Platform(s):
Windows Phone 8
Dependencies: Nokia Imaging SDK 1.0
Article
Created: Rob.Kachmar (06 Feb 2014)
Last edited: Rob.Kachmar (18 Feb 2014)

Contents

Introduction

A pixelated image is one where the individual pixels from which it is constructed are visible, resulting in curved objects and diagonal lines having a jagged and "unnatural" appearance. Pixelation was very common in the early days of computing and computer games, where screens had very low resolutions and few colour choices.

The PixelationEffect simulates this effect by setting square blocks of pixels in the image to have the same "representative" colour - each block can then be thought of as a visible "pseudo pixel". The colour of these blocks can be chosen from the block corner pixels in the original image, the centre pixel, or an average of all the pixels in the block (expensive processing).

This article uses a simple algorithm, where the colour of the block's centre pixel in the original image is applied to all the other pixels in the block. The filter also allows the size of the blocks to be specified - a larger block size will show more obvious "pixelation" of the final image.

The article provides links to the filter source code and test code, an explanation of how the filter works, and code snippets showing how it is used.

Source code

Full source code for the PixelationEffect custom effect is provided below (toggle "Expand") and also in a ready-to-use file here: PixelationEffect.cs.

PixelationEffect.cs (03 Feb 2014)

  1. // ============================================================================
  2. // DATE        AUTHOR                   DESCRIPTION
  3. // ----------  -----------------------  ---------------------------------------
  4. // 2014.01.20  Rob.Kachmar              Initial creation
  5. // ============================================================================
  6.  
  7. using System;
  8. using Nokia.Graphics.Imaging;
  9.  
  10. namespace NISDKExtendedEffects.ImageEffects
  11. {
  12.     public class PixelationEffect : CustomEffectBase
  13.     {
  14.         private int m_Scale = 1;
  15.         private int m_ProcessEveryNthRow = 1;
  16.         private int m_ProcessEveryNthColumn = 1;
  17.         private int m_RowModuloTarget = 0;
  18.         private int m_ColumnModuloTarget = 0;
  19.  
  20.         public PixelationEffect(IImageProvider source, int scale = 1) : base(source)
  21.         {
  22.             m_Scale = (scale <= 0) ? 1 : scale; // Protect against divide by zero;
  23.         }
  24.  
  25.         protected override void OnProcess(PixelRegion sourcePixelRegion, PixelRegion targetPixelRegion)
  26.         {
  27.             var sourcePixels = sourcePixelRegion.ImagePixels;
  28.             var targetPixels = targetPixelRegion.ImagePixels;
  29.  
  30.             m_ProcessEveryNthRow = m_Scale;
  31.             m_ProcessEveryNthColumn = m_Scale;
  32.             m_RowModuloTarget = m_ProcessEveryNthRow - 1;
  33.             m_ColumnModuloTarget = m_ProcessEveryNthColumn - 1;
  34.  
  35.             int rowIndex = 0;
  36.             sourcePixelRegion.ForEachRow((index, width, position) =>
  37.             {
  38.                 if ((rowIndex % m_ProcessEveryNthRow).Equals(m_RowModuloTarget)) // only process on every other Nth pixel per row
  39.                 {
  40.                     for (int x = 0; x < width; ++x, ++index)
  41.                     {
  42.                         if ((x % m_ProcessEveryNthColumn).Equals(m_ColumnModuloTarget)) // only process on every other Nth pixel per column
  43.                         {
  44.                             // Get the center pixel for the given scale we are working with, and manipulate as desired
  45.                             int centerRowOffset = -1 * ((m_ProcessEveryNthRow - 1) / 2);
  46.                             int centerColumnOffset = -1 * ((m_ProcessEveryNthColumn - 1) / 2);
  47.                             uint targetPixel = sourcePixels[FindIndex(rowIndex, x, width, centerRowOffset, centerColumnOffset)];
  48.  
  49.                             // Get the top left position of the pixel block, given the current scale
  50.                             int topRowOffset = -1 * (m_ProcessEveryNthRow - 1);
  51.                             int leftColumnOffset = -1 * (m_ProcessEveryNthColumn - 1);
  52.  
  53.                             // Loop from the top left position down to the bottom right, where we stopped to process
  54.                             for (int y1 = topRowOffset; y1 <= 0; y1++)
  55.                             {
  56.                                 for (int x1 = leftColumnOffset; x1 <= 0; x1++)
  57.                                 {
  58.                                     targetPixels[FindIndex(rowIndex, x, width, y1, x1)] = targetPixel;
  59.                                 }
  60.                             }
  61.                         }
  62.                     }
  63.                 }
  64.                 rowIndex++;
  65.             });
  66.         }
  67.  
  68.         private int FindIndex(int rowIndex, int columnIndex, int width, int rowOffset, int columnOffset)
  69.         {
  70.             return ((rowIndex + rowOffset) * width) + (columnIndex + columnOffset);
  71.         }
  72.     }
  73. }

Testing

The Test Apps for Viewing Custom Filters allow you to cycle through a number of different custom effects (including PixelationEffect) and apply them to the real-time camera preview or a static image.

In addition, the custom effect file can be dropped into your own project, and used as described in the section Using the filter.

Pre-requisites

Performance

Device Pixelation 5x Pixelation 15x Pixelation 35x Pixelation 128x
Lumia 925 14 FPS 16 FPS 16 FPS 17 FPS

The PixelationEffect class runs 14-17 FPS (Frames Per Second) depending on how intense you want the image to be pixelated.

Code walkthrough

Note.pngNote: This code walkthrough is borrowed from Custom Filter QuickStart for Nokia Imaging SDK

First, let's start off visualizing what the top left corner looks like on an image that is 640 pixels wide. As you can see, the index at row 1, column 1 is equal to 0. Then it continues to increment by one as we progress through the columns. At index 639, we have finally reach column 640 of row 1. Since we have no more columns to go to on row 1, we now progress to the 2nd row, which begins with the next index of 640.

CustomFilters PixelMap.PNG

Now we need to work out the formula necessary to access other pixels relative to the current pixel. Let's say we're at row 3, column 3, and we want to access the pixel to the immediate top left at row 2, column 2. Given our 640 pixel wide image and our current index of 1282, we would get the index value of 641 with this method. However, if you try using this with real-time effects, the performance degradation is too great.

  1. private int FindIndex(int index, int width, int rowOffset, int columnOffset)
  2. {
  3.     int currentRowIndex = (int)Math.Truncate((double)(index / width));
  4.     int currentColumnIndex = (index % width);
  5.     return ((currentRowIndex + rowOffset) * width) + (currentColumnIndex + columnOffset);
  6. }

A work around is to keep track of the row index as we iterate through the ForEachRow method. We already have the column index of x, so now we can eliminate the first 2 calculations in our method.

  1. private int FindIndex(int rowIndex, int columnIndex, int width, int rowOffset, int columnOffset)
  2. {
  3.     return ((rowIndex + rowOffset) * width) + (columnIndex + columnOffset);
  4. }

Now let's look at how to skip rows and columns. The trick is to keep track of the row index, as mentioned above, then take the modulo of the row index based on the rows you want to skip. You would also do the same thing with the column index, x. Let's take a look at the code below, which will only process every 3rd row and every 3rd column. Go ahead and play around with the processEveryNthRow and processEveryNthColumn parameters to see what happens.

Note.pngNote: If modulo is throwing you off, you can view many other explanations of it here.

  1. int processEveryNthRow = 3;
  2. int processEveryNthColumn = 3;
  3. processEveryNthRow = (processEveryNthRow <= 0) ? 1 : processEveryNthRow; // Protect against divide by zero
  4. processEveryNthColumn = (processEveryNthColumn <= 0) ? 1 : processEveryNthColumn; // Protect against divide by zero
  5. int rowModuloTarget = processEveryNthRow - 1;
  6. int columnModuloTarget = processEveryNthColumn - 1;
  7.  
  8. int rowIndex = 0;
  9. sourcePixelRegion.ForEachRow((index, width, position) =>
  10. {
  11.     if ((rowIndex % processEveryNthRow).Equals(rowModuloTarget)) // only process on every Nth pixel per row
  12.     {
  13.         for (int x = 0; x < width; ++x, ++index)
  14.         {
  15.             if ((x % processEveryNthColumn).Equals(columnModuloTarget)) // only process on every Nth pixel per column
  16.             {
  17.                 targetPixels[index] = sourcePixels[index];
  18.             }
  19.         }
  20.     }
  21.     rowIndex++;
  22. });

Now let's see how we can combine the FindIndex method and the row/column skipping technique to efficiently create an effect that will intentionally pixelate an image. Let's say we want to pixelate in groups of 9, like our pixel map above is segmented. We'll need to find a way to get the color value of the middle pixel in the group, and then apply it to all the surrounding pixels.

Looking at the pixel map above, the middle value of the first group is 641. Whatever color value is in pixel 641 is what we want to apply to the surrounding pixels 0, 1, 2, 640, 642, 1280, 1281, and 1282. One approach to solve this challenge is to skip to every 3rd row and column, which would put us at pixel 1282 for our first group. Once we get there, we need to get the value of the center pixel. Using our FindIndex method, we would go 1 row up and 1 column back. Then we would get the pixel at the targetIndex and apply it to the other pixels relative to our current location.

  1. int processEveryNthRow = 3;
  2. int processEveryNthColumn = 3;
  3. processEveryNthRow = (processEveryNthRow <= 0) ? 1 : processEveryNthRow; // Protect against divide by zero
  4. processEveryNthColumn = (processEveryNthColumn <= 0) ? 1 : processEveryNthColumn; // Protect against divide by zero
  5. int rowModuloTarget = processEveryNthRow - 1;
  6. int columnModuloTarget = processEveryNthColumn - 1;
  7.  
  8. int rowIndex = 0;
  9. sourcePixelRegion.ForEachRow((index, width, position) =>
  10. {
  11.     if ((rowIndex % processEveryNthRow).Equals(rowModuloTarget)) // only process on every other Nth pixel per row
  12.     {
  13.         for (int x = 0; x < width; ++x, ++index)
  14.         {
  15.             if ((x % processEveryNthColumn).Equals(columnModuloTarget)) // only process on every other Nth pixel per column
  16.             {
  17.                 int targetIndex = FindIndex(rowIndex, x, width, -1, -1); // Get the index of the center pixel from the group of 9
  18.                 if (targetIndex > 0) // Move foward processing if in a valid range
  19.                 {
  20.                     uint targetPixel = sourcePixels[targetIndex]; // Get the actual center pixel from the group of 9
  21.  
  22.                     // Assign the center pixel to all 9 pixels in the group
  23.                     targetPixels[FindIndex(rowIndex, x, width, -2, -2)] = targetPixel; // Top left
  24.                     targetPixels[FindIndex(rowIndex, x, width, -2, -1)] = targetPixel; // Top center
  25.                     targetPixels[FindIndex(rowIndex, x, width, -2, 0)] = targetPixel; // Top right
  26.                     targetPixels[FindIndex(rowIndex, x, width, -1, -2)] = targetPixel; // Middle left
  27.                     targetPixels[FindIndex(rowIndex, x, width, -1, -1)] = targetPixel; // Center
  28.                     targetPixels[FindIndex(rowIndex, x, width, -1, 0)] = targetPixel; // Middle right
  29.                     targetPixels[FindIndex(rowIndex, x, width, 0, -2)] = targetPixel; // Bottom left
  30.                     targetPixels[FindIndex(rowIndex, x, width, 0, -1)] = targetPixel; // Bottom center
  31.                     targetPixels[index] = targetPixel; // Bottom right - where we stopped to process
  32.                 }
  33.             }
  34.         }
  35.     }
  36.     rowIndex++;
  37. });

The above code definitely does the job of pixelating the image, but there's also a problem; it's not very scalable. What if we want to do a smaller 2 row by 2 column area or a larger 4 row by 4 column area? We could add a bunch of case statements and turn this thing into a real mess, but we should probably go another route. Let's work through this challenge to improve our pixel map navigation skills.

First, let's see another view of the top left section of the pixel map based on pixel size. As you can see from the images below, we expand out an additional row and column as we increase the scale factor of a pixel. This makes it easy for us to use our skip row/column technique above and just have the processEveryNthRow and processEveryNthColumn parameters both equal to whatever pixel scale size we choose.

Next, we need a couple of formulas that will be necessary each time we stop to process another pixel block. We need to identify the center pixel for the block, as well as the very top left pixel of the block.

  • S >>> Every row/column to stop and process, and in our current scenario, will also be the same value as the scale factor.
  • C >>> Every row/column back (offset) from the processing stop (S), which we need to run through our FindIndex method to get the center pixel.
  • T >>> Every row/column back (offset) from the processing stop (S), which we need to run through our FindIndex method to get the top left pixel.
  • C = -(S - 1) / 2
  • T = -(S - 1)

Note.pngNote: The result for C will be a decimal for any even blocks like 2x, 4x, etc. You can either round up or down, because they are both technically in the center. For our code, we'll just let the result get automatically converted to an integer, and let the framework pick the direction.

Finally, we need to double loop from the top left (T) back down to the bottom right where we stopped (S). We just increment the row and column offsets (T) until we finish off the assignments. Let's take a look at the new scalable code below that will efficiently pixelate an image.

  1.         int scale = 3;
  2.         scale = (scale <= 0) ? 1 : scale; // Protect against divide by zero
  3.         int processEveryNthRow = scale;
  4.         int processEveryNthColumn = scale;
  5.         int rowModuloTarget = processEveryNthRow - 1;
  6.         int columnModuloTarget = processEveryNthColumn - 1;
  7.  
  8.         int rowIndex = 0;
  9.         sourcePixelRegion.ForEachRow((index, width, position) =>
  10.         {
  11.             if ((rowIndex % processEveryNthRow).Equals(rowModuloTarget)) // only process on every other Nth pixel per row
  12.             {
  13.                 for (int x = 0; x < width; ++x, ++index)
  14.                 {
  15.                     if ((x % processEveryNthColumn).Equals(columnModuloTarget)) // only process on every other Nth pixel per column
  16.                     {
  17.                         // Get the center pixel for the given scale we are working with, and manipulate as desired
  18.                         int centerRowOffset = -1 * ((processEveryNthRow - 1) / 2); // C = -(S - 1) / 2
  19.                         int centerColumnOffset = -1 * ((processEveryNthColumn - 1) / 2); // C = -(S - 1) / 2
  20.                         uint targetPixel = sourcePixels[FindIndex(rowIndex, x, width, centerRowOffset, centerColumnOffset)]; 
  21.  
  22.                         // Get the top left position of the pixel block, given the current scale
  23.                         int topRowOffset = -1 * (processEveryNthColumn - 1); // T = -(S - 1)
  24.                         int leftColumnOffset = -1 * (processEveryNthColumn - 1); // T = -(S - 1)
  25.  
  26.                         // Loop from the top left position down to the bottom right, where we stopped to process
  27.                         for (int y1 = topRowOffset; y1 <= 0; y1++)
  28.                         {
  29.                             for (int x1 = leftColumnOffset; x1 <= 0; x1++)
  30.                             {
  31.                                 targetPixels[FindIndex(rowIndex, x, width, y1, x1)] = targetPixel;
  32.                             }
  33.                         }
  34.                     }
  35.                 }
  36.             }
  37.             rowIndex++;
  38.         });


Using the filter

Drop an image control into your XAML.

  1. <Image x:Name="FilterEffectImage" Width="800" Height="480" Stretch="Fill" Grid.RowSpan="2" />

Use this code to apply the filter effect to your chosen image and assign it to the XAML image control.

  1. // Initialize a WriteableBitmap with the dimensions of the XAML image control
  2. WriteableBitmap writeableBitmap = new WriteableBitmap((int)FilterEffectImage.Width, (int)FilterEffectImage.Height);
  3.  
  4. // Example: Accessing an image stream within a standard photo chooser task callback
  5. // http://msdn.microsoft.com/en-us/library/windowsphone/develop/hh394019(v=vs.105).aspx
  6. //using (var imageStream = new StreamImageSource(e.ChosenPhoto))
  7.  
  8. // Example: Accessing an image stream from a sample picture loaded with the project in a folder called "Pictures"
  9. var resource = App.GetResourceStream(new Uri(string.Concat("Pictures/", "sample_photo_08.jpg"), UriKind.Relative));
  10. using (var imageStream = new StreamImageSource(resource.Stream))
  11. {
  12.     // Applying the custom filter effect to the image stream
  13.     using (var customEffect = new PixelationEffect(imageStream, 13))
  14.     {
  15.         // Rendering the resulting image to a WriteableBitmap
  16.         using (var renderer = new WriteableBitmapRenderer(customEffect, writeableBitmap))
  17.         {
  18.             // Applying the WriteableBitmap to our xaml image control
  19.             FilterEffectImage.Source = await renderer.RenderAsync();
  20.         }
  21.     }
  22. }


License

The code has been released with the standard MIT License, and can be viewed in the Github project here.


Summary

Hopefully you've enjoyed seeing how to navigate through the pixels of an image and manipulating them into a pixelated effect. This is just one of many things you can do with the amazing Nokia Imaging SDK.

As with all articles on the wiki, you are welcome to contribute any changes to this code that would improve the quality, whether it be additional features or improving its efficiency.

This page was last modified on 18 February 2014, at 09:29.
166 page views in the last 30 days.