×
Namespaces

Variants
Actions

Blob Counter Custom Effect (Nokia Imaging SDK)

From Nokia Developer Wiki
Jump to: navigation, search

This article explains how to create an Blob Counter custom effect.

SignpostIcon XAML 40.png
Article Metadata
Tested with
Devices(s): Nokia Lumia 920
Dependencies: Nokia Imaging SDK 1.0
Article
Created: Engin Kırmacı (19 Feb 2014)
Last edited: Engin Kırmacı (01 Mar 2014)

Contents

Introduction

Source code

Full source code for the BlobCounter custom effect is provided below (toggle "Expand") and also in a ready-to-use file here: BlobCounter.cs.

BlobCounter.cs (28 Feb 2014)

  1. // ============================================================================
  2. // DATE        AUTHOR                   DESCRIPTION
  3. // ----------  -----------------------  ---------------------------------------
  4. // 2014.02.13  Engin.Kırmacı            Initial creation
  5. // ============================================================================
  6.  
  7. using NISDKExtendedEffects.Comparers;
  8. using NISDKExtendedEffects.Entities;
  9. using Nokia.Graphics.Imaging;
  10. using System;
  11. using System.Collections.Generic;
  12. using System.Windows;
  13. using Windows.UI;
  14.  
  15. namespace NISDKExtendedEffects.ImageEffects
  16. {
  17.     public class BlobCounter : CustomEffectBase
  18.     {
  19.         // found blobs
  20.         private List<Blob> blobs = new List<Blob>();
  21.  
  22.         // objects' sort order
  23.         private ObjectsOrder objectsOrder = ObjectsOrder.None;
  24.  
  25.         // filtering by size is required or not
  26.         private bool filterBlobs = false;
  27.  
  28.         private IBlobsFilter filter = null;
  29.  
  30.         // coupled size filtering or not
  31.         private bool coupledSizeFiltering = false;
  32.  
  33.         // blobs' minimal and maximal size
  34.         private int minWidth = 1;
  35.  
  36.         private int minHeight = 1;
  37.         private int maxWidth = int.MaxValue;
  38.         private int maxHeight = int.MaxValue;
  39.  
  40.         private const byte backgroundThresholdR = 0;
  41.         private const byte backgroundThresholdG = 0;
  42.         private const byte backgroundThresholdB = 0;
  43.  
  44.         private bool hasPreview = false;
  45.         private int previewCount = 10;
  46.  
  47.         /// <summary>
  48.         /// Objects count.
  49.         /// </summary>
  50.         protected int objectsCount;
  51.  
  52.         /// <summary>
  53.         /// Objects' labels.
  54.         /// </summary>
  55.         protected int[] objectLabels;
  56.  
  57.         /// <summary>
  58.         /// Width of processed image.
  59.         /// </summary>
  60.         protected int imageWidth;
  61.  
  62.         /// <summary>
  63.         /// Height of processed image.
  64.         /// </summary>
  65.         protected int imageHeight;
  66.  
  67.         /// <summary>
  68.         /// Objects count.
  69.         /// </summary>
  70.         ///
  71.         /// <remarks><para>Number of objects (blobs) found by <see cref="ProcessImage(Bitmap)"/> method.
  72.         /// </para></remarks>
  73.         ///
  74.         public int ObjectsCount
  75.         {
  76.             get { return objectsCount; }
  77.         }
  78.  
  79.         /// <summary>
  80.         /// Objects' labels.
  81.         /// </summary>
  82.         ///
  83.         /// <remarks>The array of <b>width</b> * <b>height</b> size, which holds
  84.         /// labels for all objects. Background is represented with <b>0</b> value,
  85.         /// but objects are represented with labels starting from <b>1</b>.</remarks>
  86.         ///
  87.         public int[] ObjectLabels
  88.         {
  89.             get { return objectLabels; }
  90.         }
  91.  
  92.         /// <summary>
  93.         /// Objects sort order.
  94.         /// </summary>
  95.         ///
  96.         /// <remarks><para>The property specifies objects' sort order, which are provided
  97.         /// by <see cref="GetObjectsRectangles"/>, <see cref="GetObjectsInformation"/>, etc.
  98.         /// </para></remarks>
  99.         ///
  100.         public ObjectsOrder ObjectsOrder
  101.         {
  102.             get { return objectsOrder; }
  103.             set { objectsOrder = value; }
  104.         }
  105.  
  106.         /// <summary>
  107.         /// Specifies if blobs should be filtered.
  108.         /// </summary>
  109.         ///
  110.         /// <remarks><para>If the property is equal to <b>false</b>, then there is no any additional
  111.         /// post processing after image was processed. If the property is set to <b>true</b>, then
  112.         /// blobs filtering is done right after image processing routine. If <see cref="BlobsFilter"/>
  113.         /// is set, then custom blobs' filtering is done, which is implemented by user. Otherwise
  114.         /// blobs are filtered according to dimensions specified in <see cref="MinWidth"/>,
  115.         /// <see cref="MinHeight"/>, <see cref="MaxWidth"/> and <see cref="MaxHeight"/> properties.</para>
  116.         ///
  117.         /// <para>Default value is set to <see langword="false"/>.</para></remarks>
  118.         ///
  119.         public bool FilterBlobs
  120.         {
  121.             get { return filterBlobs; }
  122.             set { filterBlobs = value; }
  123.         }
  124.  
  125.         /// <summary>
  126.         /// Specifies if size filetering should be coupled or not.
  127.         /// </summary>
  128.         ///
  129.         /// <remarks><para>In uncoupled filtering mode, objects are filtered out in the case if
  130.         /// their width is smaller than <see cref="MinWidth"/> <b>or</b> height is smaller than
  131.         /// <see cref="MinHeight"/>. But in coupled filtering mode, objects are filtered out in
  132.         /// the case if their width is smaller than <see cref="MinWidth"/> <b>and</b> height is
  133.         /// smaller than <see cref="MinHeight"/>. In both modes the idea with filtering by objects'
  134.         /// maximum size is the same as filtering by objects' minimum size.</para>
  135.         ///
  136.         /// <para>Default value is set to <see langword="false"/>, what means uncoupled filtering by size.</para>
  137.         /// </remarks>
  138.         ///
  139.         public bool CoupledSizeFiltering
  140.         {
  141.             get { return coupledSizeFiltering; }
  142.             set { coupledSizeFiltering = value; }
  143.         }
  144.  
  145.         /// <summary>
  146.         /// Minimum allowed width of blob.
  147.         /// </summary>
  148.         ///
  149.         /// <remarks><para>The property specifies minimum object's width acceptable by blob counting
  150.         /// routine and has power only when <see cref="FilterBlobs"/> property is set to
  151.         /// <see langword="true"/> and <see cref="BlobsFilter">custom blobs' filter</see> is
  152.         /// set to <see langword="null"/>.</para>
  153.         ///
  154.         /// <para>See documentation to <see cref="CoupledSizeFiltering"/> for additional information.</para>
  155.         /// </remarks>
  156.         ///
  157.         public int MinWidth
  158.         {
  159.             get { return minWidth; }
  160.             set { minWidth = value; }
  161.         }
  162.  
  163.         /// <summary>
  164.         /// Minimum allowed height of blob.
  165.         /// </summary>
  166.         ///
  167.         /// <remarks><para>The property specifies minimum object's height acceptable by blob counting
  168.         /// routine and has power only when <see cref="FilterBlobs"/> property is set to
  169.         /// <see langword="true"/> and <see cref="BlobsFilter">custom blobs' filter</see> is
  170.         /// set to <see langword="null"/>.</para>
  171.         ///
  172.         /// <para>See documentation to <see cref="CoupledSizeFiltering"/> for additional information.</para>
  173.         /// </remarks>
  174.         ///
  175.         public int MinHeight
  176.         {
  177.             get { return minHeight; }
  178.             set { minHeight = value; }
  179.         }
  180.  
  181.         /// <summary>
  182.         /// Maximum allowed width of blob.
  183.         /// </summary>
  184.         ///
  185.         /// <remarks><para>The property specifies maximum object's width acceptable by blob counting
  186.         /// routine and has power only when <see cref="FilterBlobs"/> property is set to
  187.         /// <see langword="true"/> and <see cref="BlobsFilter">custom blobs' filter</see> is
  188.         /// set to <see langword="null"/>.</para>
  189.         ///
  190.         /// <para>See documentation to <see cref="CoupledSizeFiltering"/> for additional information.</para>
  191.         /// </remarks>
  192.         ///
  193.         public int MaxWidth
  194.         {
  195.             get { return maxWidth; }
  196.             set { maxWidth = value; }
  197.         }
  198.  
  199.         /// <summary>
  200.         /// Maximum allowed height of blob.
  201.         /// </summary>
  202.         ///
  203.         /// <remarks><para>The property specifies maximum object's height acceptable by blob counting
  204.         /// routine and has power only when <see cref="FilterBlobs"/> property is set to
  205.         /// <see langword="true"/> and <see cref="BlobsFilter">custom blobs' filter</see> is
  206.         /// set to <see langword="null"/>.</para>
  207.         ///
  208.         /// <para>See documentation to <see cref="CoupledSizeFiltering"/> for additional information.</para>
  209.         /// </remarks>
  210.         ///
  211.         public int MaxHeight
  212.         {
  213.             get { return maxHeight; }
  214.             set { maxHeight = value; }
  215.         }
  216.  
  217.         /// <summary>
  218.         /// Custom blobs' filter to use.
  219.         /// </summary>
  220.         ///
  221.         /// <remarks><para>The property specifies custom blobs' filtering routine to use. It has
  222.         /// effect only in the case if <see cref="FilterBlobs"/> property is set to <see langword="true"/>.</para>
  223.         ///
  224.         /// <para><note>When custom blobs' filtering routine is set, it has priority over default filtering done
  225.         /// with <see cref="MinWidth"/>, <see cref="MinHeight"/>, <see cref="MaxWidth"/> and <see cref="MaxHeight"/>.</note></para>
  226.         /// </remarks>
  227.         ///
  228.         public IBlobsFilter BlobsFilter
  229.         {
  230.             get { return filter; }
  231.             set { filter = value; }
  232.         }
  233.  
  234.         public bool HasPreview
  235.         {
  236.             get { return hasPreview; }
  237.             set { hasPreview = value; }
  238.         }
  239.  
  240.         public int PreviewCount
  241.         {
  242.             get { return previewCount; }
  243.             set { previewCount = value; }
  244.         }
  245.  
  246.         public BlobCounter(IImageProvider source)
  247.             : base(source, true)
  248.         {
  249.         }
  250.  
  251.         protected override void OnProcess(PixelRegion sourcePixelRegion, PixelRegion targetPixelRegion)
  252.         {
  253.             imageWidth = (int)sourcePixelRegion.Bounds.Width;
  254.             imageHeight = (int)sourcePixelRegion.Bounds.Height;
  255.  
  256.             // do actual objects map building
  257.             BuildObjectsMap(sourcePixelRegion);
  258.  
  259.             // collect information about blobs
  260.             CollectObjectsInfo(sourcePixelRegion);
  261.  
  262.             // filter blobs by size if required
  263.             if (filterBlobs)
  264.             {
  265.                 // labels remapping array
  266.                 int[] labelsMap = new int[objectsCount + 1];
  267.  
  268.                 for (int i = 1; i <= objectsCount; i++)
  269.                 {
  270.                     labelsMap[i] = i;
  271.                 }
  272.  
  273.                 // check dimension of all objects and filter them
  274.                 int objectsToRemove = 0;
  275.  
  276.                 if (filter == null)
  277.                 {
  278.                     for (int i = objectsCount - 1; i >= 0; i--)
  279.                     {
  280.                         int blobWidth = (int)blobs[i].Rectangle.Width;
  281.  
  282.                         int blobHeight = (int)blobs[i].Rectangle.Height;
  283.  
  284.                         if (coupledSizeFiltering == false)
  285.                         {
  286.                             // uncoupled filtering
  287.                             if (
  288.                                 (blobWidth < minWidth) || (blobHeight < minHeight) ||
  289.                                 (blobWidth > maxWidth) || (blobHeight > maxHeight))
  290.                             {
  291.                                 labelsMap[i + 1] = 0;
  292.  
  293.                                 objectsToRemove++;
  294.                                 blobs.RemoveAt(i);
  295.                             }
  296.                         }
  297.                         else
  298.                         {
  299.                             // coupled filtering
  300.                             if (
  301.                                 ((blobWidth < minWidth) && (blobHeight < minHeight)) ||
  302.                                 ((blobWidth > maxWidth) && (blobHeight > maxHeight)))
  303.                             {
  304.                                 labelsMap[i + 1] = 0;
  305.                                 objectsToRemove++;
  306.                                 blobs.RemoveAt(i);
  307.                             }
  308.                         }
  309.                     }
  310.                 }
  311.                 else
  312.                 {
  313.                     for (int i = objectsCount - 1; i >= 0; i--)
  314.                     {
  315.                         if (!filter.Check(blobs[i]))
  316.                         {
  317.                             labelsMap[i + 1] = 0;
  318.                             objectsToRemove++;
  319.                             blobs.RemoveAt(i);
  320.                         }
  321.                     }
  322.                 }
  323.  
  324.                 // update labels remapping array
  325.                 int label = 0;
  326.                 for (int i = 1; i <= objectsCount; i++)
  327.                 {
  328.                     if (labelsMap[i] != 0)
  329.                     {
  330.                         label++;
  331.                         // update remapping array
  332.                         labelsMap[i] = label;
  333.                     }
  334.                 }
  335.  
  336.                 // repair object labels
  337.                 for (int i = 0, n = objectLabels.Length; i < n; i++)
  338.                 {
  339.                     objectLabels[i] = labelsMap[objectLabels[i]];
  340.                 }
  341.  
  342.                 objectsCount -= objectsToRemove;
  343.  
  344.                 // repair IDs
  345.                 for (int i = 0, n = blobs.Count; i < n; i++)
  346.                 {
  347.                     blobs[i].ID = i + 1;
  348.                 }
  349.             }
  350.  
  351.             // do we need to sort the list?
  352.             if (objectsOrder != ObjectsOrder.None)
  353.             {
  354.                 blobs.Sort(new BlobsSorter(objectsOrder));
  355.             }
  356.  
  357.             if (HasPreview)
  358.             {
  359.                 var rects = GetObjectsRectangles();
  360.                 Random rand = new Random();
  361.  
  362.                 int count = 0;
  363.                 foreach (var rect in rects)
  364.                 {
  365.                     DrawRectangle(sourcePixelRegion, rect, new Color() { A = 255, R = (byte)rand.Next(0, 255), G = (byte)rand.Next(0, 255), B = (byte)rand.Next(0, 255) });
  366.  
  367.                     count++;
  368.  
  369.                     if (count == PreviewCount)
  370.                         break;
  371.                 }
  372.             }
  373.         }
  374.  
  375.         private void DrawRectangle(PixelRegion sourcePixelRegion, Rect rectangle, Color color)
  376.         {
  377.             uint Color = FromColor(color);
  378.  
  379.             var topLeft = new Point(rectangle.X, rectangle.Y);
  380.             var topRight = new Point(rectangle.X + rectangle.Width, rectangle.Y);
  381.             var bottomLeft = new Point(rectangle.X, rectangle.Y + rectangle.Height);
  382.             var bottomRight = new Point(rectangle.X + rectangle.Width, rectangle.Y + rectangle.Height);
  383.             // draw line on the image
  384.             DrawLineBresenham(sourcePixelRegion,
  385.                 topLeft,
  386.                 topRight,
  387.                 Color);
  388.  
  389.             DrawLineBresenham(sourcePixelRegion,
  390.                 topRight,
  391.                 bottomRight,
  392.                 Color);
  393.  
  394.             DrawLineBresenham(sourcePixelRegion,
  395.                 bottomRight,
  396.                 bottomLeft,
  397.                 Color);
  398.  
  399.             DrawLineBresenham(sourcePixelRegion,
  400.                 bottomLeft,
  401.                topLeft,
  402.                 Color);
  403.         }
  404.  
  405.         public void DrawLineBresenham(PixelRegion sourcePixelRegion, Point p1, Point p2, uint color)
  406.         {
  407.             int w = (int)sourcePixelRegion.ImageSize.Width;
  408.             int h = (int)sourcePixelRegion.ImageSize.Height;
  409.             var pixels = sourcePixelRegion.ImagePixels;
  410.  
  411.             int x1 = (int)p1.X;
  412.             int y1 = (int)p1.Y;
  413.  
  414.             int x2 = (int)p2.X;
  415.             int y2 = (int)p2.Y;
  416.  
  417.             // Distance start and end point
  418.             int dx = x2 - x1;
  419.             int dy = y2 - y1;
  420.  
  421.             // Determine sign for direction x
  422.             int incx = 0;
  423.             if (dx < 0)
  424.             {
  425.                 dx = -dx;
  426.                 incx = -1;
  427.             }
  428.             else if (dx > 0)
  429.             {
  430.                 incx = 1;
  431.             }
  432.  
  433.             // Determine sign for direction y
  434.             int incy = 0;
  435.             if (dy < 0)
  436.             {
  437.                 dy = -dy;
  438.                 incy = -1;
  439.             }
  440.             else if (dy > 0)
  441.             {
  442.                 incy = 1;
  443.             }
  444.  
  445.             // Which gradient is larger
  446.             int pdx, pdy, odx, ody, es, el;
  447.             if (dx > dy)
  448.             {
  449.                 pdx = incx;
  450.                 pdy = 0;
  451.                 odx = incx;
  452.                 ody = incy;
  453.                 es = dy;
  454.                 el = dx;
  455.             }
  456.             else
  457.             {
  458.                 pdx = 0;
  459.                 pdy = incy;
  460.                 odx = incx;
  461.                 ody = incy;
  462.                 es = dx;
  463.                 el = dy;
  464.             }
  465.  
  466.             // Init start
  467.             int x = x1;
  468.             int y = y1;
  469.             int error = el >> 1;
  470.             if (y < h && y >= 0 && x < w && x >= 0)
  471.             {
  472.                 pixels[y * w + x] = color;
  473.             }
  474.  
  475.             // Walk the line!
  476.             for (int i = 0; i < el; i++)
  477.             {
  478.                 // Update error term
  479.                 error -= es;
  480.  
  481.                 // Decide which coord to use
  482.                 if (error < 0)
  483.                 {
  484.                     error += el;
  485.                     x += odx;
  486.                     y += ody;
  487.                 }
  488.                 else
  489.                 {
  490.                     x += pdx;
  491.                     y += pdy;
  492.                 }
  493.  
  494.                 // Set pixel
  495.                 if (y < h && y >= 0 && x < w && x >= 0)
  496.                 {
  497.                     pixels[y * w + x] = color;
  498.                 }
  499.             }
  500.         }
  501.  
  502.         public Rect[] GetObjectsRectangles()
  503.         {
  504.             // check if objects map was collected
  505.             if (objectLabels == null)
  506.                 throw new Exception("Image should be processed before to collect objects map.");
  507.  
  508.             Rect[] rects = new Rect[objectsCount];
  509.  
  510.             for (int i = 0; i < objectsCount; i++)
  511.             {
  512.                 rects[i] = blobs[i].Rectangle;
  513.             }
  514.  
  515.             return rects;
  516.         }
  517.  
  518.         public Blob[] GetObjectsInformation()
  519.         {
  520.             // check if objects map was collected
  521.             if (objectLabels == null)
  522.                 throw new ApplicationException("Image should be processed before to collect objects map.");
  523.  
  524.             Blob[] blobsToReturn = new Blob[objectsCount];
  525.  
  526.             // create each blob
  527.             for (int k = 0; k < objectsCount; k++)
  528.             {
  529.                 blobsToReturn[k] = new Blob(blobs[k]);
  530.             }
  531.  
  532.             return blobsToReturn;
  533.         }
  534.  
  535.         public void GetBlobsLeftAndRightEdges(Blob blob, out List<Point> leftEdge, out List<Point> rightEdge)
  536.         {
  537.             // check if objects map was collected
  538.             if (objectLabels == null)
  539.                 throw new Exception("Image should be processed before to collect objects map.");
  540.  
  541.             leftEdge = new List<Point>();
  542.             rightEdge = new List<Point>();
  543.  
  544.             int xmin = (int)blob.Rectangle.Left;
  545.             int xmax = (int)(xmin + blob.Rectangle.Width - 1);
  546.             int ymin = (int)blob.Rectangle.Top;
  547.             int ymax = (int)(ymin + blob.Rectangle.Height - 1);
  548.  
  549.             int label = blob.ID;
  550.  
  551.             // for each line
  552.             for (int y = ymin; y <= ymax; y++)
  553.             {
  554.                 // scan from left to right
  555.                 int p = y * imageWidth + xmin;
  556.                 for (int x = xmin; x <= xmax; x++, p++)
  557.                 {
  558.                     if (objectLabels[p] == label)
  559.                     {
  560.                         leftEdge.Add(new Point(x, y));
  561.                         break;
  562.                     }
  563.                 }
  564.  
  565.                 // scan from right to left
  566.                 p = y * imageWidth + xmax;
  567.                 for (int x = xmax; x >= xmin; x--, p--)
  568.                 {
  569.                     if (objectLabels[p] == label)
  570.                     {
  571.                         rightEdge.Add(new Point(x, y));
  572.                         break;
  573.                     }
  574.                 }
  575.             }
  576.         }
  577.  
  578.         public void GetBlobsTopAndBottomEdges(Blob blob, out List<Point> topEdge, out List<Point> bottomEdge)
  579.         {
  580.             // check if objects map was collected
  581.             if (objectLabels == null)
  582.                 throw new ApplicationException("Image should be processed before to collect objects map.");
  583.  
  584.             topEdge = new List<Point>();
  585.             bottomEdge = new List<Point>();
  586.  
  587.             int xmin = (int)blob.Rectangle.Left;
  588.             int xmax = (int)(xmin + blob.Rectangle.Width - 1);
  589.             int ymin = (int)blob.Rectangle.Top;
  590.             int ymax = (int)(ymin + blob.Rectangle.Height - 1);
  591.  
  592.             int label = blob.ID;
  593.  
  594.             // for each column
  595.             for (int x = xmin; x <= xmax; x++)
  596.             {
  597.                 // scan from top to bottom
  598.                 int p = ymin * imageWidth + x;
  599.                 for (int y = ymin; y <= ymax; y++, p += imageWidth)
  600.                 {
  601.                     if (objectLabels[p] == label)
  602.                     {
  603.                         topEdge.Add(new Point(x, y));
  604.                         break;
  605.                     }
  606.                 }
  607.  
  608.                 // scan from bottom to top
  609.                 p = ymax * imageWidth + x;
  610.                 for (int y = ymax; y >= ymin; y--, p -= imageWidth)
  611.                 {
  612.                     if (objectLabels[p] == label)
  613.                     {
  614.                         bottomEdge.Add(new Point(x, y));
  615.                         break;
  616.                     }
  617.                 }
  618.             }
  619.         }
  620.  
  621.         public List<Point> GetBlobsEdgePoints(Blob blob)
  622.         {
  623.             // check if objects map was collected
  624.             if (objectLabels == null)
  625.                 throw new ApplicationException("Image should be processed before to collect objects map.");
  626.  
  627.             List<Point> edgePoints = new List<Point>();
  628.  
  629.             int xmin = (int)blob.Rectangle.Left;
  630.             int xmax = (int)(xmin + blob.Rectangle.Width - 1);
  631.             int ymin = (int)blob.Rectangle.Top;
  632.             int ymax = (int)(ymin + blob.Rectangle.Height - 1);
  633.  
  634.             int label = blob.ID;
  635.  
  636.             // array of already processed points on left/right edges
  637.             // (index in these arrays represent Y coordinate, but value - X coordinate)
  638.             int[] leftProcessedPoints = new int[(int)blob.Rectangle.Height];
  639.             int[] rightProcessedPoints = new int[(int)blob.Rectangle.Height];
  640.  
  641.             // for each line
  642.             for (int y = ymin; y <= ymax; y++)
  643.             {
  644.                 // scan from left to right
  645.                 int p = y * imageWidth + xmin;
  646.                 for (int x = xmin; x <= xmax; x++, p++)
  647.                 {
  648.                     if (objectLabels[p] == label)
  649.                     {
  650.                         edgePoints.Add(new Point(x, y));
  651.                         leftProcessedPoints[y - ymin] = x;
  652.                         break;
  653.                     }
  654.                 }
  655.  
  656.                 // scan from right to left
  657.                 p = y * imageWidth + xmax;
  658.                 for (int x = xmax; x >= xmin; x--, p--)
  659.                 {
  660.                     if (objectLabels[p] == label)
  661.                     {
  662.                         // avoid adding the point we already have
  663.                         if (leftProcessedPoints[y - ymin] != x)
  664.                         {
  665.                             edgePoints.Add(new Point(x, y));
  666.                         }
  667.                         rightProcessedPoints[y - ymin] = x;
  668.                         break;
  669.                     }
  670.                 }
  671.             }
  672.  
  673.             // for each column
  674.             for (int x = xmin; x <= xmax; x++)
  675.             {
  676.                 // scan from top to bottom
  677.                 int p = ymin * imageWidth + x;
  678.                 for (int y = ymin, y0 = 0; y <= ymax; y++, y0++, p += imageWidth)
  679.                 {
  680.                     if (objectLabels[p] == label)
  681.                     {
  682.                         // avoid adding the point we already have
  683.                         if ((leftProcessedPoints[y0] != x) &&
  684.                              (rightProcessedPoints[y0] != x))
  685.                         {
  686.                             edgePoints.Add(new Point(x, y));
  687.                         }
  688.                         break;
  689.                     }
  690.                 }
  691.  
  692.                 // scan from bottom to top
  693.                 p = ymax * imageWidth + x;
  694.                 for (int y = ymax, y0 = ymax - ymin; y >= ymin; y--, y0--, p -= imageWidth)
  695.                 {
  696.                     if (objectLabels[p] == label)
  697.                     {
  698.                         // avoid adding the point we already have
  699.                         if ((leftProcessedPoints[y0] != x) &&
  700.                              (rightProcessedPoints[y0] != x))
  701.                         {
  702.                             edgePoints.Add(new Point(x, y));
  703.                         }
  704.                         break;
  705.                     }
  706.                 }
  707.             }
  708.  
  709.             return edgePoints;
  710.         }
  711.  
  712.         private void BuildObjectsMap(PixelRegion sourcePixelRegion)
  713.         {
  714.             // we don't want one pixel width images
  715.             if (imageWidth == 1)
  716.             {
  717.                 throw new Exception("BlobCounter cannot process images that are one pixel wide. Rotate the image or use RecursiveBlobCounter.");
  718.             }
  719.  
  720.             int imageWidthM1 = imageWidth - 1;
  721.  
  722.             // allocate labels array
  723.             objectLabels = new int[imageWidth * imageHeight];
  724.             // initial labels count
  725.             int labelsCount = 0;
  726.  
  727.             // create map
  728.             int maxObjects = ((imageWidth / 2) + 1) * ((imageHeight / 2) + 1) + 1;
  729.             int[] map = new int[maxObjects];
  730.  
  731.             // initially map all labels to themself
  732.             for (int i = 0; i < maxObjects; i++)
  733.             {
  734.                 map[i] = i;
  735.             }
  736.  
  737.             // do the job
  738.             int pos = 0;
  739.             int p = 0;
  740.  
  741.             // color images
  742.             int pixelSize = 1;
  743.             int offset = imageWidth * pixelSize;
  744.  
  745.             int stride = imageWidth;
  746.             int strideM1 = stride - pixelSize;
  747.             int strideP1 = stride + pixelSize;
  748.  
  749.             // 1 - for pixels of the first row
  750.             Color firstColor = ToColor(sourcePixelRegion.ImagePixels[pos]);
  751.  
  752.             if ((firstColor.R | firstColor.G | firstColor.B) != 0)
  753.             {
  754.                 objectLabels[p] = ++labelsCount;
  755.             }
  756.             pos += pixelSize;
  757.             ++p;
  758.  
  759.             for (int x = 1; x < imageWidth; x++, pos += pixelSize, p++)
  760.             {
  761.                 Color color = ToColor(sourcePixelRegion.ImagePixels[pos]);
  762.  
  763.                 // check if we need to label current pixel
  764.                 if ((color.R > backgroundThresholdR) ||
  765.                      (color.G > backgroundThresholdG) ||
  766.                      (color.B > backgroundThresholdB))
  767.                 {
  768.                     Color prevColor = ToColor(sourcePixelRegion.ImagePixels[pos - 1]);
  769.                     // check if the previous pixel already was labeled
  770.                     if ((prevColor.R > backgroundThresholdR) ||
  771.                          (prevColor.G > backgroundThresholdG) ||
  772.                          (prevColor.B > backgroundThresholdB))
  773.                     {
  774.                         // label current pixel, as the previous
  775.                         objectLabels[p] = objectLabels[p - 1];
  776.                     }
  777.                     else
  778.                     {
  779.                         // create new label
  780.                         objectLabels[p] = ++labelsCount;
  781.                     }
  782.                 }
  783.             }
  784.  
  785.             // 2 - for other rows
  786.             // for each row
  787.             for (int y = 1; y < imageHeight; y++)
  788.             {
  789.                 Color rowFirstColor = ToColor(sourcePixelRegion.ImagePixels[pos]);
  790.                 // for the first pixel of the row, we need to check
  791.                 // only upper and upper-right pixels
  792.                 if ((rowFirstColor.R > backgroundThresholdR) ||
  793.                         (rowFirstColor.G > backgroundThresholdG) ||
  794.                         (rowFirstColor.B > backgroundThresholdB))
  795.                 {
  796.                     Color aboveColor = ToColor(sourcePixelRegion.ImagePixels[pos - stride]);
  797.                     Color aboveRightColor = ToColor(sourcePixelRegion.ImagePixels[pos - strideM1]);
  798.                     // check surrounding pixels
  799.                     if ((aboveColor.R > backgroundThresholdR) ||
  800.                          (aboveColor.G > backgroundThresholdG) ||
  801.                          (aboveColor.B > backgroundThresholdB))
  802.                     {
  803.                         // label current pixel, as the above
  804.                         objectLabels[p] = objectLabels[p - imageWidth];
  805.                     }
  806.                     else if ((aboveRightColor.R > backgroundThresholdR) ||
  807.                               (aboveRightColor.G > backgroundThresholdG) ||
  808.                               (aboveRightColor.B > backgroundThresholdB))
  809.                     {
  810.                         // label current pixel, as the above right
  811.                         objectLabels[p] = objectLabels[p + 1 - imageWidth];
  812.                     }
  813.                     else
  814.                     {
  815.                         // create new label
  816.                         objectLabels[p] = ++labelsCount;
  817.                     }
  818.                 }
  819.                 pos += pixelSize;
  820.                 ++p;
  821.  
  822.                 // check left pixel and three upper pixels for the rest of pixels
  823.                 for (int x = 1; x < imageWidth - 1; x++, pos += pixelSize, p++)
  824.                 {
  825.                     Color color = ToColor(sourcePixelRegion.ImagePixels[pos]);
  826.                     if ((color.R > backgroundThresholdR) ||
  827.                          (color.G > backgroundThresholdG) ||
  828.                          (color.B > backgroundThresholdB))
  829.                     {
  830.                         Color leftColor = ToColor(sourcePixelRegion.ImagePixels[pos - pixelSize]);
  831.                         Color aboveLeftColor = ToColor(sourcePixelRegion.ImagePixels[pos - strideP1]);
  832.                         Color aboveColor = ToColor(sourcePixelRegion.ImagePixels[pos - stride]);
  833.                         Color aboveRightColor = ToColor(sourcePixelRegion.ImagePixels[pos - strideM1]);
  834.  
  835.                         // check surrounding pixels
  836.                         if ((leftColor.R > backgroundThresholdR) ||
  837.                              (leftColor.G > backgroundThresholdG) ||
  838.                              (leftColor.B > backgroundThresholdB))
  839.                         {
  840.                             // label current pixel, as the left
  841.                             objectLabels[p] = objectLabels[p - 1];
  842.                         }
  843.                         else if ((aboveLeftColor.R > backgroundThresholdR) ||
  844.                                   (aboveLeftColor.G > backgroundThresholdG) ||
  845.                                   (aboveLeftColor.B > backgroundThresholdB))
  846.                         {
  847.                             // label current pixel, as the above left
  848.                             objectLabels[p] = objectLabels[p - 1 - imageWidth];
  849.                         }
  850.                         else if ((aboveColor.R > backgroundThresholdR) ||
  851.                                   (aboveColor.G > backgroundThresholdG) ||
  852.                                   (aboveColor.B > backgroundThresholdB))
  853.                         {
  854.                             // label current pixel, as the above
  855.                             objectLabels[p] = objectLabels[p - imageWidth];
  856.                         }
  857.  
  858.                         if ((aboveRightColor.R > backgroundThresholdR) ||
  859.                              (aboveRightColor.G > backgroundThresholdG) ||
  860.                              (aboveRightColor.B > backgroundThresholdB))
  861.                         {
  862.                             if (objectLabels[p] == 0)
  863.                             {
  864.                                 // label current pixel, as the above right
  865.                                 objectLabels[p] = objectLabels[p + 1 - imageWidth];
  866.                             }
  867.                             else
  868.                             {
  869.                                 int l1 = objectLabels[p];
  870.                                 int l2 = objectLabels[p + 1 - imageWidth];
  871.  
  872.                                 if ((l1 != l2) && (map[l1] != map[l2]))
  873.                                 {
  874.                                     // merge
  875.                                     if (map[l1] == l1)
  876.                                     {
  877.                                         // map left value to the right
  878.                                         map[l1] = map[l2];
  879.                                     }
  880.                                     else if (map[l2] == l2)
  881.                                     {
  882.                                         // map right value to the left
  883.                                         map[l2] = map[l1];
  884.                                     }
  885.                                     else
  886.                                     {
  887.                                         // both values already mapped
  888.                                         map[map[l1]] = map[l2];
  889.                                         map[l1] = map[l2];
  890.                                     }
  891.  
  892.                                     // reindex
  893.                                     for (int i = 1; i <= labelsCount; i++)
  894.                                     {
  895.                                         if (map[i] != i)
  896.                                         {
  897.                                             // reindex
  898.                                             int j = map[i];
  899.                                             while (j != map[j])
  900.                                             {
  901.                                                 j = map[j];
  902.                                             }
  903.                                             map[i] = j;
  904.                                         }
  905.                                     }
  906.                                 }
  907.                             }
  908.                         }
  909.  
  910.                         // label the object if it is not yet
  911.                         if (objectLabels[p] == 0)
  912.                         {
  913.                             // create new label
  914.                             objectLabels[p] = ++labelsCount;
  915.                         }
  916.                     }
  917.                 }
  918.  
  919.                 // for the last pixel of the row, we need to check
  920.                 // only upper and upper-left pixels
  921.                 Color upperColor = ToColor(sourcePixelRegion.ImagePixels[pos]);
  922.                 if ((upperColor.R > backgroundThresholdR) ||
  923.                      (upperColor.G > backgroundThresholdG) ||
  924.                      (upperColor.B > backgroundThresholdB))
  925.                 {
  926.                     Color upperLeftColor = ToColor(sourcePixelRegion.ImagePixels[pos - pixelSize]);
  927.                     Color aboveLeftColor = ToColor(sourcePixelRegion.ImagePixels[pos - strideP1]);
  928.                     Color aboveColor = ToColor(sourcePixelRegion.ImagePixels[pos - stride]);
  929.                     // check surrounding pixels
  930.                     if ((upperLeftColor.R > backgroundThresholdR) ||
  931.                          (upperLeftColor.G > backgroundThresholdG) ||
  932.                          (upperLeftColor.B > backgroundThresholdB))
  933.                     {
  934.                         // label current pixel, as the left
  935.                         objectLabels[p] = objectLabels[p - 1];
  936.                     }
  937.                     else if ((aboveLeftColor.R > backgroundThresholdR) ||
  938.                               (aboveLeftColor.G > backgroundThresholdG) ||
  939.                               (aboveLeftColor.B > backgroundThresholdB))
  940.                     {
  941.                         // label current pixel, as the above left
  942.                         objectLabels[p] = objectLabels[p - 1 - imageWidth];
  943.                     }
  944.                     else if ((aboveColor.R > backgroundThresholdR) ||
  945.                               (aboveColor.G > backgroundThresholdG) ||
  946.                               (aboveColor.B > backgroundThresholdB))
  947.                     {
  948.                         // label current pixel, as the above
  949.                         objectLabels[p] = objectLabels[p - imageWidth];
  950.                     }
  951.                     else
  952.                     {
  953.                         // create new label
  954.                         objectLabels[p] = ++labelsCount;
  955.                     }
  956.                 }
  957.                 pos += pixelSize;
  958.                 ++p;
  959.             }
  960.  
  961.             // allocate remapping array
  962.             int[] reMap = new int[map.Length];
  963.  
  964.             // count objects and prepare remapping array
  965.             objectsCount = 0;
  966.             for (int i = 1; i <= labelsCount; i++)
  967.             {
  968.                 if (map[i] == i)
  969.                 {
  970.                     // increase objects count
  971.                     reMap[i] = ++objectsCount;
  972.                 }
  973.             }
  974.             // second pass to complete remapping
  975.             for (int i = 1; i <= labelsCount; i++)
  976.             {
  977.                 if (map[i] != i)
  978.                 {
  979.                     reMap[i] = reMap[map[i]];
  980.                 }
  981.             }
  982.  
  983.             // repair object labels
  984.             for (int i = 0, n = objectLabels.Length; i < n; i++)
  985.             {
  986.                 objectLabels[i] = reMap[objectLabels[i]];
  987.             }
  988.         }
  989.  
  990.         // Collect objects' rectangles
  991.         private void CollectObjectsInfo(PixelRegion sourcePixelRegion)
  992.         {
  993.             int i = 0, label;
  994.  
  995.             // create object coordinates arrays
  996.             int[] x1 = new int[objectsCount + 1];
  997.             int[] y1 = new int[objectsCount + 1];
  998.             int[] x2 = new int[objectsCount + 1];
  999.             int[] y2 = new int[objectsCount + 1];
  1000.  
  1001.             int[] area = new int[objectsCount + 1];
  1002.             long[] xc = new long[objectsCount + 1];
  1003.             long[] yc = new long[objectsCount + 1];
  1004.  
  1005.             long[] meanR = new long[objectsCount + 1];
  1006.             long[] meanG = new long[objectsCount + 1];
  1007.             long[] meanB = new long[objectsCount + 1];
  1008.  
  1009.             long[] stdDevR = new long[objectsCount + 1];
  1010.             long[] stdDevG = new long[objectsCount + 1];
  1011.             long[] stdDevB = new long[objectsCount + 1];
  1012.  
  1013.             for (int j = 1; j <= objectsCount; j++)
  1014.             {
  1015.                 x1[j] = imageWidth;
  1016.                 y1[j] = imageHeight;
  1017.             }
  1018.  
  1019.             // color images
  1020.             byte r, g, b; // RGB value
  1021.  
  1022.             // walk through labels array
  1023.             for (int y = 0; y < imageHeight; y++)
  1024.             {
  1025.                 for (int x = 0; x < imageWidth; x++, i++)
  1026.                 {
  1027.                     // get current label
  1028.                     label = objectLabels[i];
  1029.  
  1030.                     // skip unlabeled pixels
  1031.                     if (label == 0)
  1032.                         continue;
  1033.  
  1034.                     // check and update all coordinates
  1035.  
  1036.                     if (x < x1[label])
  1037.                     {
  1038.                         x1[label] = x;
  1039.                     }
  1040.                     if (x > x2[label])
  1041.                     {
  1042.                         x2[label] = x;
  1043.                     }
  1044.                     if (y < y1[label])
  1045.                     {
  1046.                         y1[label] = y;
  1047.                     }
  1048.                     if (y > y2[label])
  1049.                     {
  1050.                         y2[label] = y;
  1051.                     }
  1052.  
  1053.                     area[label]++;
  1054.                     xc[label] += x;
  1055.                     yc[label] += y;
  1056.  
  1057.                     Color c = ToColor(sourcePixelRegion.ImagePixels[y * imageWidth + x]);
  1058.                     r = c.R;
  1059.                     g = c.G;
  1060.                     b = c.B;
  1061.  
  1062.                     meanR[label] += r;
  1063.                     meanG[label] += g;
  1064.                     meanB[label] += b;
  1065.  
  1066.                     stdDevR[label] += r * r;
  1067.                     stdDevG[label] += g * g;
  1068.                     stdDevB[label] += b * b;
  1069.                 }
  1070.             }
  1071.  
  1072.             // create blobs
  1073.             blobs.Clear();
  1074.  
  1075.             for (int j = 1; j <= objectsCount; j++)
  1076.             {
  1077.                 int blobArea = area[j];
  1078.  
  1079.                 Blob blob = new Blob(j, new Rect(x1[j], y1[j], x2[j] - x1[j] + 1, y2[j] - y1[j] + 1));
  1080.                 blob.Area = blobArea;
  1081.                 blob.Fullness = (double)blobArea / ((x2[j] - x1[j] + 1) * (y2[j] - y1[j] + 1));
  1082.                 blob.CenterOfGravity = new Point((float)xc[j] / blobArea, (float)yc[j] / blobArea);
  1083.                 blob.ColorMean = Color.FromArgb(255, (byte)(meanR[j] / blobArea), (byte)(meanG[j] / blobArea), (byte)(meanB[j] / blobArea));
  1084.                 blob.ColorStdDev = Color.FromArgb(
  1085.                     255,
  1086.                     (byte)(Math.Sqrt(stdDevR[j] / blobArea - blob.ColorMean.R * blob.ColorMean.R)),
  1087.                     (byte)(Math.Sqrt(stdDevG[j] / blobArea - blob.ColorMean.G * blob.ColorMean.G)),
  1088.                     (byte)(Math.Sqrt(stdDevB[j] / blobArea - blob.ColorMean.B * blob.ColorMean.B)));
  1089.  
  1090.                 blobs.Add(blob);
  1091.             }
  1092.         }
  1093.     }
  1094. }

Testing

The Test Apps for Viewing Custom Filters allow you to cycle through a number of different custom effects (including BlobCounter) and apply them to the real-time camera preview or a static image.

In addition, the custom effect file can be dropped into your own project, and used as described in the section Using the filter.

Pre-requisites

Performance

Device OtsuThreshold + BlobCounter
Lumia 920 3-5 FPS

The BlobCounter class runs 3-5 FPS (Frames Per Second) with OtsuThreshold .

Code walkthrough

Using the filter

Drop an image control into your XAML.

  1. <Image x:Name="FilterEffectImage" Width="800" Height="480" Stretch="Fill" Grid.RowSpan="2" />

Use this code to apply the filter effect to your chosen image and assign it to the XAML image control.

  1. // Initialize a WriteableBitmap with the dimensions of the XAML image control
  2. WriteableBitmap writeableBitmap = new WriteableBitmap((int)FilterEffectImage.Width, (int)FilterEffectImage.Height);
  3.  
  4. // Example: Accessing an image stream within a standard photo chooser task callback
  5. // http://msdn.microsoft.com/en-us/library/windowsphone/develop/hh394019(v=vs.105).aspx
  6. //using (var imageStream = new StreamImageSource(e.ChosenPhoto))
  7.  
  8. // Example: Accessing an image stream from a sample picture loaded with the project in a folder called "Pictures"
  9. var resource = App.GetResourceStream(new Uri(string.Concat("Pictures/", "sample_photo_08.jpg"), UriKind.Relative));
  10. using (var imageStream = new StreamImageSource(resource.Stream))
  11. {
  12.     // Applying the custom filter effect to the image stream
  13.     using (var otsuThresholdEffect = new OtsuThresholdEffect(imageStream))
  14.     {
  15.         using (var customEffect = new BlobCounter(otsuThresholdEffect)
  16.         {
  17.             //Draws detected objects as rectangle, for more information http://www.aforgenet.com/articles/shape_checker/
  18.             HasPreview = true,
  19.             PreviewCount = 10,
  20.             ObjectsOrder = NISDKExtendedEffects.Entities.ObjectsOrder.Area
  21.         })
  22.         {
  23.             // Rendering the resulting image to a WriteableBitmap
  24.             using (var renderer = new WriteableBitmapRenderer(customEffect, writeableBitmap))
  25.             {
  26.                 // Applying the WriteableBitmap to our xaml image control
  27.                 FilterEffectImage.Source = await renderer.RenderAsync();
  28.             }
  29.         }
  30.     }
  31. }

License

The code has been released with the standard MIT License, and can be viewed in the Github project here.

Summary

Hopefully you've enjoyed seeing how to find blobs in an image using Otsu Threshold and Blob Counter. This is just one of many things you can do with the amazing Nokia Imaging SDK.

As with all articles on the wiki, you are welcome to contribute any changes to this code that would improve the quality, whether it be additional features or improving its efficiency.

This page was last modified on 1 March 2014, at 05:31.
58 page views in the last 30 days.

Was this page helpful?

Your feedback about this content is important. Let us know what you think.

 

Thank you!

We appreciate your feedback.

×