×
Namespaces

Variants
Actions

FishEye lens filter effect

From Nokia Developer Wiki
Jump to: navigation, search
Featured Article
07 Jul
2014

This article explains how to implement a FishEye Lens filter effect.

SignpostIcon XAML 40.png
WP Metro Icon WP8.png
Article Metadata
Code ExampleTested with
SDK: Windows Phone 8.0, 8.1
Devices(s): Nokia Lumia 920, 625 , 620
Compatibility
Platform(s):
Windows Phone 8
Article
Created: Loukt (17 Jun 2014)
Last edited: kiran10182 (07 Jul 2014)

Contents

Introduction

A FishEye lens is an ultra wide-angle lens that produces strong visual distortion to create a wide panoramic or hemispherical image.

You can use the FishEye Lens effect described in this article to create fun and visually interesting selfies.

Celebrity selfie from the Oscars, with FishEye effect (oval and circular).


FishEye Effect

FishEye lenses achieve extremely wide angles of view. Rather than producing images with straight lines of perspective (rectilinear images), they use a special mapping (for example: equisolid angle). This gives the impression that the lines of the image are curved, not straight.

Image projection example
Usually, a FishEye image is created using a special lens. However, we can replicate this effect on any image by doing a simple projection of each pixel of an image on to a spherical surface. It doesn't need to be a perfect sphere.
Before and After applying the FishEye effect
Generally, the output is a round image that is zoomed in the center and curved towards the edges. All of the pixels outside of the circle are discarded.

Creating the effect

Before creating a custom effect, we recommend you read Custom Filter QuickStart for Nokia Imaging SDK.

How to calculate the numerical distortion values

Basically, to go from a normal picture to a FishEye one, we need a function to handle the projection of the pixels from the original image to the new one.

Such an operation comes with a heavy computational cost, which results in 3-5 fps. Therefore, to lighten the projection process, we start by normalizing all the pixels to a range of [-1 1] for both height and width, leaving the (0,0) in the centre of the square. The transformation to FishEye will result in a circle of radius 1.

For example, imagine we have a 400x400 image, meaning that its boundaries are [0,0,400,400]. The height goes from [0-400] and the width from [0-400] too. To normalize this image, we will divide it by the maximum value and subtract 1.

0< x <400
0< x/400 <1 -divided by the max value = 400
-1 < (x/400)-1 <1 remove 1
-1 < x' <1 let's put x' = (x/400)-1 and do the same with y (the height)

From a bound box of [0,0,400,400] we've now shifted to [-1,-1,1,1].

Applying the FishEye effect now becomes computationally easier. It could work with different sizes and shapes. Even if you start with a rectangle, the normalized picture will be a square, resulting in a circle transformation, which, in the end, gives a ellipsis.

The circle resulting from normalization is of radius 1, which means it's a unit circle with the following rules:

x² + y² = 1  //radius of 1
Solving y
y² = 1 - x²
and then finally
y = sqrt(1 - x²)
The orange square is the image after being normalized with the bounds [-1,-1,1,1]. The transformation with the FishEye gives an inner circle with radius = 1, discarding the pixels with radius over 1 (gray zones).
From another point of view (surface projection), given a pixel P1 on an image with a distance R from the center, its projection on a sphere surface will result in a new distance from the sphere's center on the surface R'. We'll simply have to use this new distance in a 2D image in order to get the output we want.

Now we know that what we need is to calculate that blue line, "new radius". Being within a boundary box of [-1,-1,1,1] with sphere and circles, it's better to work with polar coordinates instead of Cartesian coordinates to compute the new radius. Remember that since there is a new radius, there is also a new x,y position.

The Cartesian coordinates use x and y to determine the position. On the contrary, the Polar coordinates use the radius and angle from the center to determine the position. Hence we will need to do a conversion from cartesian to polar, and back from polar to cartesian.

Cartesian => Polar:
r = sqrt(x² + y²) // r² = x² + y²
theta = atan2(y, x)
 
Polar => Cartesian:
x = r*cos(theta)
y = r*sin(theta)

Using the Polar coordinates with Spheres (3D objects) means there is an elevation which is determined by the 3rd element z(z = 0 in 2D).

The radius of the sphere is the same as the primitive circle r=1.

x²+y²+z² = 1  this is the radius equation,
x²+y² = r² from the primitive circle
r² + z² = 1 <=> z = sqrt(1 - r²)

After we do the projection, we need to go back from the spherical projection to the primitive circle using each pixel's new radius in the conversion from Polar to Cartesian. Suppose we're applying an Equal-Area Spherical projection, using the following equations:

sqrt((x² + y² + (1 - z)²)/2) = sqrt(1 - z)

The formula to get the new radius is simplified by this:

 r' = r + (1 - sqrt(1 -r²)) / 2

After getting the new radius, one more small step remains: converting from polar to Cartesian coordinates.

Finally the code

protected override void OnProcess(PixelRegion sourcePixelRegion, PixelRegion targetPixelRegion)
{
var sourcePixels = sourcePixelRegion.ImagePixels;
var targetPixels = targetPixelRegion.ImagePixels;
int rowindex = 0;
sourcePixelRegion.ForEachRow((index, width, position) =>
{
// normalize y coordinate to -1 ... 1
double ny = ((2*rowindex)/sourcePixelRegion.ImageSize.Height) - 1;
// for each column
for (int x = 0; x < sourcePixelRegion.ImageSize.Width; x++,index++)
{
// normalize x coordinate to -1 ... 1
double nx = ((2*x)/sourcePixelRegion.ImageSize.Width) - 1;
// calculate distance from center (0,0)
double radius = Math.Sqrt(nx*nx + ny*ny);
// discard pixels outside from circle!
if (0 <= radius && radius <= 1)
{
//compute the distorted radius
double newRadius = (radius + (1 - Math.Sqrt(1 - radius*radius)))/2;
// discard radius greater than 1, which will result in black zones
if (newRadius <= 1)
{
// calculate the angle for polar coordinates
double theta = Math.Atan2(ny, nx);
// calculate new x,y position using new distance in same angle
double nxn = newRadius*Math.Cos(theta);
double nyn = newRadius*Math.Sin(theta);
// map from -1 ... 1 to image coordinates
int x2 = (int) (((nxn + 1)*sourcePixelRegion.ImageSize.Width)/2);
int y2 = (int) (((nyn + 1)*sourcePixelRegion.ImageSize.Height)/2);
// find (x2,y2) position from source pixels
int srcpos = (int) (y2*sourcePixelRegion.ImageSize.Width + x2);
// make sure that position stays within arrays
if (srcpos >= 0 & srcpos < sourcePixelRegion.ImageSize.Width*sourcePixelRegion.ImageSize.Height)
{
targetPixels[index] = sourcePixels[srcpos];
}
}
}
}
rowindex++;
});
}

Performance

Device FishEyeEffect
Lumia 920 3-5 FPS
Lumia 620 2-3 FPS

The FishEyeEffect class runs 2-5 FPS (Frames Per Second).

Using the FishEye Effect

There are multiple ways of using the filter, but here is a quick and easy way to see an example. Download the RealTimeFilterDemo solution available here.

It's recommended to read the walk-through of the solution available here too).

case xx: //replace xx with a number according the effectCount in your list
{
EffectName = String.Format(nameFormat, (_effectIndex + 1), "FishEye");
_customEffect = new FishEyeEffect(_cameraPreviewImageSource); // or ImageSource if your on the FixedViewer project
}
break;

Conclusion

Now that you understand how the distortion from a normal picture to a FishEye effect is done, here are some picture examples done with this filter.

More information

Different Spherical projections formulas
This code was based on this blog post from Popscan.
FishEye Lens.

This page was last modified on 7 July 2014, at 17:53.
958 page views in the last 30 days.
×