FishEye lens filter effect
Note: This is an entry in the Nokia Original Imaging Effect Wiki Challenge 2014Q2
This article explains how to implement a FishEye Lens filter effect.
A FishEye lens is an ultra wide-angle lens that produces strong visual distortion to create a wide panoramic or hemispherical image.
You can use the FishEye Lens effect described in this article to create fun and visually interesting selfies.
FishEye lenses achieve extremely wide angles of view. Rather than producing images with straight lines of perspective (rectilinear images), they use a special mapping (for example: equisolid angle). This gives the impression that the lines of the image are curved, not straight.
|Usually, a FishEye image is created using a special lens. However, we can replicate this effect on any image by doing a simple projection of each pixel of an image on to a spherical surface. It doesn't need to be a perfect sphere.|
|Generally, the output is a round image that is zoomed in the center and curved towards the edges. All of the pixels outside of the circle are discarded.|
Creating the effect
Before creating a custom effect, we recommend you read Custom Filter QuickStart for Nokia Imaging SDK.
How to calculate the numerical distortion values
Basically, to go from a normal picture to a FishEye one, we need a function to handle the projection of the pixels from the original image to the new one.
Such an operation comes with a heavy computational cost, which results in 3-5 fps. Therefore, to lighten the projection process, we start by normalizing all the pixels to a range of [-1 1] for both height and width, leaving the (0,0) in the centre of the square. The transformation to FishEye will result in a circle of radius 1.
For example, imagine we have a 400x400 image, meaning that its boundaries are [0,0,400,400]. The height goes from [0-400] and the width from [0-400] too. To normalize this image, we will divide it by the maximum value and subtract 1.
0< x <400
0< x/400 <1 -divided by the max value = 400
-1 < (x/400)-1 <1 remove 1
-1 < x' <1 let's put x' = (x/400)-1 and do the same with y (the height)
From a bound box of [0,0,400,400] we've now shifted to [-1,-1,1,1].
Applying the FishEye effect now becomes computationally easier. It could work with different sizes and shapes. Even if you start with a rectangle, the normalized picture will be a square, resulting in a circle transformation, which, in the end, gives a ellipsis.
The circle resulting from normalization is of radius 1, which means it's a unit circle with the following rules:
x² + y² = 1 //radius of 1
y² = 1 - x²
and then finally
y = sqrt(1 - x²)
Now we know that what we need is to calculate that blue line, "new radius". Being within a boundary box of [-1,-1,1,1] with sphere and circles, it's better to work with polar coordinates instead of Cartesian coordinates to compute the new radius. Remember that since there is a new radius, there is also a new x,y position.
The Cartesian coordinates use x and y to determine the position. On the contrary, the Polar coordinates use the radius and angle from the center to determine the position. Hence we will need to do a conversion from cartesian to polar, and back from polar to cartesian.
Cartesian => Polar:
r = sqrt(x² + y²) // r² = x² + y²
theta = atan2(y, x)
Polar => Cartesian:
x = r*cos(theta)
y = r*sin(theta)
Using the Polar coordinates with Spheres (3D objects) means there is an elevation which is determined by the 3rd element z(z = 0 in 2D).
The radius of the sphere is the same as the primitive circle r=1.
x²+y²+z² = 1 this is the radius equation,
x²+y² = r² from the primitive circle
r² + z² = 1 <=> z = sqrt(1 - r²)
After we do the projection, we need to go back from the spherical projection to the primitive circle using each pixel's new radius in the conversion from Polar to Cartesian. Suppose we're applying an Equal-Area Spherical projection, using the following equations:
sqrt((x² + y² + (1 - z)²)/2) = sqrt(1 - z)
The formula to get the new radius is simplified by this:
r' = r + (1 - sqrt(1 -r²)) / 2
After getting the new radius, one more small step remains: converting from polar to Cartesian coordinates.
Finally the code
protected override void OnProcess(PixelRegion sourcePixelRegion, PixelRegion targetPixelRegion)
var sourcePixels = sourcePixelRegion.ImagePixels;
var targetPixels = targetPixelRegion.ImagePixels;
int rowindex = 0;
sourcePixelRegion.ForEachRow((index, width, position) =>
// normalize y coordinate to -1 ... 1
double ny = ((2*rowindex)/sourcePixelRegion.ImageSize.Height) - 1;
// for each column
for (int x = 0; x < sourcePixelRegion.ImageSize.Width; x++,index++)
// normalize x coordinate to -1 ... 1
double nx = ((2*x)/sourcePixelRegion.ImageSize.Width) - 1;
// calculate distance from center (0,0)
double radius = Math.Sqrt(nx*nx + ny*ny);
// discard pixels outside from circle!
if (0 <= radius && radius <= 1)
//compute the distorted radius
double newRadius = (radius + (1 - Math.Sqrt(1 - radius*radius)))/2;
// discard radius greater than 1, which will result in black zones
if (newRadius <= 1)
// calculate the angle for polar coordinates
double theta = Math.Atan2(ny, nx);
// calculate new x,y position using new distance in same angle
double nxn = newRadius*Math.Cos(theta);
double nyn = newRadius*Math.Sin(theta);
// map from -1 ... 1 to image coordinates
int x2 = (int) (((nxn + 1)*sourcePixelRegion.ImageSize.Width)/2);
int y2 = (int) (((nyn + 1)*sourcePixelRegion.ImageSize.Height)/2);
// find (x2,y2) position from source pixels
int srcpos = (int) (y2*sourcePixelRegion.ImageSize.Width + x2);
// make sure that position stays within arrays
if (srcpos >= 0 & srcpos < sourcePixelRegion.ImageSize.Width*sourcePixelRegion.ImageSize.Height)
targetPixels[index] = sourcePixels[srcpos];
|Lumia 920||3-5 FPS|
|Lumia 620||2-3 FPS|
The FishEyeEffect class runs 2-5 FPS (Frames Per Second).
Using the FishEye Effect
There are multiple ways of using the filter, but here is a quick and easy way to see an example. Download the RealTimeFilterDemo solution available here.
It's recommended to read the walk-through of the solution available here too).
case xx: //replace xx with a number according the effectCount in your list
EffectName = String.Format(nameFormat, (_effectIndex + 1), "FishEye");
_customEffect = new FishEyeEffect(_cameraPreviewImageSource); // or ImageSource if your on the FixedViewer project
Now that you understand how the distortion from a normal picture to a FishEye effect is done, here are some picture examples done with this filter.