Namespaces

Variants
Actions

Please note that as of October 24, 2014, the Nokia Developer Wiki will no longer be accepting user contributions, including new entries, edits and comments, as we begin transitioning to our new home, in the Windows Phone Development Wiki. We plan to move over the majority of the existing entries over the next few weeks. Thanks for all your past and future contributions.

Talk:Real-time camera viewfinder filters in Native code

From Wiki
Jump to: navigation, search

Contents

Chintandave er - Thanks.

Hi Mansewiz, Thanks for this nice article.

I have sub-edited it and added category Windows phone 8 and did some code formatting.

Whenever you are done with this article, remove draft category. Once you finish this article, we will further sub-edit it and give you some suggestions after review.

Thanks

Chintan Dave.

Chintandave er 11:04, 27 November 2012 (EET)

Mansewiz - Not a draft anymore

Hi Dave, I'm pretty satisfied with what I have right now, so I removed the draft category. Looking forward review suggestions!

Thanks

Mansewiz 10:57, 2 December 2012 (EET)

TonyTam - Slow performance

Nice article but why there is only 9 fps on my Lumia 920 with Neon implementation.

TonyTam 17:01, 4 April 2013 (EEST)

Mansewiz - Slow performance.

@TonyTam: You're right, the version in project is not really fast.

There is a simple fix that should increase significantly the perf, change the line from CameraStreamSource.cs from :

   _frameTime = (int)TimeSpan.FromSeconds((double)1 / 30).Ticks;

to

  _frameTime = (int)TimeSpan.FromSeconds((double)0).Ticks;
I will check the fix in the project soon. Thanks for pointing this out!

Mansewiz 10:28, 8 April 2013 (EEST)

TonyTam - Slow performance

Thanks your quick reply! I tried the above code but the performance still the same.

TonyTam 17:50, 8 April 2013 (EEST)

Mansewiz - Slow performance.

With my current project, I get 15-16 fps on the Lumia 920 with a gray effect. (18fps without any effects). It's still a far cry from 30fps, but pretty decently smooth. I made quite a few changes to get there, my code needs quite a lot of cleanup, but I'm planning to update the project with these changes during the week-end.

Mansewiz 08:44, 9 April 2013 (EEST)

Small2 - Slow performance

Well , it wasn't works smoothly with complex effect. And I thought it was unnecessary to pass a PhotoCaptureDevice object to the d3d layer, just pass the camera data.

void ApplyEffect(const Platform::Array<int,1U>^ imageBuffer,Platform::WriteOnlyArray<uint8,1U>^outBuffer, int effectIndex);

ICameraCaptureDevice _camera; WindowsPhoneRuntimeComponent _cameraBuffer;

_camera.GetPreviewBufferArgb(_cameraData);

_cameraBuffer.ApplyEffect(_cameraData, _cameraFilteredData, EffectId);

I got an idea to implement the real-time effect with d3d, but I have little knowledge about the d3d. And I post it on the discussion board, but few guys replied. here was the url: http://www.developer.nokia.com/Community/Discussion/showthread.php?239753-troubles-on-showing-argb-data-in-a-surface-control

looking forward your ideas, many thanks for the wonderful wiki post!

small2 12:53, 13 April 2013 (EEST)

Mansewiz - Slow performance

Hey there. I just pushed an update to the project. I now get 23fps with a Lumia 920, compared to the original 9 fps. Not bad! I did a *lot* of changes, the code is still a bit dirty, I will clean it up and make update the wiki article when I get a minute. I don't really know what in all the changes I did improved the speed that much, but most likely the change of the interface between the native and managed components helped a lot. Hopefully those changes work for you too!

-Bert

Mansewiz 21:59, 13 April 2013 (EEST)

Intellsys - Slow performance

Nice article, I have traced a bit more accurately the FPS and processing time as follows:

In CameraStreamSource.cs

MediaStreamSample msSamp = new MediaStreamSample(

                   videoStreamDescription, 
                   dataSource.FrameStream, 
                   frameStreamOffset,
                   dataSource.FrameBufferSize,
                   currentTime,
                   emptySampleDict);
               ReportGetSampleCompleted(msSamp);
               DateTime now = DateTime.Now;
               frameDuration = (now - startTime).Ticks;
               currentTime += frameDuration;
               startTime = now;

And in MainPage.xaml.cs:

void m_timer_Tick(object sender, EventArgs e)

       {
           string str = "FPS: " + MyCameraMediaElement.RenderedFramesPerSecond + " Frame duration: " + source.frameDuration + " Processing FPS: " + 10000000.0/source.frameDuration;
           System.Diagnostics.Debug.WriteLine(str);
           this.FramerateInfo.Text = str;
       }

In debug I get:

FPS: 20 Frame duration: 519468 Processing FPS: 19.2504639361809 FPS: 20 Frame duration: 434049 Processing FPS: 23.0388734912418 FPS: 21 Frame duration: 499328 Processing FPS: 20.0269161753397 FPS: 21 Frame duration: 423652 Processing FPS: 23.6042789836942 FPS: 21 Frame duration: 472787 Processing FPS: 21.1511737843892 FPS: 21 Frame duration: 504598 Processing FPS: 19.817755916591 FPS: 21 Frame duration: 509412 Processing FPS: 19.6304759212582 FPS: 21 Frame duration: 376692 Processing FPS: 26.5468871120172 FPS: 21 Frame duration: 459377 Processing FPS: 21.7686127080807 FPS: 21 Frame duration: 469194 Processing FPS: 21.313145521895 FPS: 21 Frame duration: 449063 Processing FPS: 22.2685903759606 FPS: 21 Frame duration: 474571 Processing FPS: 21.0716626173955 FPS: 21 Frame duration: 444133 Processing FPS: 22.5157779313854 FPS: 21 Frame duration: 419735 Processing FPS: 23.8245559698381 FPS: 21 Frame duration: 476437 Processing FPS: 20.9891339253668 FPS: 21 Frame duration: 474308 Processing FPS: 21.0833466861196 FPS: 21 Frame duration: 432776 Processing FPS: 23.1066417731113 FPS: 21 Frame duration: 499353 Processing FPS: 20.0259135321106 FPS: 21 Frame duration: 344602 Processing FPS: 29.0189842194764 FPS: 21 Frame duration: 438889 Processing FPS: 22.7848043582774 FPS: 21 Frame duration: 369723 Processing FPS: 27.0472759336043 FPS: 21 Frame duration: 390983 Processing FPS: 25.5765595946627 FPS: 21 Frame duration: 364431 Processing FPS: 27.440036659889 FPS: 21 Frame duration: 333032 Processing FPS: 30.027144538663

And in release I constantly get FPS 25-26 (from MediaElement RenderedFramesPerSecond ) and Processing FPS 20-37. Looks like the framerate reported by the MediaElement is not fully correlated with the processing time of the frames.(Btw, I have used a Samsung Ativ S.)

BR,

Lau

intellsys 12:50, 18 April 2013 (EEST)

Mansewiz - Slow performance

Thanks Lau,

It is great to hear that performance of the latest version of the project is good on the Ativ S ! I guess the difference you're seeing between the fps from the MediaElement and processing counter comes from the ME doing the fps calculation on a longer time window: averaging over a longer time period (25-26 fps is somewhere in the middle of FPS20-37)

Mansewiz 08:32, 19 April 2013 (EEST)

Arash k - Great Atcile

Hi Mansewiz

Thanks for the great article. It helped me alot.

I am quite new in Silverlight and Windows Phone development. I have tried to find a way to save the output of MediaStreamSource in your article to a file (for example mp4 format) in Place of playing it in a MediaElemtn and I couldn't find any. Is it possible to do that?

BR,

Arash.

arash_k 12:01, 23 April 2013 (EEST)

Mansewiz - Saving mediaElement stream as mp4

Arash, you will find a good example how to record video here: http://www.developer.nokia.com/Community/Wiki/What%27s_new_in_Windows_Phone_8#Camera:_Video_Recording

Unfortunately, right now in WP8 it is not possible to encode as a video (mp4) any image frame streams other than the one coming directly from the camera. Say you want to modify the frames coming from camera (like doing a black and white effect) and save the resulting stream as a .mp4, the platform won't provide you the functionality you need. Your app could provide it's own video encoder, but building your own encoder is a *lot* of hard work.

Mansewiz 12:54, 23 April 2013 (EEST)

Arash k -

Thank a lot

arash_k 13:17, 23 April 2013 (EEST)

Besessener - Errors

Hi Mansewiz,

I tried your source code. I clone the git repository and opened the SLN file in Visual Studio 2012. But I can't compile it. It outputs the following errors:

Error 1 error C1010: unexpected end of file while looking for precompiled header. Did you forget to add '#include "pch.h"' to your source? c:\users\matze\desktop\nativefilterdemo\cameraeffectinterface\icameraeffect.cpp 7 Error 2 error C1192: #using failed on 'C:\Users\Matze\Desktop\NativeFilterDemo\Debug\CameraEffectInterface\CameraEffectInterface.winmd' c:\users\matze\desktop\nativefilterdemo\nativecomponent\pch.cpp 1 Error 3 Metadata file 'C:\Users\Matze\Desktop\NativeFilterDemo\Debug\CameraEffectInterface\CameraEffectInterface.winmd' could not be found C:\Users\Matze\Desktop\NativeFilterDemo\NativeFilterDemo\CSC Error 4 Metadata file 'C:\Users\Matze\Desktop\NativeFilterDemo\Debug\NativeComponent\NativeComponent.winmd' could not be found C:\Users\Matze\Desktop\NativeFilterDemo\NativeFilterDemo\CSC

I hope it's somthing that's very easy to fix and I also hope you tell me :)

Cheers,

Matthias

besessener 17:20, 10 May 2013 (EEST)

Besessener - save image

Me again...

How would one save a single frame of this stream? When I try to store the stream in a BitmapImage: im.SetSource(_frameStream);

i get null exceeption although the stream is not null :(

cheers

besessener 22:05, 12 May 2013 (EEST)

Mansewiz - Errors

Oh! I didn't try to deploy the latest commit to the emulator, and it obviously didn't work. I've fix the problem (a simple configuration flag) to the tip. Thanks Matthias!

Mansewiz 08:35, 13 May 2013 (EEST)

Mansewiz - save image

@Besessener, To save the image to a file, you should save to a png, see this entry in Stackoverflow: http://stackoverflow.com/questions/7378946/saving-bitmap-as-png-on-wp7

If what you're looking for is create a bitmap from the stream, you can use the WriteableBitmap:

               System.Windows.Deployment.Current.Dispatcher.BeginInvoke(() =>
               {
                   WriteableBitmap wb = new WriteableBitmap(dataSource.FrameWidth, dataSource.FrameHeight);
                   byte[] buf = dataSource.FrameStream.GetBuffer(); //When creating FrameStream make sure the buffer is publicly visible, it is not by default.
                   Buffer.BlockCopy(buf, 0, wb.Pixels, 0, buf.Length);
               });
This isn't super efficient, since it introduces a thread switch to UI and an full copy of the array, but if you're not doing it several times a seconds, it should be ok.

Mansewiz 09:50, 13 May 2013 (EEST)

Ngoctri - Video Encoding

Great article. It's so helpful for me.

And I want to make Video Recorder app that recorder video from Filter Camera. How can save video from frames of Media Element Source ? Any guide for me ?

Thanks,

ngoctri (talk) 04:55, 29 July 2013 (EEST)

Tunght 53 - why CameraEffectInterface project does not have pch.h file

Why it does not include pch.h file but can be compiled successfully. I do not have much experience in Visual Studio, please tell me why.

tunght_53 (talk) 21:09, 31 July 2013 (EEST)

Tunght 53 - pch.h file

I did some more searching and I found out that the Using precompiled header should be unchecked. Thank your for this great article.

tunght_53 (talk) 21:16, 31 July 2013 (EEST)

Hamishwillee - Thanks Tunght - Mansewiz - can you add instruction if needed to the article - Ngoctri

Ngoctri - Mansewiz is on holiday. You might want to ask this question on the windows phone imaging forum if they are not answered by the code in this article (or in any of the other Category:Camera on Windows Phone or Category:Imaging on Windows Phone articles.

hamishwillee (talk) 02:32, 1 August 2013 (EEST)

 

Was this page helpful?

Your feedback about this content is important. Let us know what you think.

 

Thank you!

We appreciate your feedback.

×