×
Namespaces

Variants
Actions
(Difference between revisions)

Enabling wallet payment by face recognition

From Nokia Developer Wiki
Jump to: navigation, search
galazzo (Talk | contribs)
(Galazzo - - Wallet.xaml)
SB Dev (Talk | contribs)
m (SB Dev -)
 
(41 intermediate revisions by 6 users not shown)
Line 1: Line 1:
[[Category:Draft]]
+
[[Category:Camera on Windows Phone]][[Category:Imaging on Windows Phone]][[Category:Wallet]][[Category:Code Examples]][[Category:Tutorial]][[Category:XAML]][[Category:Windows Phone 8]]
{{Abstract|This article, the second of a series based on [http://projects.developer.nokia.com/DSP/wiki DSP project] and focused on face recognition, shows step by step how to implement a way to process wallet payment authorization.}}
+
{{Abstract|This article shows how you can use face recognition to authorize a wallet payment. This is the second of a series of articles based on the Digital Signal Processing and focused on face recognition.}}
 
{{ArticleMetaData <!-- v1.2 -->
 
{{ArticleMetaData <!-- v1.2 -->
|sourcecode= <!-- Link to example source code e.g. [[Media:The Code Example ZIP.zip]] -->
+
|sourcecode= [[File:FaceWallet.zip|thumb| FaceWallet.zip]]
 
|installfile= <!-- Link to installation file (e.g. [[Media:The Installation File.sis]]) -->
 
|installfile= <!-- Link to installation file (e.g. [[Media:The Installation File.sis]]) -->
 
|devices= Nokia Lumia
 
|devices= Nokia Lumia
Line 8: Line 8:
 
|platform= Windows Phone 8
 
|platform= Windows Phone 8
 
|devicecompatability= <!-- Compatible devices e.g.: All* (must have internal GPS) -->
 
|devicecompatability= <!-- Compatible devices e.g.: All* (must have internal GPS) -->
|dependencies= <!-- Any other/external dependencies e.g.: Google Maps Api v1.0 -->  
+
|dependencies= <!-- Any other/external dependencies e.g.: Google Maps Api v1.0 -->
|signing=<!-- Signing requirements - empty or one of: Self-Signed, DevCert, Manufacturer -->
+
|signing= <!-- Signing requirements - empty or one of: Self-Signed, DevCert, Manufacturer -->
|capabilities= <!-- Capabilities required by the article/code example (e.g. Location, NetworkServices. -->
+
|capabilities= ID_CAP_WALLET, ID_CAP_WALLET_PAYMENTINSTRUMENTS, ID_CAP_WALLET_SECUREELEMENT
 
|keywords= Wallet, Payment, Face Recognition
 
|keywords= Wallet, Payment, Face Recognition
 
|language= <!-- Language category code for non-English topics - e.g. Lang-Chinese -->
 
|language= <!-- Language category code for non-English topics - e.g. Lang-Chinese -->
 
|translated-by= <!-- [[User:XXXX]] -->
 
|translated-by= <!-- [[User:XXXX]] -->
|translated-from-title= <!-- Title only -->  
+
|translated-from-title= <!-- Title only -->
 
|translated-from-id= <!-- Id of translated revision -->
 
|translated-from-id= <!-- Id of translated revision -->
|review-by=<!-- After re-review: [[User:username]] -->
+
|review-by= <!-- After re-review: [[User:username]] -->
 
|review-timestamp= <!-- After re-review: YYYYMMDD -->
 
|review-timestamp= <!-- After re-review: YYYYMMDD -->
 
|update-by= <!-- After significant update: [[User:username]]-->
 
|update-by= <!-- After significant update: [[User:username]]-->
Line 22: Line 22:
 
|creationdate= 20121211
 
|creationdate= 20121211
 
|author= [[User:galazzo]]
 
|author= [[User:galazzo]]
 +
}}
 +
{{SeeAlso|
 +
* [[Face Recognition using 2D Fast Fourier Transform]]
 +
* [[http://msdn.microsoft.com/en-us/library/windowsphone/develop/jj206987%28v=vs.105%29.aspx Auto-launching apps using file and URI associations for Windows Phone 8]]
 +
* [[http://msdn.microsoft.com/en-US/library/Microsoft.Phone.Wallet.aspx Microsoft.Phone.Wallet Namespace]]
 +
* [[http://msdn.microsoft.com/en-us/library/windowsphone/develop/jj207001%28v=vs.105%29.aspx How to use the add Wallet item task for Windows Phone]]
 +
* [[http://msdn.microsoft.com/en-US/library/windowsphone/develop/microsoft.phone.secureelement.secureelementsession(v=vs.105).aspx SecureElementSession]]
 +
* [http://www.heraldsun.com.au/news/national/paypal-set-to-launch-facial-recognition-for-ins-store-purchasing-via-smartphones/story-fndo471r-1226516195461 PayPal set to launch facial-recognition for ins-store purchasing via smartphones ]
 
}}
 
}}
  
 
== Introduction ==
 
== Introduction ==
 
A new interesting feature of Windows Phone 8 is certainly the Wallet and Deals APIs which allows users to do the following:
 
A new interesting feature of Windows Phone 8 is certainly the Wallet and Deals APIs which allows users to do the following:
 
 
* Collect coupons, credit cards, memberships, loyalty cards, and more in one place.
 
* Collect coupons, credit cards, memberships, loyalty cards, and more in one place.
 
* Manage the payment instruments that they use in the app and music store.
 
* Manage the payment instruments that they use in the app and music store.
 
* Link items in the Wallet to apps on their phone.
 
* Link items in the Wallet to apps on their phone.
* Make contactless transactions, using Near-Field Communication (NFC), in some markets.<br />
+
* Make contact-less transactions, using Near-Field Communication (NFC), in some markets.
  
The Wallet API offers full programmatic access to the Wallet. It allows you to create, read, update, and delete Wallet items to implement {{Icode|Deals}} or {{Icode|Payments}}.<br />
+
The Wallet API offers full programmatic access to the Wallet. It allows you to create, read, update, and delete Wallet items to implement {{Icode|Deals}} or {{Icode|Payments}}.
Deals are useful to manage coupons, loyalty cards,  Payment instrument manage a balance on an account maintained by your backend. <br />
+
 
The sample application we will create first acquires the face sample to compare and store it int oa file. Throught {{Icode|RootFrame.UriMapper}} rewriting when the app is called from Wallet before to process payment is required to compare your face with the one saved into the system. If there is a match payment process can continue else is stopped.
+
Deals are useful to manage coupons, loyalty cards,  Payment instrument manage a balance on an account maintained by your back-end.
 +
 
 +
The sample application we will create first acquires the face sample to compare and store it into a file. Through {{Icode|RootFrame.UriMapper}} rewriting when the app is called from Wallet before to process payment is required to compare your face with the one saved into the system. If there is a match payment process can continue else is stopped.
  
 
== Required capabilities ==
 
== Required capabilities ==
In order to use Wallet API in Windows Phone 8 you need to specify in the WMAppManifest.xml the following reqiored capabilities otherwise your app might not work correctly or it might fail the process of submission to the Store.<br />
+
In order to use Wallet API in Windows Phone 8 you need to specify in the '''WMAppManifest.xml''' the following required capabilities otherwise your app might not work correctly or it might fail the process of submission to the Store.
 +
 
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
 
! Capability !! API that requires this capability
 
! Capability !! API that requires this capability
 
|-
 
|-
| {{Icode|ID_CAP_WALLET}} || Required for all Wallet API, which is anything in Microsoft.Phone.Wallet or Microsoft.Phone.SecureElement.
+
| {{Icode|ID_CAP_WALLET}} || Required for all Wallet API, which is anything in {{Icode|Microsoft.Phone.Wallet}} or {{Icode|Microsoft.Phone.SecureElement}}.
 
|-
 
|-
| {{Icode|ID_CAP_WALLET_PAYMENTINSTRUMENTS}} || Required for PaymentInstrument and OnlinePaymentInstrument.
+
| {{Icode|ID_CAP_WALLET_PAYMENTINSTRUMENTS}} || Required for {{Icode|PaymentInstrument}} and {{Icode|OnlinePaymentInstrument}}.
 
|-
 
|-
| {{Icode|ID_CAP_WALLET_SECUREELEMENT}} || Required for SecureElementSession, SecureElementChannel and SecureElementReader.
+
| {{Icode|ID_CAP_WALLET_SECUREELEMENT}} || Required for {{Icode|SecureElementSession}}, {{Icode|SecureElementChannel}} and {{Icode|SecureElementReader}}.
 
|}
 
|}
  
{{Warning|To deploy or submit an app that uses ID_CAP_WALLET_SECUREELEMENT or ID_CAP_WALLET_PAYMENTINSTRUMENTS, you must request special permissions and have that permission applied to your developer account.}}
+
{{Warning|To deploy or submit an app that uses {{Icode|ID_CAP_WALLET_SECUREELEMENT}} or {{Icode|ID_CAP_WALLET_PAYMENTINSTRUMENTS}}, you must request special permissions and have that permission applied to your developer account. As reported by [[http://msdn.microsoft.com/en-US/library/windowsphone/develop/microsoft.phone.secureelement.secureelementsession(v=vs.105).aspx documentation]] contact the [[https://dev.windowsphone.com/en-us/support developer support]]}}
  
== Creating the UI ==
+
== Creating the UI ==
 
The following code create the object to display the camera video flow and the snap button to save the captured image, compute or save it.
 
The following code create the object to display the camera video flow and the snap button to save the captured image, compute or save it.
Besides MainPage.xml the same code is needed for the secondary screen called '''Wallet.xaml''' we need to create to be opened when the application is called from the wallet.
+
Besides '''MainPage.xml''' the same code is needed for the secondary screen called '''Wallet.xaml''' we need to create to be opened when the application is called from the wallet.
 
<code xml>         
 
<code xml>         
        <Grid x:Name="ContentPanel" Grid.Row="1" Margin="12,0,12,0">
+
<Grid x:Name="ContentPanel" Grid.Row="1" Margin="12,0,12,0">
                <Grid.Background>
+
      <Grid.Background>
 
                 <VideoBrush x:Name="viewfinderBrush">
 
                 <VideoBrush x:Name="viewfinderBrush">
 
                     <VideoBrush.RelativeTransform>
 
                     <VideoBrush.RelativeTransform>
Line 62: Line 72:
 
                     </VideoBrush.RelativeTransform>
 
                     </VideoBrush.RelativeTransform>
 
                 </VideoBrush>
 
                 </VideoBrush>
            </Grid.Background>
+
      </Grid.Background>
        </Grid>
+
</Grid>
        <Button Content="Snap Picture" Click="SaveImage" Margin="0,624,0,0" Grid.Row="1" />
+
<Button Content="Snap Picture" Click="SaveImage" Margin="0,624,0,0" Grid.Row="1" />
        <!--Application progress bar.-->
+
<!--Application progress bar.-->
        <ProgressBar Height="10" Name="progressBar" Width="460" Visibility="Collapsed" Margin="12,365,8,286" Grid.Row="1" IsIndeterminate="True" />
+
<ProgressBar Height="10" Name="progressBar" Width="460" Visibility="Collapsed" Margin="12,365,8,286" Grid.Row="1" IsIndeterminate="True" />
 
</code>
 
</code>
  
 
=== Remapping URI ===
 
=== Remapping URI ===
Into {{Icode|App.xaml.cs}} add the following code:
+
In Windows Phone 8 we can use URI associations to automatically launch our app from another app launches with a specific file type or URI scheme. When launched, a deep link URI is used to send the file (a reference to the file) or URI to your app.
 +
We use URI scheme feature in order to open the {{Icode|Wallet.xaml}} when the app is called from the Wallet, MainPage.xaml else. We need to choose a parameter to intercept,{{Icode|PaymentInstrument}} in our sample, but you can choose your own name. So we need also to create a class that inherits UriMapperBase. The object created from this class will be delegated to manage page switching.
 +
 
 +
Into '''App.xaml.cs''' add the following code:
 
<code csharp>
 
<code csharp>
 
class MyAppUriMapper : UriMapperBase
 
class MyAppUriMapper : UriMapperBase
Line 77: Line 90:
 
         {
 
         {
 
             string tempUri = uri.ToString();
 
             string tempUri = uri.ToString();
 +
            // Check if the URI contains the parameter we choosed.
 +
            // If yes return a new URI corresponding to the .xaml we want to open when the App is called from another App
 +
            // else the uri is unchanged
 
             if (tempUri.Contains("PaymentInstrument"))
 
             if (tempUri.Contains("PaymentInstrument"))
             {
+
             {              
 
                 return new Uri("/Wallet.xaml", UriKind.Relative);
 
                 return new Uri("/Wallet.xaml", UriKind.Relative);
 
             }
 
             }
Line 95: Line 111:
 
         {
 
         {
 
               ..........
 
               ..........
 +
              // Override the application UriMapper
 
               RootFrame.UriMapper = new MyAppUriMapper();
 
               RootFrame.UriMapper = new MyAppUriMapper();
 
         }
 
         }
Line 101: Line 118:
  
 
== Camera ==
 
== Camera ==
In common with {{Icode|Wallet.xaml}}.
+
In common with '''Wallet.xaml'''.
 +
<code csharp>
 +
using Microsoft.Devices; // Needed for PhotoCamera
 +
using Microsoft.Xna.Framework.Media;
 +
 
 +
using System.Windows.Media.Imaging;
 +
using Microsoft.Phone; // Needed for PictureDecoder
 +
using System.IO;
 +
</code><br />
 +
 
 
<code csharp>
 
<code csharp>
 
public partial class MainPage : PhoneApplicationPage
 
public partial class MainPage : PhoneApplicationPage
Line 139: Line 165:
  
 
== Face Recognition Support ==
 
== Face Recognition Support ==
Add the following lines to {{Icode|MainPage.xaml}}, in common with {{Icode|Wallet.xaml}}.
+
For a deeper understanding of face recognition please refer to [[Face Recognition using 2D Fast Fourier Transform]].
 +
 
 +
* Download '''DSP.cs''' file from [[File:FaceWallet.zip|thumb| FaceWallet.zip]] and add it into your project.  '''DSP.cs''' provides a namespace called {{Icode|DSP}} and a class {{Icode|FourierTransform}} containing a set of functions to compute the FFT.
 +
 
 +
Include the namespace DSP:
 +
<code csharp>
 +
using DSP;
 +
</code>
 +
 
 +
Add the following lines to '''MainPage.xaml''', in common with '''Wallet.xaml'''.
 
<code csharp>
 
<code csharp>
 
public partial class MainPage : PhoneApplicationPage
 
public partial class MainPage : PhoneApplicationPage
Line 166: Line 201:
 
<code csharp>
 
<code csharp>
 
  void cam_CaptureImageAvailable(object sender, ContentReadyEventArgs e)
 
  void cam_CaptureImageAvailable(object sender, ContentReadyEventArgs e)
        {
+
{
 
             //Take JPEG stream and decode into a WriteableBitmap object
 
             //Take JPEG stream and decode into a WriteableBitmap object
 
             Deployment.Current.Dispatcher.BeginInvoke(delegate()
 
             Deployment.Current.Dispatcher.BeginInvoke(delegate()
Line 204: Line 239:
  
 
== Wallet:PaymentInstruments ==
 
== Wallet:PaymentInstruments ==
 +
The following code is needed to register our application into the wallet in order to be able to call it from the Wallet. First we need to create a [http://msdn.microsoft.com/en-US/library/microsoft.phone.wallet.paymentinstrument.aspx {{Icode|PaymentInstruments}}] object whose methods are intuitive enough.
 +
Created the Item we use [http://msdn.microsoft.com/en-US/library/windowsphone/develop/microsoft.phone.tasks.addwalletitemtask%28v=vs.105%29.aspx AddWalletItemTask] allowing our application to launch the Wallet application.<br />
 +
The provided Wallet item is displayed to the user and the user can choose to add the item to his or her Wallet.
 +
 
Include the following namespaces:
 
Include the following namespaces:
 
<code csharp>
 
<code csharp>
Line 211: Line 250:
 
using Microsoft.Phone.Wallet;
 
using Microsoft.Phone.Wallet;
 
using System.Windows.Media;
 
using System.Windows.Media;
</code><br />
+
</code>
  
 
<code csharp>
 
<code csharp>
The following code is needed to register our application into the wallet in order to be able to call it from the Wallet.
+
private void Wallet_CreateNewPaymentInstrument()
private void Wallet_CreateNewPaymentInstrument()
+
 
{           
 
{           
 
             // Check if the object is already created
 
             // Check if the object is already created
 
             if ((Microsoft.Phone.Wallet.Wallet.FindItem("facedetectionwallet") as PaymentInstrument) == null)
 
             if ((Microsoft.Phone.Wallet.Wallet.FindItem("facedetectionwallet") as PaymentInstrument) == null)
 
             {
 
             {
 +
              // Create the PaymentInstruments object
 
                 var item = new PaymentInstrument()
 
                 var item = new PaymentInstrument()
 
                 {                 
 
                 {                 
Line 241: Line 280:
 
                     NavigationUri = new Uri("/mainpage.xaml?wallet=PaymentInstrument", UriKind.Relative)
 
                     NavigationUri = new Uri("/mainpage.xaml?wallet=PaymentInstrument", UriKind.Relative)
 
                 };
 
                 };
 +
                // Assign an ID we use to avoid duplication of our App into the Wallet
 
                 item.UpdateId("facedetectionwallet");
 
                 item.UpdateId("facedetectionwallet");
 +
 +
                // Create the task able to register into the Wallet the Item created above
 
                 AddWalletItemTask task = new AddWalletItemTask()
 
                 AddWalletItemTask task = new AddWalletItemTask()
 
                 {
 
                 {
 
                         Item = item
 
                         Item = item
 
                 };
 
                 };
 +
               
 +
                // The code to execute when the registration task is completed
 
                 task.Completed += (s, args) => MessageBox.Show(args.TaskResult.ToString());
 
                 task.Completed += (s, args) => MessageBox.Show(args.TaskResult.ToString());
 +
               
 
                 task.Show();
 
                 task.Show();
  
Line 263: Line 308:
 
</code>
 
</code>
  
[[Category:Windows Phone 8]]
 
 
== Wallet.xaml ==
 
== Wallet.xaml ==
This page is opened when the application is called from Wallet. Most of the code is in common with {{Icode|MainPage.xaml}} except {{Icode|cam_CaptureImageAvailable}} code and the lack of {{Icode| Wallet_CreateNewPaymentInstrument}} that is needed to register application into the wallet but unnecessary here.
+
This page is opened when the application is called from Wallet. Most of the code is in common with '''MainPage.xaml''' except {{Icode|cam_CaptureImageAvailable}} code and the lack of {{Icode|Wallet_CreateNewPaymentInstrument}} that is needed to register application into the wallet but unnecessary here.<br />
 +
The sample code stops after the recognition process and it's limited to display just a {{Icode|MessageBox}}. This is because from this point onwards you put your own implementation based on your needs. In the same way when recognition fails you can choose to simply  stop the process or start another action. An idea could be for example to send an alert email  after three attemps attaching the photo of the face is trying to access, assuming is an unauthorized person that wants to access to your data.
  
 
<code csharp>
 
<code csharp>
Line 359: Line 404:
 
                 if (mse < 55)
 
                 if (mse < 55)
 
                 {
 
                 {
 +
                  // There was a recognition. You can proceed with your payment implementation.
 
                     MessageBox.Show("You have been recognized!\nMSE:" + mse + "\nWe can proceed with payment");
 
                     MessageBox.Show("You have been recognized!\nMSE:" + mse + "\nWe can proceed with payment");
 
                 }
 
                 }
 
                 else
 
                 else
 
                 {
 
                 {
 +
                    // There wasn't a recognition. You can choose to simply stop the payment or start another process based on your needs
 
                     MessageBox.Show("You have not been recognized!\nMSE:"+mse+"\nWe can't proceed with payment.\nIf you are the correct people, please retry to shoot a new photo with different environment conditions.");
 
                     MessageBox.Show("You have not been recognized!\nMSE:"+mse+"\nWe can't proceed with payment.\nIf you are the correct people, please retry to shoot a new photo with different environment conditions.");
 
                 }
 
                 }
Line 381: Line 428:
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
! MainPage.xaml<br />Acquiring the sample image !! MainPage.xaml<br/>Updated the face into file !! Wallet.xaml<br/>Recognition process
+
! MainPage.xaml<br />Acquiring the sample image !! MainPage.xaml<br/>Updated the face into file !! How Application looks into Wallet !! Wallet.xaml<br/>Recognition process
 
|-
 
|-
| [[File:FaceWallet-01.png|350px]] || [[File:FaceWallet-02.png|350px]] || [[File:FaceWallet-03.png|350px]]
+
| [[File:FaceWallet-01.png|310x420px]] || [[File:FaceWallet-02.png|310x420px]] || [[File:Face-wallet-08.png|310x420px]] ||  [[File:FaceWallet-03.png|310x420px]]
 
|}
 
|}
 +
 +
{{Warning|Please be aware that recognition process is strongly influenced by the background, how the face is rotated or light conditions. When {{Icode|Skin Detection}} will be available we can count on better performance}}
 +
 +
== Summary ==
 +
 +
Although it is still possible to greatly improve the algorithm the results already achieved are very good. Anyway it's important to understand that results are strongly influenced by the background, how the face is rotated or light conditions when compared to the master image. Provided that there is no method completely safe, the proposed solution is intended as an additional method to the normal safety systems not necessarily as a substitute. The choice depends most on the security level you want to achieve regard to the data that you want to protect.
 +
 +
== Interesting links ==
 +
* [http://www.heraldsun.com.au/news/national/paypal-set-to-launch-facial-recognition-for-ins-store-purchasing-via-smartphones/story-fndo471r-1226516195461 PayPal set to launch facial-recognition for ins-store purchasing via smartphones]

Latest revision as of 11:13, 13 September 2013

This article shows how you can use face recognition to authorize a wallet payment. This is the second of a series of articles based on the Digital Signal Processing and focused on face recognition.

WP Metro Icon Multimedia.png
SignpostIcon XAML 40.png
WP Metro Icon WP8.png
Article Metadata
Code Example
Source file: File:FaceWallet.zip
Tested with
Devices(s): Nokia Lumia
Compatibility
Platform(s): Windows Phone 8
Windows Phone 8
Platform Security
Capabilities: ID_CAP_WALLET, ID_CAP_WALLET_PAYMENTINSTRUMENTS, ID_CAP_WALLET_SECUREELEMENT
Article
Keywords: Wallet, Payment, Face Recognition
Created: galazzo (11 Dec 2012)
Last edited: SB Dev (13 Sep 2013)

Contents

[edit] Introduction

A new interesting feature of Windows Phone 8 is certainly the Wallet and Deals APIs which allows users to do the following:

  • Collect coupons, credit cards, memberships, loyalty cards, and more in one place.
  • Manage the payment instruments that they use in the app and music store.
  • Link items in the Wallet to apps on their phone.
  • Make contact-less transactions, using Near-Field Communication (NFC), in some markets.

The Wallet API offers full programmatic access to the Wallet. It allows you to create, read, update, and delete Wallet items to implement Deals or Payments.

Deals are useful to manage coupons, loyalty cards, Payment instrument manage a balance on an account maintained by your back-end.

The sample application we will create first acquires the face sample to compare and store it into a file. Through RootFrame.UriMapper rewriting when the app is called from Wallet before to process payment is required to compare your face with the one saved into the system. If there is a match payment process can continue else is stopped.

[edit] Required capabilities

In order to use Wallet API in Windows Phone 8 you need to specify in the WMAppManifest.xml the following required capabilities otherwise your app might not work correctly or it might fail the process of submission to the Store.

Capability API that requires this capability
ID_CAP_WALLET Required for all Wallet API, which is anything in Microsoft.Phone.Wallet or Microsoft.Phone.SecureElement.
ID_CAP_WALLET_PAYMENTINSTRUMENTS Required for PaymentInstrument and OnlinePaymentInstrument.
ID_CAP_WALLET_SECUREELEMENT Required for SecureElementSession, SecureElementChannel and SecureElementReader.

Warning.pngWarning: To deploy or submit an app that uses ID_CAP_WALLET_SECUREELEMENT or ID_CAP_WALLET_PAYMENTINSTRUMENTS, you must request special permissions and have that permission applied to your developer account. As reported by [documentation] contact the [developer support]

[edit] Creating the UI

The following code create the object to display the camera video flow and the snap button to save the captured image, compute or save it. Besides MainPage.xml the same code is needed for the secondary screen called Wallet.xaml we need to create to be opened when the application is called from the wallet.

<Grid x:Name="ContentPanel" Grid.Row="1" Margin="12,0,12,0">
<Grid.Background>
<VideoBrush x:Name="viewfinderBrush">
<VideoBrush.RelativeTransform>
<CompositeTransform CenterY="0.5" CenterX="0.5" Rotation="-90"/>
</VideoBrush.RelativeTransform>
</VideoBrush>
</Grid.Background>
</Grid>
<Button Content="Snap Picture" Click="SaveImage" Margin="0,624,0,0" Grid.Row="1" />
<!--Application progress bar.-->
<ProgressBar Height="10" Name="progressBar" Width="460" Visibility="Collapsed" Margin="12,365,8,286" Grid.Row="1" IsIndeterminate="True" />

[edit] Remapping URI

In Windows Phone 8 we can use URI associations to automatically launch our app from another app launches with a specific file type or URI scheme. When launched, a deep link URI is used to send the file (a reference to the file) or URI to your app. We use URI scheme feature in order to open the Wallet.xaml when the app is called from the Wallet, MainPage.xaml else. We need to choose a parameter to intercept,PaymentInstrument in our sample, but you can choose your own name. So we need also to create a class that inherits UriMapperBase. The object created from this class will be delegated to manage page switching.

Into App.xaml.cs add the following code:

class MyAppUriMapper : UriMapperBase
{
public override Uri MapUri(Uri uri)
{
string tempUri = uri.ToString();
// Check if the URI contains the parameter we choosed.
// If yes return a new URI corresponding to the .xaml we want to open when the App is called from another App
// else the uri is unchanged
if (tempUri.Contains("PaymentInstrument"))
{
return new Uri("/Wallet.xaml", UriKind.Relative);
}
else
{
return uri;
}
}
}
 
public partial class App : Application
{
..........
 
public App()
{
..........
// Override the application UriMapper
RootFrame.UriMapper = new MyAppUriMapper();
}
}

[edit] Camera

In common with Wallet.xaml.

using Microsoft.Devices; // Needed for PhotoCamera
using Microsoft.Xna.Framework.Media;
 
using System.Windows.Media.Imaging;
using Microsoft.Phone; // Needed for PictureDecoder
using System.IO;

public partial class MainPage : PhoneApplicationPage
{
PhotoCamera cam;
MediaLibrary library = new MediaLibrary();
 
public MainPage()
{
InitializeComponent();
this.Loaded += Page_Loaded;
}
 
void Page_Loaded(object sender, RoutedEventArgs e)
{
if (PhotoCamera.IsCameraTypeSupported(CameraType.FrontFacing))
{
cam = new Microsoft.Devices.PhotoCamera(CameraType.FrontFacing);
cam.CaptureImageAvailable += cam_CaptureImageAvailable;
viewfinderBrush.SetSource(cam);
} else
if (PhotoCamera.IsCameraTypeSupported(CameraType.Primary))
{
cam = new Microsoft.Devices.PhotoCamera(CameraType.Primary);
 
cam.CaptureImageAvailable += cam_CaptureImageAvailable;
viewfinderBrush.SetSource(cam);
}
}
 
private void SaveImage(object sender, RoutedEventArgs e)
{
cam.CaptureImage();
}
}

[edit] Face Recognition Support

For a deeper understanding of face recognition please refer to Face Recognition using 2D Fast Fourier Transform.

  • Download DSP.cs file from File:FaceWallet.zip and add it into your project. DSP.cs provides a namespace called DSP and a class FourierTransform containing a set of functions to compute the FFT.

Include the namespace DSP:

using DSP;

Add the following lines to MainPage.xaml, in common with Wallet.xaml.

public partial class MainPage : PhoneApplicationPage
{
PhotoCamera cam;
MediaLibrary library = new MediaLibrary();
 
public static WriteableBitmap CapturedImage;
private static int W = 256;
private static int H = 256;
private static int matchSamples = 25;
 
private double[] compareSignal = new Double[matchSamples];
 
private Double[] pRealIn = new Double[W * H];
private Double[] pImagIn = new Double[W * H];
private Double[] pRealOut = new Double[W * H];
private Double[] pImagOut = new Double[W * H];
private Double[] pRealOut2 = new Double[W * H];
private Double[] pImagOut2 = new Double[W * H];
 
private const string faceFileName = "face.csv";
.......

 void cam_CaptureImageAvailable(object sender, ContentReadyEventArgs e)
{
//Take JPEG stream and decode into a WriteableBitmap object
Deployment.Current.Dispatcher.BeginInvoke(delegate()
{
MainPage.CapturedImage = PictureDecoder.DecodeJpeg(e.ImageStream, W, H);
 
//Collapse visibility on the progress bar once writeable bitmap is visible.
progressBar.Visibility = Visibility.Collapsed;
 
int[] pixel = MainPage.CapturedImage.Pixels;
 
int color = 0;
for (int y = 0; y < MainPage.CapturedImage.PixelHeight; y++)
{
for (int x = 0; x < MainPage.CapturedImage.PixelWidth; x++)
{
color = MainPage.CapturedImage.Pixels[x + (y * MainPage.CapturedImage.PixelWidth)];
pRealIn[x + (y * MainPage.CapturedImage.PixelWidth)] = DSP.Utilities.ColorToGray(color) & 0xFF;
 
}
}
 
Double[] signal;
 
System.Diagnostics.Debug.WriteLine("Using Fourier");
DSP.FourierTransform.Compute2D((uint)W, (uint)H, ref pRealIn, null, ref pRealOut, ref pImagOut, false);
signal = DSP.Utilities.triangularExtraction(ref pRealOut, (uint)W, (uint)H, (uint)matchSamples, 0);
 
DSP.Utilities.saveSignal(ref signal, faceFileName);
 
Wallet_CreateNewPaymentInstrument();
 
});
 
}

[edit] Wallet:PaymentInstruments

The following code is needed to register our application into the wallet in order to be able to call it from the Wallet. First we need to create a PaymentInstruments object whose methods are intuitive enough. Created the Item we use AddWalletItemTask allowing our application to launch the Wallet application.
The provided Wallet item is displayed to the user and the user can choose to add the item to his or her Wallet.

Include the following namespaces:

using System.Windows.Media.Imaging;
using Windows.System;
using Microsoft.Phone.Tasks;
using Microsoft.Phone.Wallet;
using System.Windows.Media;
private void Wallet_CreateNewPaymentInstrument()
{
// Check if the object is already created
if ((Microsoft.Phone.Wallet.Wallet.FindItem("facedetectionwallet") as PaymentInstrument) == null)
{
// Create the PaymentInstruments object
var item = new PaymentInstrument()
{
IssuerName = "sebastiano.galazzo@gmail.com",
CustomerName = "Sebastiano Galazzo",
ExpirationDate = DateTime.Now.AddDays(1),
PaymentInstrumentKinds = PaymentInstrumentKinds.Debit,
DisplayName = "Face Recognition Wallet",
Logo99x99 = GetBitmapSource("99x99.png"),
Logo336x336 = GetBitmapSource("336x336.png"),
Logo159x159 = GetBitmapSource("159x159.png"),
DisplayAvailableBalance = "50€?",
DisplayBalance = "100€?",
DisplayCreditLimit = "1000€",
DisplayAvailableCredit = "1000€",
BackgroundColor = Color.FromArgb(255, 70, 150, 250),
Nickname = "Bino",
Message = "Face recognition based wallet.",
MemberSince = DateTime.Now.AddYears(-1),
AccountNumber = "12345679",
NavigationUri = new Uri("/mainpage.xaml?wallet=PaymentInstrument", UriKind.Relative)
};
// Assign an ID we use to avoid duplication of our App into the Wallet
item.UpdateId("facedetectionwallet");
 
// Create the task able to register into the Wallet the Item created above
AddWalletItemTask task = new AddWalletItemTask()
{
Item = item
};
 
// The code to execute when the registration task is completed
task.Completed += (s, args) => MessageBox.Show(args.TaskResult.ToString());
 
task.Show();
 
} else {
MessageBox.Show("Face updated!");
}
Launcher.LaunchUriAsync(new Uri("wallet://", UriKind.RelativeOrAbsolute));
}
 
private BitmapSource GetBitmapSource(string url)
{
var bmp = new BitmapImage();
bmp.SetSource(Application.GetResourceStream(new Uri(url, UriKind.Relative)).Stream);
return bmp;
}

[edit] Wallet.xaml

This page is opened when the application is called from Wallet. Most of the code is in common with MainPage.xaml except cam_CaptureImageAvailable code and the lack of Wallet_CreateNewPaymentInstrument that is needed to register application into the wallet but unnecessary here.
The sample code stops after the recognition process and it's limited to display just a MessageBox. This is because from this point onwards you put your own implementation based on your needs. In the same way when recognition fails you can choose to simply stop the process or start another action. An idea could be for example to send an alert email after three attemps attaching the photo of the face is trying to access, assuming is an unauthorized person that wants to access to your data.

using DSP;
 
namespace FaceWallet
{
public partial class Wallet : PhoneApplicationPage
{
PhotoCamera cam;
MediaLibrary library = new MediaLibrary();
 
public static WriteableBitmap CapturedImage;
private static int W = 256;
private static int H = 256;
private static int matchSamples = 25;
 
private double[] compareSignal = new Double[matchSamples];
 
private Double[] pRealIn = new Double[W * H];
private Double[] pImagIn = new Double[W * H];
private Double[] pRealOut = new Double[W * H];
private Double[] pImagOut = new Double[W * H];
private Double[] pRealOut2 = new Double[W * H];
private Double[] pImagOut2 = new Double[W * H];
 
private const string faceFileName = "face.csv";
 
public Wallet()
{
InitializeComponent();
this.Loaded += Page_Loaded;
}
 
void Page_Loaded(object sender, RoutedEventArgs e)
{
if ((compareSignal = DSP.Utilities.loadSignal(faceFileName)) == null) { MessageBox.Show("Face to recognize has not been sampled.\nPlease procedd to sampling before continue."); return; }
 
if (PhotoCamera.IsCameraTypeSupported(CameraType.FrontFacing))
{
cam = new Microsoft.Devices.PhotoCamera(CameraType.FrontFacing);
cam.CaptureImageAvailable += cam_CaptureImageAvailable;
viewfinderBrush.SetSource(cam);
}
else
if (PhotoCamera.IsCameraTypeSupported(CameraType.Primary))
{
cam = new Microsoft.Devices.PhotoCamera(CameraType.Primary);
 
cam.CaptureImageAvailable += cam_CaptureImageAvailable;
viewfinderBrush.SetSource(cam);
}
}
 
void cam_CaptureImageAvailable(object sender, ContentReadyEventArgs e)
{
Deployment.Current.Dispatcher.BeginInvoke(delegate()
{
//Take JPEG stream and decode into a WriteableBitmap object
MainPage.CapturedImage = PictureDecoder.DecodeJpeg(e.ImageStream, W, H);
 
//Collapse visibility on the progress bar once writeable bitmap is visible.
progressBar.Visibility = Visibility.Collapsed;
 
int[] pixel = MainPage.CapturedImage.Pixels;
 
int color = 0;
for (int y = 0; y < MainPage.CapturedImage.PixelHeight; y++)
{
for (int x = 0; x < MainPage.CapturedImage.PixelWidth; x++)
{
color = MainPage.CapturedImage.Pixels[x + (y * MainPage.CapturedImage.PixelWidth)];
pRealIn[x + (y * MainPage.CapturedImage.PixelWidth)] = DSP.Utilities.ColorToGray(color) & 0xFF;
}
}
 
Double[] match;
double mse = 0;
 
System.Diagnostics.Debug.WriteLine("Using Fourier");
DSP.FourierTransform.Compute2D((uint)W, (uint)H, ref pRealIn, null, ref pRealOut, ref pImagOut, false);
match = DSP.Utilities.triangularExtraction(ref pRealOut, (uint)W, (uint)H, (uint)matchSamples, 0);
 
mse = DSP.Utilities.MSE(ref compareSignal, ref match, (int)matchSamples);
 
// Normalize Fourier result
mse /= 1000000000;
 
mse = Math.Round(mse);
 
System.Diagnostics.Debug.WriteLine("MSE:" + mse);
 
if (mse < 55)
{
// There was a recognition. You can proceed with your payment implementation.
MessageBox.Show("You have been recognized!\nMSE:" + mse + "\nWe can proceed with payment");
}
else
{
// There wasn't a recognition. You can choose to simply stop the payment or start another process based on your needs
MessageBox.Show("You have not been recognized!\nMSE:"+mse+"\nWe can't proceed with payment.\nIf you are the correct people, please retry to shoot a new photo with different environment conditions.");
}
 
});
}
 
private void SaveImage(object sender, RoutedEventArgs e)
{
cam.CaptureImage();
}
 
}
}

[edit] Screenshots

MainPage.xaml
Acquiring the sample image
MainPage.xaml
Updated the face into file
How Application looks into Wallet Wallet.xaml
Recognition process
FaceWallet-01.png FaceWallet-02.png Face-wallet-08.png FaceWallet-03.png

Warning.pngWarning: Please be aware that recognition process is strongly influenced by the background, how the face is rotated or light conditions. When Skin Detection will be available we can count on better performance

[edit] Summary

Although it is still possible to greatly improve the algorithm the results already achieved are very good. Anyway it's important to understand that results are strongly influenced by the background, how the face is rotated or light conditions when compared to the master image. Provided that there is no method completely safe, the proposed solution is intended as an additional method to the normal safety systems not necessarily as a substitute. The choice depends most on the security level you want to achieve regard to the data that you want to protect.

[edit] Interesting links

This page was last modified on 13 September 2013, at 11:13.
483 page views in the last 30 days.
×