×
Namespaces

Variants
Actions
(Difference between revisions)

Multi-touch handling using DirectX C++

From Nokia Developer Wiki
Jump to: navigation, search
hamishwillee (Talk | contribs)
m (Hamishwillee - Add Icode wiki markup)
hamishwillee (Talk | contribs)
m (Hamishwillee - Subedited/Reviewed)
Line 1: Line 1:
 
[[Category:UI on Windows Phone]][[Category:C++/CX]][[Category:DirectX]][[Category:Code Snippet]][[Category:Windows Phone 8]]
 
[[Category:UI on Windows Phone]][[Category:C++/CX]][[Category:DirectX]][[Category:Code Snippet]][[Category:Windows Phone 8]]
{{Abstract|This article explains how to handle multiple touches (multi-touch) from a C++ DirectX app (or WinRT component). }}  
+
{{Abstract|This article explains how to handle multiple touches (multi-touch) from a C++ DirectX app or WinRT component. }}  
  
 
{{ArticleMetaData <!-- v1.3 -->
 
{{ArticleMetaData <!-- v1.3 -->
Line 24: Line 24:
 
== Introduction ==
 
== Introduction ==
  
I am a C++ novice, but decided to dive in when I wanted to try out some really low-level coding.  I have written several XNA apps in C#, but since XNA is no longer supported, I had to decide if I wanted to try a package like SharpDX with C# or just bite the bullet and code everything in pure C++ using DirectX. I learn by copying samples and I was looking for ways to handle multiple touches in a DirectX C++ app.  Those handy manipulation events in C# are not available in C++.  Gestures are not supported for Windows Phone 8.0 in C++ even though they are supported in Windows Store apps(as of November 2013)
+
Developers programming in C# and XNA will be used to having a number of handy pointer manipulation and gesture events. Unfortunately in these sorts of handy events are not available in C++.  Gestures are not supported for Windows Phone 8.0 in C++ even though they are supported in Windows Store apps (as of November 2013).
  
In DirectX/C++, we have the {{Icode|DrawingSurfaceManipulationHost}}'s {{Icode|PointerPressed}}, {{Icode|PointerMoved}}, and {{Icode|PointerReleased}} events to use... and as far as I can tell, that's it.
+
In DirectX/C++, the {{Icode|DrawingSurfaceManipulationHost}}'s {{Icode|PointerPressed}}, {{Icode|PointerMoved}}, and {{Icode|PointerReleased}} events are all that is available.  
  
<nowiki>***</nowiki> The code in this article works best with the default templates for a DirectX Windows Phone 8 app. <nowiki>***</nowiki>
+
This article explains how the events work, and how to get enough information in order to build gesture recognition into your app.
  
== PointerPressed, PointerMoved, and PointerReleased events:  How do they work? ==
+
{{Tip|The code in this article has been tested using the default templates for a DirectX Windows Phone 8 app. }}
  
It may seem obvious at first how they work because they seem similar to what you see in a C# environment, but through trial and error, I found them to be a little different.
+
{{Note|The author is a C# XNA programmer and C++ novice. The code here has been compiled by examining touch handling in a number of examples, and may not be coded/optimised as would be done by an experienced C++ developer.}}
  
The most important thing is that these events can be called '''more than once''' per frame.  If you touch the screen with one finger, the {{Icode|PointerPressed}} event will fire.  OK.  If you touch the screen with a second finger, the PointerPressed event will fire again.  OK.  Theoretically, if you touch two fingers to the screen at the same time, the {{Icode|PointerPressed}} event will fire TWICE in one frame.  This applies to all of the events... and it's going to make handling {{Icode|PointerMoved}} trickier than usual.
+
== How do the pointer events work? ==
  
When you handle these events, a {{Icode|PointerEventArgs}} argument contains a few things, one of which is a {{Icode|PointerPoint}} representing the current point that is being touched.  {{Icode|PointerPoint}} contains all the information we will need to perform calculations for translating, rotating, or scaling your model/camera/whatever.
+
The {{Icode|PointerPressed}}, {{Icode|PointerMoved}}, and {{Icode|PointerReleased}} events can be called ''more than once'' per frameA single finger touching the screen will cause the {{Icode|PointerPressed}} event to fire once: a second finger touching the screen will cause the {{Icode|PointerPressed}} event to fire again.  The fact that multiple events can be called for each of the events makes handling {{Icode|PointerMoved}} (in particular) non-trivial.
  
== PointerPoint ==
+
The events include a {{Icode|PointerEventArgs}} argument which contains a few items, the most important of which for our purposes is a {{Icode|PointerPoint}} representing the current point that is being touched.  {{Icode|PointerPoint}} contains all the information we will need to perform calculations for translating, rotating, or scaling your model/camera/whatever.
  
We're going to be using three properties within the {{Icode|PointerPoint}}:  {{Icode|FrameId}}, {{Icode|PointerId}}, and {{Icode|Position}}.
+
We're going to be using three properties within the {{Icode|PointerPoint}}:
 +
* {{Icode|FrameId}} contains an {{Icode|unsigned int}} that just counts up for each frame drawn. You can have more than one {{Icode|PointerPoint}} with the same {{Icode|FrameId}}.
 +
* {{Icode|PointerId}} is also an {{Icode|unsigned int}}, but it represents one of the touch points.  If you place your finger on the screen, it will have an {{Icode|PointerId}} of 1.  If you remove and then touch the screen again, the {{Icode|PointerId}} will be 2 now.  It increments for each new touch.  If you place two fingers on the screen, they will have sequential {{Icode|PointerId}}s.
 +
* {{Icode|Position}} contains your X and Y positions relative to screen resolution divided by the scale factor.  This means that 720P screens will actually have positions up to only 480x800.  WXGA screens are something like 480x768.
  
{{Icode|FrameId}} contains an {{Icode|unsigned int}} that just counts up for each frame drawn.  You can have more than one {{Icode|PointerPoint}} with the same {{Icode|FrameId}}.
 
  
{{Icode|PointerId}} is also an {{Icode|unsigned int}}, but it represents one of the touch points.  If you place your finger on the screen, it will have an {{Icode|PointerId}} of 1.  If you remove and then touch the screen again, the {{Icode|PointerId}} will be 2 now.  It increments for each new touch.  If you place two fingers on the screen, they will have sequential {{Icode|PointerId}}s.
+
== Implementation ==
 
+
Position contains your X and Y positions relative to screen resolution divided by the scale factor.  This means that 720P screens will actually have positions up to only 480x800.  WXGA screens are something like 480x768.
+
 
+
== The CODE ==
+
 
   
 
   
As a novice C++ programmer and someone who does not have any formal education in computer programming (other than a college course in Pascal programming), this code is not the fastest, nor does it save the most memory. But it works!
 
 
 
First of all, we need to keep track of our {{Icode|touchId}}s and positions, so I came up with the idea to store the {{Icode|PointerPoint}} in a {{Icode|std::unordered_map}} using the {{Icode|TouchId}} as the key.  Most implementations of multi-touch handling require you to know the previous touch positions, so I'm going to create a second {{Icode|std::unordered_map}} to hold the previous {{Icode|PointerPoint}} for a particular {{Icode|TouchId}}.  Stick this in the header that contains the event handler declarations.
 
First of all, we need to keep track of our {{Icode|touchId}}s and positions, so I came up with the idea to store the {{Icode|PointerPoint}} in a {{Icode|std::unordered_map}} using the {{Icode|TouchId}} as the key.  Most implementations of multi-touch handling require you to know the previous touch positions, so I'm going to create a second {{Icode|std::unordered_map}} to hold the previous {{Icode|PointerPoint}} for a particular {{Icode|TouchId}}.  Stick this in the header that contains the event handler declarations.
  
Line 61: Line 57:
 
</code>
 
</code>
  
Now, the easy part.  When you touch the screen and {{Icode|PointerPressed}} is called, store the {{Icode|PointerPoint}} in the {{Icode|m_pointerIds}} map using its {{Icode|PointerId}} as a key.  Probably superfluous, but I also stored it in the {{Icode|m_oldPoints}} map.  If you release your finger and the {{Icode|PointerRelease}} event is called, remove that {{Icode|PointerId}} from both of the maps.  Code below:
 
  
 +
When the screen is touched and {{Icode|PointerPressed}} is called, store the {{Icode|PointerPoint}} in the {{Icode|m_pointerIds}} map using its {{Icode|PointerId}} as a key (the code also stores the point in the {{Icode|m_oldPoints}} map, but this may be superfluous).  When the finger is released the {{Icode|PointerRelease}} event is called - remove that {{Icode|PointerId}} from both of the maps.
 
<code cpp>
 
<code cpp>
 
void Direct3DBackground::OnPointerPressed(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
 
void Direct3DBackground::OnPointerPressed(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
Line 78: Line 74:
 
</code>
 
</code>
  
The tricky part is handling {{Icode|PointerMoved}}.  If you have more than one touch, you need to make sure that every touch has called {{Icode|PointerMoved}} ''before'' you do any calculations using the currently stored positions.  If you don't, you will get some strange effects.  It was most pronounced when I tried to scale using a pinch motion.  The calculations were correct, but doing them twice (one for each {{Icode|PointerMoved}} event) somehow cancelled out my calculated scale factor. 
 
  
Here is the code:
+
Handling {{Icode|PointerMoved}} is more complicated.  If there is more than one touch, you need to make sure that every touch has called {{Icode|PointerMoved}} ''before'' doing any calculations using the currently stored positions. 
 +
{{Note|If you do perform the calculations early you will get some strange effects.  In one example of trying to scale using a pinch motion the  calculations were correct, but doing them for each {{Icode|PointerMoved}} event cancelled out the calculated scale factor. }}
 +
 
 
<code cpp>
 
<code cpp>
 
void Direct3DBackground::OnPointerMoved(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
 
void Direct3DBackground::OnPointerMoved(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
Line 120: Line 117:
 
</code>
 
</code>
  
The way this works is we first check to see how many touches there are.  If two (and you can extrapolate all of this to more than two), we're going to do a couple of things first.
+
The way this works is we first check to see how many touches there are.  If two (and you can extrapolate all of this to more than two), we're going to do a couple of things first:
 
# Find the {{Icode|PointerId}} of the other touch.  We know the current one's Id because it's in the event argument.
 
# Find the {{Icode|PointerId}} of the other touch.  We know the current one's Id because it's in the event argument.
 
# Move the current touch's stored {{Icode|PointerPoint}} in {{Icode|m_pointerIds}} to {{Icode|m_oldPoints}}.   
 
# Move the current touch's stored {{Icode|PointerPoint}} in {{Icode|m_pointerIds}} to {{Icode|m_oldPoints}}.   
Line 130: Line 127:
 
If they are NOT the same, then you do nothing and wait for the next call.
 
If they are NOT the same, then you do nothing and wait for the next call.
  
I hope this helps!
+
== Summary ==
 +
 
 +
This article has explained how to events work in a C++ app, and how to get enough information in order to build gesture recognition into your app. I hope it helps!

Revision as of 07:31, 19 November 2013

This article explains how to handle multiple touches (multi-touch) from a C++ DirectX app or WinRT component.

WP Metro Icon UI.png
WP Metro Icon DirectX.png
WP Metro Icon WP8.png
Article Metadata
Tested with
SDK: Windows Phone 8.0 SDK
Devices(s): Nokia Lumia 920
Compatibility
Platform(s):
Windows Phone 8
Article
Created: leemcpherson (14 Nov 2013)
Last edited: hamishwillee (19 Nov 2013)

Contents

Introduction

Developers programming in C# and XNA will be used to having a number of handy pointer manipulation and gesture events. Unfortunately in these sorts of handy events are not available in C++. Gestures are not supported for Windows Phone 8.0 in C++ even though they are supported in Windows Store apps (as of November 2013).

In DirectX/C++, the DrawingSurfaceManipulationHost's PointerPressed, PointerMoved, and PointerReleased events are all that is available.

This article explains how the events work, and how to get enough information in order to build gesture recognition into your app.

Tip.pngTip: The code in this article has been tested using the default templates for a DirectX Windows Phone 8 app.

Note.pngNote: The author is a C# XNA programmer and C++ novice. The code here has been compiled by examining touch handling in a number of examples, and may not be coded/optimised as would be done by an experienced C++ developer.

How do the pointer events work?

The PointerPressed, PointerMoved, and PointerReleased events can be called more than once per frame. A single finger touching the screen will cause the PointerPressed event to fire once: a second finger touching the screen will cause the PointerPressed event to fire again. The fact that multiple events can be called for each of the events makes handling PointerMoved (in particular) non-trivial.

The events include a PointerEventArgs argument which contains a few items, the most important of which for our purposes is a PointerPoint representing the current point that is being touched. PointerPoint contains all the information we will need to perform calculations for translating, rotating, or scaling your model/camera/whatever.

We're going to be using three properties within the PointerPoint:

  • FrameId contains an unsigned int that just counts up for each frame drawn. You can have more than one PointerPoint with the same FrameId.
  • PointerId is also an unsigned int, but it represents one of the touch points. If you place your finger on the screen, it will have an PointerId of 1. If you remove and then touch the screen again, the PointerId will be 2 now. It increments for each new touch. If you place two fingers on the screen, they will have sequential PointerIds.
  • Position contains your X and Y positions relative to screen resolution divided by the scale factor. This means that 720P screens will actually have positions up to only 480x800. WXGA screens are something like 480x768.


Implementation

First of all, we need to keep track of our touchIds and positions, so I came up with the idea to store the PointerPoint in a std::unordered_map using the TouchId as the key. Most implementations of multi-touch handling require you to know the previous touch positions, so I'm going to create a second std::unordered_map to hold the previous PointerPoint for a particular TouchId. Stick this in the header that contains the event handler declarations.

#include <unordered_map>
 
std::unordered_map<unsigned int, Windows::UI::Input::PointerPoint^> m_pointerIds;
std::unordered_map<unsigned int, Windows::UI::Input::PointerPoint^> m_oldPoints;


When the screen is touched and PointerPressed is called, store the PointerPoint in the m_pointerIds map using its PointerId as a key (the code also stores the point in the m_oldPoints map, but this may be superfluous). When the finger is released the PointerRelease event is called - remove that PointerId from both of the maps.

void Direct3DBackground::OnPointerPressed(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
{
m_pointerIds.emplace(args->CurrentPoint->PointerId, args->CurrentPoint);
m_oldPoints.emplace(args->CurrentPoint->PointerId, args->CurrentPoint);
}
 
 
void Direct3DBackground::OnPointerReleased(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
{
m_pointerIds.erase(args->CurrentPoint->PointerId);
m_oldPoints.erase(args->CurrentPoint->PointerId);
}


Handling PointerMoved is more complicated. If there is more than one touch, you need to make sure that every touch has called PointerMoved before doing any calculations using the currently stored positions.

Note.pngNote: If you do perform the calculations early you will get some strange effects. In one example of trying to scale using a pinch motion the calculations were correct, but doing them for each PointerMoved event cancelled out the calculated scale factor.

void Direct3DBackground::OnPointerMoved(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
{
if (m_pointerIds.size() == 1)
{
RotateWithOneFinger(args); // <- not shown in this article, you can see Microsoft's samples for this.
}
else if (m_pointerIds.size() == 2)
{
MoveWithTwoFingers(args);
}
}
 
void Direct3DBackground::MoveWithTwoFingers(PointerEventArgs^ args)
{
UINT changedPointId = args->CurrentPoint->PointerId;
UINT frameId = args->CurrentPoint->FrameId;
 
UINT otherPointId;
for (auto it = m_pointerIds.begin(); it != m_pointerIds.end(); ++it)
{
if (it->first != changedPointId)
{
otherPointId = it->first;
break;
}
}
m_oldPoints[changedPointId] = m_pointerIds[changedPointId];
m_pointerIds[changedPointId] = args->CurrentPoint;
 
if (m_pointerIds[otherPointId]->FrameId == frameId)
{
//the other point has been updated already and we are on the update of the 2nd point... store it in memory and do calculations
// IF NOT TRUE then
//the first point is being updated, we need to wait for the 2nd point to be updated... so just store it in memory for now
}
}

The way this works is we first check to see how many touches there are. If two (and you can extrapolate all of this to more than two), we're going to do a couple of things first:

  1. Find the PointerId of the other touch. We know the current one's Id because it's in the event argument.
  2. Move the current touch's stored PointerPoint in m_pointerIds to m_oldPoints.
  3. Copy the current touch's new PointerPoint (from args) to m_pointerIds
  4. Check to see if the other touch's PointerPoint stored in m_pointerIds has the same FrameId as the current touch's FrameId.

That last step is critical. If they are the same FrameId, it means they have both been updated now and you can use the values stored in m_pointerIds to do your calculations.

If they are NOT the same, then you do nothing and wait for the next call.

Summary

This article has explained how to events work in a C++ app, and how to get enough information in order to build gesture recognition into your app. I hope it helps!

221 page views in the last 30 days.

Was this page helpful?

Your feedback about this content is important. Let us know what you think.

 

Thank you!

We appreciate your feedback.

×