×
Namespaces

Variants
Actions
(Difference between revisions)

Multi-touch handling using DirectX C++

From Nokia Developer Wiki
Jump to: navigation, search
hamishwillee (Talk | contribs)
m (Hamishwillee - Bot update - Fix metadata)
hamishwillee (Talk | contribs)
m (Hamishwillee - Add Icode wiki markup)
Line 26: Line 26:
 
I am a C++ novice, but decided to dive in when I wanted to try out some really low-level coding.  I have written several XNA apps in C#, but since XNA is no longer supported, I had to decide if I wanted to try a package like SharpDX with C# or just bite the bullet and code everything in pure C++ using DirectX.  I learn by copying samples and I was looking for ways to handle multiple touches in a DirectX C++ app.  Those handy manipulation events in C# are not available in C++.  Gestures are not supported for Windows Phone 8.0 in C++ even though they are supported in Windows Store apps.  (as of November 2013)   
 
I am a C++ novice, but decided to dive in when I wanted to try out some really low-level coding.  I have written several XNA apps in C#, but since XNA is no longer supported, I had to decide if I wanted to try a package like SharpDX with C# or just bite the bullet and code everything in pure C++ using DirectX.  I learn by copying samples and I was looking for ways to handle multiple touches in a DirectX C++ app.  Those handy manipulation events in C# are not available in C++.  Gestures are not supported for Windows Phone 8.0 in C++ even though they are supported in Windows Store apps.  (as of November 2013)   
  
In DirectX/C++, we have the DrawingSurfaceManipulationHost's PointerPressed, PointerMoved, and PointerReleased events to use... and as far as I can tell, that's it.   
+
In DirectX/C++, we have the {{Icode|DrawingSurfaceManipulationHost}}'s {{Icode|PointerPressed}}, {{Icode|PointerMoved}}, and {{Icode|PointerReleased}} events to use... and as far as I can tell, that's it.   
  
 
<nowiki>***</nowiki> The code in this article works best with the default templates for a DirectX Windows Phone 8 app. <nowiki>***</nowiki>
 
<nowiki>***</nowiki> The code in this article works best with the default templates for a DirectX Windows Phone 8 app. <nowiki>***</nowiki>
Line 34: Line 34:
 
It may seem obvious at first how they work because they seem similar to what you see in a C# environment, but through trial and error, I found them to be a little different.
 
It may seem obvious at first how they work because they seem similar to what you see in a C# environment, but through trial and error, I found them to be a little different.
  
The most important thing is that these events can be called '''more than once''' per frame.  If you touch the screen with one finger, the PointerPressed event will fire.  OK.  If you touch the screen with a second finger, the PointerPressed event will fire again.  OK.  Theoretically, if you touch two fingers to the screen at the same time, the PointerPressed event will fire TWICE in one frame.  This applies to all of the events... and it's going to make handling PointerMoved trickier than usual.
+
The most important thing is that these events can be called '''more than once''' per frame.  If you touch the screen with one finger, the {{Icode|PointerPressed}} event will fire.  OK.  If you touch the screen with a second finger, the PointerPressed event will fire again.  OK.  Theoretically, if you touch two fingers to the screen at the same time, the {{Icode|PointerPressed}} event will fire TWICE in one frame.  This applies to all of the events... and it's going to make handling {{Icode|PointerMoved}} trickier than usual.
  
When you handle these events, a PointerEventArgs argument contains a few things, one of which is a PointerPoint representing the current point that is being touched.  PointerPoint contains all the information we will need to perform calculations for translating, rotating, or scaling your model/camera/whatever.
+
When you handle these events, a {{Icode|PointerEventArgs}} argument contains a few things, one of which is a {{Icode|PointerPoint}} representing the current point that is being touched.  {{Icode|PointerPoint}} contains all the information we will need to perform calculations for translating, rotating, or scaling your model/camera/whatever.
  
 
== PointerPoint ==
 
== PointerPoint ==
  
We're going to be using three properties within the PointerPoint:  FrameId, PointerId, and Position.
+
We're going to be using three properties within the {{Icode|PointerPoint}}{{Icode|FrameId}}, {{Icode|PointerId}}, and {{Icode|Position}}.
  
FrameId contains an unsigned int that just counts up for each frame drawn.  You can have more than one PointerPoint with the same FrameId.
+
{{Icode|FrameId}} contains an {{Icode|unsigned int}} that just counts up for each frame drawn.  You can have more than one {{Icode|PointerPoint}} with the same {{Icode|FrameId}}.
  
PointerId is also an unsigned int, but it represents one of the touch points.  If you place your finger on the screen, it will have an PointerId of 1.  If you remove and then touch the screen again, the PointerId will be 2 now.  It increments for each new touch.  If you place two fingers on the screen, they will have sequential PointerIds.
+
{{Icode|PointerId}} is also an {{Icode|unsigned int}}, but it represents one of the touch points.  If you place your finger on the screen, it will have an {{Icode|PointerId}} of 1.  If you remove and then touch the screen again, the {{Icode|PointerId}} will be 2 now.  It increments for each new touch.  If you place two fingers on the screen, they will have sequential {{Icode|PointerId}}s.
  
 
Position contains your X and Y positions relative to screen resolution divided by the scale factor.  This means that 720P screens will actually have positions up to only 480x800.  WXGA screens are something like 480x768.
 
Position contains your X and Y positions relative to screen resolution divided by the scale factor.  This means that 720P screens will actually have positions up to only 480x800.  WXGA screens are something like 480x768.
Line 52: Line 52:
 
As a novice C++ programmer and someone who does not have any formal education in computer programming (other than a college course in Pascal programming), this code is not the fastest, nor does it save the most memory. But it works!
 
As a novice C++ programmer and someone who does not have any formal education in computer programming (other than a college course in Pascal programming), this code is not the fastest, nor does it save the most memory. But it works!
  
First of all, we need to keep track of our touchIds and positions, so I came up with the idea to store the PointerPoint in a std::unordered_map using the TouchId as the key.  Most implementations of multi-touch handling require you to know the previous touch positions, so I'm going to create a second std::unordered_map to hold the previous PointerPoint for a particular TouchId.  Stick this in the header that contains the event handler declarations.
+
First of all, we need to keep track of our {{Icode|touchId}}s and positions, so I came up with the idea to store the {{Icode|PointerPoint}} in a {{Icode|std::unordered_map}} using the {{Icode|TouchId}} as the key.  Most implementations of multi-touch handling require you to know the previous touch positions, so I'm going to create a second {{Icode|std::unordered_map}} to hold the previous {{Icode|PointerPoint}} for a particular {{Icode|TouchId}}.  Stick this in the header that contains the event handler declarations.
  
 
<code cpp>
 
<code cpp>
Line 61: Line 61:
 
</code>
 
</code>
  
Now, the easy part.  When you touch the screen and PointerPressed is called, store the PointerPoint in the m_pointerIds map using its PointerId as a key.  Probably superfluous, but I also stored it in the m_oldPoints map.  If you release your finger and the PointerRelease event is called, remove that PointerId from both of the maps.  Code below:
+
Now, the easy part.  When you touch the screen and {{Icode|PointerPressed}} is called, store the {{Icode|PointerPoint}} in the {{Icode|m_pointerIds}} map using its {{Icode|PointerId}} as a key.  Probably superfluous, but I also stored it in the {{Icode|m_oldPoints}} map.  If you release your finger and the {{Icode|PointerRelease}} event is called, remove that {{Icode|PointerId}} from both of the maps.  Code below:
  
 
<code cpp>
 
<code cpp>
Line 78: Line 78:
 
</code>
 
</code>
  
The tricky part is handling PointerMoved.  If you have more than one touch, you need to make sure that every touch has called PointerMoved ''before'' you do any calculations using the currently stored positions.  If you don't, you will get some strange effects.  It was most pronounced when I tried to scale using a pinch motion.  The calculations were correct, but doing them twice (one for each PointerMoved event) somehow cancelled out my calculated scale factor.  Anwyays, here is the code.   
+
The tricky part is handling {{Icode|PointerMoved}}.  If you have more than one touch, you need to make sure that every touch has called {{Icode|PointerMoved}} ''before'' you do any calculations using the currently stored positions.  If you don't, you will get some strange effects.  It was most pronounced when I tried to scale using a pinch motion.  The calculations were correct, but doing them twice (one for each {{Icode|PointerMoved}} event) somehow cancelled out my calculated scale factor.   
  
 +
Here is the code:
 
<code cpp>
 
<code cpp>
 
void Direct3DBackground::OnPointerMoved(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
 
void Direct3DBackground::OnPointerMoved(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
Line 120: Line 121:
  
 
The way this works is we first check to see how many touches there are.  If two (and you can extrapolate all of this to more than two), we're going to do a couple of things first.
 
The way this works is we first check to see how many touches there are.  If two (and you can extrapolate all of this to more than two), we're going to do a couple of things first.
# Find the PointerId of the other touch.  We know the current one's Id because it's in the event argument.
+
# Find the {{Icode|PointerId}} of the other touch.  We know the current one's Id because it's in the event argument.
# Move the current touch's stored PointerPoint in m_pointerIds to m_oldPoints.   
+
# Move the current touch's stored {{Icode|PointerPoint}} in {{Icode|m_pointerIds}} to {{Icode|m_oldPoints}}.   
# Copy the current touch's new PointerPoint (from args) to m_pointerIds
+
# Copy the current touch's new {{Icode|PointerPoint}} (from args) to {{Icode|m_pointerIds}}
# '''Check to see if the other touch's PointerPoint stored in m_pointerIds has the same FrameId as the current touch's FrameId.'''   
+
# '''Check to see if the other touch's {{Icode|PointerPoint}} stored in {{Icode|m_pointerIds}} has the same {{Icode|FrameId}} as the current touch's {{Icode|FrameId}}.'''   
  
That last step is critical.  If they are the same FrameId, it means they have both been updated now and you can use the values stored in m_pointerIds to do your calculations.
+
That last step is critical.  If they are the same {{Icode|FrameId}}, it means they have both been updated now and you can use the values stored in {{Icode|m_pointerIds}} to do your calculations.
  
 
If they are NOT the same, then you do nothing and wait for the next call.
 
If they are NOT the same, then you do nothing and wait for the next call.
  
 
I hope this helps!
 
I hope this helps!
 
 
 
{{VersionHint}}
 

Revision as of 07:04, 19 November 2013

This article explains how to handle multiple touches (multi-touch) from a C++ DirectX app (or WinRT component).

WP Metro Icon UI.png
WP Metro Icon DirectX.png
WP Metro Icon WP8.png
Article Metadata
Tested with
SDK: Windows Phone 8.0 SDK
Devices(s): Nokia Lumia 920
Compatibility
Platform(s):
Windows Phone 8
Article
Created: leemcpherson (14 Nov 2013)
Last edited: hamishwillee (19 Nov 2013)

Contents

Introduction

I am a C++ novice, but decided to dive in when I wanted to try out some really low-level coding. I have written several XNA apps in C#, but since XNA is no longer supported, I had to decide if I wanted to try a package like SharpDX with C# or just bite the bullet and code everything in pure C++ using DirectX. I learn by copying samples and I was looking for ways to handle multiple touches in a DirectX C++ app. Those handy manipulation events in C# are not available in C++. Gestures are not supported for Windows Phone 8.0 in C++ even though they are supported in Windows Store apps. (as of November 2013)

In DirectX/C++, we have the DrawingSurfaceManipulationHost's PointerPressed, PointerMoved, and PointerReleased events to use... and as far as I can tell, that's it.

*** The code in this article works best with the default templates for a DirectX Windows Phone 8 app. ***

PointerPressed, PointerMoved, and PointerReleased events: How do they work?

It may seem obvious at first how they work because they seem similar to what you see in a C# environment, but through trial and error, I found them to be a little different.

The most important thing is that these events can be called more than once per frame. If you touch the screen with one finger, the PointerPressed event will fire. OK. If you touch the screen with a second finger, the PointerPressed event will fire again. OK. Theoretically, if you touch two fingers to the screen at the same time, the PointerPressed event will fire TWICE in one frame. This applies to all of the events... and it's going to make handling PointerMoved trickier than usual.

When you handle these events, a PointerEventArgs argument contains a few things, one of which is a PointerPoint representing the current point that is being touched. PointerPoint contains all the information we will need to perform calculations for translating, rotating, or scaling your model/camera/whatever.

PointerPoint

We're going to be using three properties within the PointerPoint: FrameId, PointerId, and Position.

FrameId contains an unsigned int that just counts up for each frame drawn. You can have more than one PointerPoint with the same FrameId.

PointerId is also an unsigned int, but it represents one of the touch points. If you place your finger on the screen, it will have an PointerId of 1. If you remove and then touch the screen again, the PointerId will be 2 now. It increments for each new touch. If you place two fingers on the screen, they will have sequential PointerIds.

Position contains your X and Y positions relative to screen resolution divided by the scale factor. This means that 720P screens will actually have positions up to only 480x800. WXGA screens are something like 480x768.

The CODE

As a novice C++ programmer and someone who does not have any formal education in computer programming (other than a college course in Pascal programming), this code is not the fastest, nor does it save the most memory. But it works!

First of all, we need to keep track of our touchIds and positions, so I came up with the idea to store the PointerPoint in a std::unordered_map using the TouchId as the key. Most implementations of multi-touch handling require you to know the previous touch positions, so I'm going to create a second std::unordered_map to hold the previous PointerPoint for a particular TouchId. Stick this in the header that contains the event handler declarations.

#include <unordered_map>
 
std::unordered_map<unsigned int, Windows::UI::Input::PointerPoint^> m_pointerIds;
std::unordered_map<unsigned int, Windows::UI::Input::PointerPoint^> m_oldPoints;

Now, the easy part. When you touch the screen and PointerPressed is called, store the PointerPoint in the m_pointerIds map using its PointerId as a key. Probably superfluous, but I also stored it in the m_oldPoints map. If you release your finger and the PointerRelease event is called, remove that PointerId from both of the maps. Code below:

void Direct3DBackground::OnPointerPressed(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
{
m_pointerIds.emplace(args->CurrentPoint->PointerId, args->CurrentPoint);
m_oldPoints.emplace(args->CurrentPoint->PointerId, args->CurrentPoint);
}
 
 
void Direct3DBackground::OnPointerReleased(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
{
m_pointerIds.erase(args->CurrentPoint->PointerId);
m_oldPoints.erase(args->CurrentPoint->PointerId);
}

The tricky part is handling PointerMoved. If you have more than one touch, you need to make sure that every touch has called PointerMoved before you do any calculations using the currently stored positions. If you don't, you will get some strange effects. It was most pronounced when I tried to scale using a pinch motion. The calculations were correct, but doing them twice (one for each PointerMoved event) somehow cancelled out my calculated scale factor.

Here is the code:

void Direct3DBackground::OnPointerMoved(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
{
if (m_pointerIds.size() == 1)
{
RotateWithOneFinger(args); // <- not shown in this article, you can see Microsoft's samples for this.
}
else if (m_pointerIds.size() == 2)
{
MoveWithTwoFingers(args);
}
}
 
void Direct3DBackground::MoveWithTwoFingers(PointerEventArgs^ args)
{
UINT changedPointId = args->CurrentPoint->PointerId;
UINT frameId = args->CurrentPoint->FrameId;
 
UINT otherPointId;
for (auto it = m_pointerIds.begin(); it != m_pointerIds.end(); ++it)
{
if (it->first != changedPointId)
{
otherPointId = it->first;
break;
}
}
m_oldPoints[changedPointId] = m_pointerIds[changedPointId];
m_pointerIds[changedPointId] = args->CurrentPoint;
 
if (m_pointerIds[otherPointId]->FrameId == frameId)
{
//the other point has been updated already and we are on the update of the 2nd point... store it in memory and do calculations
// IF NOT TRUE then
//the first point is being updated, we need to wait for the 2nd point to be updated... so just store it in memory for now
}
}

The way this works is we first check to see how many touches there are. If two (and you can extrapolate all of this to more than two), we're going to do a couple of things first.

  1. Find the PointerId of the other touch. We know the current one's Id because it's in the event argument.
  2. Move the current touch's stored PointerPoint in m_pointerIds to m_oldPoints.
  3. Copy the current touch's new PointerPoint (from args) to m_pointerIds
  4. Check to see if the other touch's PointerPoint stored in m_pointerIds has the same FrameId as the current touch's FrameId.

That last step is critical. If they are the same FrameId, it means they have both been updated now and you can use the values stored in m_pointerIds to do your calculations.

If they are NOT the same, then you do nothing and wait for the next call.

I hope this helps!

186 page views in the last 30 days.