Multi-touch handling using DirectX C++
This article explains how to handle multiple touches (multi-touch) from a C++ DirectX app or WinRT component.
Windows Phone 8
Developers programming in C# and XNA will be used to having a number of handy pointer manipulation and gesture events. Unfortunately in these sorts of handy events are not available in C++. Gestures are not supported for Windows Phone 8.0 in C++ even though they are supported in Windows Store apps (as of November 2013).
In DirectX/C++, the DrawingSurfaceManipulationHost]'s PointerPressed, PointerMoved, and PointerReleased events are all that is available.
This article explains how the events work, and how to get enough information in order to build gesture recognition into your app.
How do the pointer events work?
The PointerPressed, PointerMoved, and PointerReleased events can be called more than once per frame. A single finger touching the screen will cause the PointerPressed event to fire once: a second finger touching the screen will cause the PointerPressed event to fire again. The fact that multiple events can be called for each of the events makes handling PointerMoved (in particular) non-trivial.
The events include a PointerEventArgs argument which contains a few items, the most important of which for our purposes is a PointerPoint representing the current point that is being touched. PointerPoint contains all the information we will need to perform calculations for translating, rotating, or scaling your model/camera/whatever.
We're going to be using three properties within the PointerPoint:
- FrameId contains an unsigned int that just counts up for each frame drawn. You can have more than one PointerPoint with the same FrameId.
- PointerId is also an unsigned int, but it represents one of the touch points. If you place your finger on the screen, it will have an PointerId of 1. If you remove and then touch the screen again, the PointerId will be 2 now. It increments for each new touch. If you place two fingers on the screen, they will have sequential PointerIds.
- Position contains your X and Y positions relative to screen resolution divided by the scale factor. This means that 720P screens will actually have positions up to only 480x800. WXGA screens are something like 480x768.
First of all, we need to keep track of our touchIds and positions, so I came up with the idea to store the PointerPoint in a std::unordered_map using the TouchId as the key. Most implementations of multi-touch handling require you to know the previous touch positions, so I'm going to create a second std::unordered_map to hold the previous PointerPoint for a particular TouchId. Stick this in the header that contains the event handler declarations.
std::unordered_map<unsigned int, Windows::UI::Input::PointerPoint^> m_pointerIds;
std::unordered_map<unsigned int, Windows::UI::Input::PointerPoint^> m_oldPoints;
When the screen is touched and PointerPressed is called, store the PointerPoint in the m_pointerIds map using its PointerId as a key (the code also stores the point in the m_oldPoints map, but this may be superfluous). When the finger is released the PointerRelease event is called - remove that PointerId from both of the maps.
void Direct3DBackground::OnPointerPressed(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
void Direct3DBackground::OnPointerReleased(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
Handling PointerMoved is more complicated. If there is more than one touch, you need to make sure that every touch has called PointerMoved before doing any calculations using the currently stored positions.
void Direct3DBackground::OnPointerMoved(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
if (m_pointerIds.size() == 1)
RotateWithOneFinger(args); // <- not shown in this article, you can see Microsoft's samples for this.
else if (m_pointerIds.size() == 2)
void Direct3DBackground::MoveWithTwoFingers(PointerEventArgs^ args)
UINT changedPointId = args->CurrentPoint->PointerId;
UINT frameId = args->CurrentPoint->FrameId;
for (auto it = m_pointerIds.begin(); it != m_pointerIds.end(); ++it)
if (it->first != changedPointId)
otherPointId = it->first;
m_oldPoints[changedPointId] = m_pointerIds[changedPointId];
m_pointerIds[changedPointId] = args->CurrentPoint;
if (m_pointerIds[otherPointId]->FrameId == frameId)
//the other point has been updated already and we are on the update of the 2nd point... store it in memory and do calculations
// IF NOT TRUE then
//the first point is being updated, we need to wait for the 2nd point to be updated... so just store it in memory for now
The way this works is we first check to see how many touches there are. If two (and you can extrapolate all of this to more than two), we're going to do a couple of things first:
- Find the PointerId of the other touch. We know the current one's Id because it's in the event argument.
- Move the current touch's stored PointerPoint in m_pointerIds to m_oldPoints.
- Copy the current touch's new PointerPoint (from args) to m_pointerIds
- Check to see if the other touch's PointerPoint stored in m_pointerIds has the same FrameId as the current touch's FrameId.
That last step is critical. If they are the same FrameId, it means they have both been updated now and you can use the values stored in m_pointerIds to do your calculations.
If they are NOT the same, then you do nothing and wait for the next call.
This article has explained how to events work in a C++ app, and how to get enough information in order to build gesture recognition into your app. I hope it helps!