Introduction

As Microsoft's latest mobile operating system, Windows Phone 7 is a vast improvement over its previous buddies. Based on the ready-made facilities, such as Silverlight, XNA, and C#, with a slight modification you can develop most types of extremely powerful mobile applications. And also, as you can image, Windows Phone 7 aims at high platforms and high configurations.

In this article, we will focus upon MultiTouch and manipulation related operations in Silverlight for Windows Phone programming. In fact, there are several ways to obtain the equipment touch information and gain control over the phone. The most closely related event should be Touch.FrameReported and three other ones defined in UIElement, i.e. ManipulationStarted, ManipulationDelta, and ManipulationCompleted. In addition, there are some more, to be covered later on.

First of all, let's research into the Touch class and its FrameReported event.

The Touch Class and Its FrameReported Event

The Touch.FrameReported event serves as an application-level service, providing an underlying touch programming interface for Silverlight for Windows Phone. In detail, we can use it to get information for each touch point whether in scope of the operating system or the entire application. Touch is a public static class, which only contains a static member - the FrameReported event.

As you can easily image, we can, in our applications, subscribe to this event and then through the TouchFrameEventArgs parameter in the event handler to obtain data we want to work with.

Subscribe to the Touch.FrameReported event

Now, let's first look at the event subscription in our sample page TouchPage.xaml (the sample project is named WP7Touch).

Completely the same event subscription sketch as that in Silverlight, isn't it? TouchFrameEventArgs contains a couple of secrets that can tell all we want to make clear. So, let's continue to look at this class definition.

First, the GetTouchPoints method returns a collection of touch points, which will be of great help when you want to use multiple fingers to manipulate your Windows Phone 7 application and at the same time you want to identify the specific location for each finger.

Next, the GetPrimaryTouchPoint method returns the primary touch point for the current frame. Note the primary touch point refers to the corresponding touch point when you use only one finger to touch the screen. If you use only one finger when you touch the screen, it must be the primary touch point. Consider another case: the first finger is touching the screen while the second finger is put on the screen. In this case, the second finger isn't the primary touch point. But now if you are still putting the second finger on the screen while lift the first finger and then put it back on the screen. It is no more the primary touch point! Primary touch point will only appear only in the absence of other fingers touching the screen.

Note GetPrimaryTouchPoint and GetTouchPoints both require a parameter of type UIElement, both returning information associated with the touch point.

Next, let's look more closely at the TouchPoint class related definition.

In essence, TouchPoint is an abstraction of the touching finger across the screen. It provides 4 read- only properties:

Action: of enum type TouchAction, contains 3 values: Down, Move and Up.

Position: of type Point, which is relative to the upper left corner of the referencing element ( i.e. the UIElement parameter in the above-mentioned methods GetPrimaryTouchPoint or GetTouchPoints). If the parameter passed in is null, then the resulting Position is relative to the upper left corner of the screen (the Position may be negative when the incoming value is non- null).

Size: of type Size, corresponding to a rectangular area within the touch on the screen. But the value of the property seemingly cannot obtain the desired valid value because, in my demo, its Width and Height properties are always 1 (perhaps because we are running on an emulator?).

TouchDevice: of type TouchDevice, contains 2 read-only properties: one is Id property (of type int) to differentiate fingers; the other is the DirectlyOver property of type UIElement, which is the top-level UI element closest to the fingers. Note if we need to distinguish between multiple fingers, then the Id attribute comes to help. When a particular finger touches the screen, a series of specific events associated with the finger always begin with a Down operation, followed by the Move event, and finally Up event. All of these events are associated with the same Id. (However, we should not take for granted that the primary touch point related Id value is always 0 or 1, although it is always 0 in my demo.)

Now let's tidy up: first, subscribe to the Touch.FrameReported event; second, invoke the GetPrimaryTouchPoint or GetTouchPoints method of the TouchFrameEventArgs parameter in the event handler to obtain the corresponding touch point data. Note the method GetPrimaryTouchPoint returns a TouchPoint while the method GetTouchPoints returns a TouchPointCollection (this collection contains 0 or more TouchPoints). Last, from the TouchPoint corresponding attribute, we can get more detailed information. Please see the following code:

There are 3 types of information, Down, Move and Up, hidden inside the TouchPoint structure returned in the GetPrimaryTouchPoint method or TouchPointCollection returned in the GetTouchPoints method. In another word, when you press and then release the fingers the FrameReported event will be fired once respectively, the action information being Up and Down, respectively. If you move your fingers on the screen, then the FrameReported event will be triggered at least 3 times, and the action information being Down, Move, and Up. Let's look at the running-time screenshots, as shown from Figure 1 to 3.

Figure 1: When the mouse is pressed (in the Simulator)

Figure 2: When the mouse moves(in the Simulator)

Figure 3: When the mouse is released (in the Simulator)

In addition, if you use two fingers touching the screen, the FrameReported event will be triggered twice respectively. But, if you call the GetPrimaryTouchPoint method it only returns the first finger touching screen information. GetPrimaryTouchPoint will return null for the second finger. When you touch the screen using more than two fingers the case will be same as using two fingers.

About the SuspendMousePromotionUntilTouchUp method

Till now, we've not mentioned another method SuspendMousePromotionUntilTouchUp of the TouchFrameEventArgs class. This method is used to disable automatic mouse-event promotion for the primary touch point until all touch points report as Up. The mouse event promotion term derives from the desktop version of Silverlight, so that MultiTouch users can use touch and gestures to substitute the mouse moves or mouse clicks. For example, when a user uses the MultiTouch devices to click on a button, the expected behavior of the button will be the same as when the mouse is clicked, so the desktop version of Silverlight delivers the touch input mechanism for automatic promotion to mouse events, and later it gets extended to Silverlight for Windows Phone. However, mouse event promotion is only valid for the primary touch point (the first finger touching the screen and at the same time there are no other fingers touching the screen), so if you don't want to make the movements of a specified finger be promoted, you can use the SuspendMousePromotionUntilTouchUp method to suspend mouse event promotion.

Note the SuspendMousePromotionUntilTouchUp method can only be invoked in the primary touch point down case; otherwise, an InvalidOperationException exception will be thrown.

When the above invocation condition is not met, the program will throw an exception, as shown in Figure 4.

Figure 4: An exception is thrown when the SuspendMousePromotionUntilTouchUp method calling condition is not met

The correct calling approach should be like the following.

The last point to notice is in Windows Phone 7 the mouse operation usually does not need to be handled, unless the mobile contains controls that cannot be dealt with MultiTouch input. So, in most cases, we do not have to suppress the mouse event promotion.

Well, regarding the underlying Touch interface we've introduced so much. Starting from the next section, I will introduce the high-level MultiTouch programming interface, associated with the UIElement class.

The Manipulation Related Events

The three events, i.e. ManipulationStarted, ManipulationDelta and ManipulationCompleted are not alone to deal with each finger's touch information. In fact, all finger- panning and zooming manipulations have been integrated. Due to these three events are all defined in the UIElement class, based on a specific element, rather than the application-level events, so we can add them to any UI element, such as ListBox, Canvas, Rectangle, and so on. The following XAML code (in the sample page ManipulationPage.xaml) illustrates an example of subscribing to these three events for a Rectangle element.

Seen from their name, the order in which they are triggered can be easily deduced: first a ManipulationStarted event, followed by 0 or more of the ManipulationDelta events, and finally the ManipulationCompleted event. Their corresponding event handlers defined in the Code-Behind file are as follows:

In most cases we need to deal with these three types of event parameters. Despite different types, they all have the following same properties:

• OriginalSource: of type object and defined within the RoutedEventArgs class, through which we can obtain the original object to trigger the event.
• ManipulationContainer: of type UIElement, through which we can get the object that defines the current touch operation coordinate (usually the same as OriginalSource).
• ManipulationOrigin: of type Point, through which we can get the original coordinates of the action, i.e. the finger touch point coordinates (whose values are relative to the left corner of the ManipulationContainer object). If you have two or more fingers touching one element, then the ManipulationOrigin property gives the average coordinates of multiple fingers.
• Handled: of type bool, used to indicate the event handling states of routed events during the routing process. If we don't want the current event to be propagated down the visual tree we can set it to true.

In the next section, I will describe all the Manipulation events-related stuff through a concrete example.

Manipulating the rectangle

This sample page is also simple, with a Rectangle control in the Grid named ContentPanel (see Figure 5). We can drag the rectangle with the parent control to look at the interested Manipulation events. To gain a better result I've also capture the document tree in the left of Figure 5.

Figure 5: A screenshot in the design time

Now, let's look at the Behind-Code implementation.

Some of the running-time output results are shown in Figure 6.

Figure 6: The tracking manipulation result is output to the Debug window

It is the same order as anticipated above. And also, the preceding actions trigger the ManipulationDelta event 3 times. Because the fingers (here is a mouse, because it is the act on the Simulator) move on the screen, so each time the ManipulationOrigin value is not the same. At the same time, we can also see OriginalSource and ManipulationContainer are the same.

Select the correct events triggered on an element

Different from the Touch.FrameReported event that associates specific fingers with a different Id, the Manipulation related events are based on the UI element, so these events do not require Id. When multiple fingers touch a single element they will be converted to a series of Manipulation related events, while when the two fingers touch the different elements two different series of Manipulation related events occur (and the two series are independent). Of course, they can be distinguished using the ManipulationContainer property. For example, placing a finger upon an element, a ManipulatedStarted event will be triggered; and if your finger moves the ManipulationDelta event will also be triggered. Keeping this finger intact, placing another finger on the same element will not trigger a new ManipulatonStarted event. But if I put another finger on the other element, it triggers the ManipulationStarted event of the corresponding element.

If you want to keep track of different fingers touch information on a single element, then you should fall back upon the Touch.FrameReported event.

Detailing 3 kinds of Manipulation event arguments

First, let's look at the simplest ManipulationStartedEventArgs class. In addition to the above 4 same properties, it also has a Complete method, which is used to tell the system to end off the ManipulationStarted event, so that if your fingers move on the screen the ManipulationDelta will not be triggered.

Next is the ManipulationDeltaEventArgs class. In addition to common properties, this class also contains 2 properties of type ManipulationDelta: CumulativeManipulation and DeltaManipulation. And ManipulationDelta contains 2 properties of type Point: Scale and Translation.

Scale and Translation can help us resolve the compound action of one or more fingers on an element into moving and sizing of the element itself. Scale represents a scaling factor; Translation is a translation of the distance. We can change the value of Translation using one finger, but if you want to change Scale you need to use two fingers. When you move your finger on an element, the difference of new and original locations of the finger will be reflected in Translation. If you are using two fingers to zoom, the difference of distances between the original fingers and new fingers after zooming will be reflected in Scale. One thing to note is: if we do not have the element scaled, the Scale value is (0, 0).

Now, let's look at the differences between CumulativeManipulation and DeltaManipulation. Although both of them contains Scale and Translation, those in CumulativeManipulation are the result of adding accumulation from the ManipulationStarted event to the current event (can be seen by the property name), while those in DeltaManipulation are the result of the latest ManipulationDelta event relative to the previous ManipulationDelta or ManipulationStarted event, just a onetime change.

In addition to the CumulativeManipulation and DeltaManipulation properties, ManipulationDeltaEventArgs also has a Complete method, whose role is same as that in the ManipulationStarted event, notifying system the current operation is over. So, after calling this method even if the finger moves on some element the ManipulationDelta event will only be triggered one time, as a result of which this series of Manipulation operations will no longer follow up the ManipulationDelta event (However the ManipulationCompleted event can be triggered).

In addition, ManipulationDeltaEventArgs also has an IsInertial property and a Velocities property. For the related details, readers can continue the related researching.

Finally, ManipulationCompletedEventArgs also contains the following 3 attributes:

• FinalVelocities: of same type as Velocities in the ManipulationDeltaEventArgs class - ManipulationVelocities, through which you can obtain the speed when your finger leaves the screen.
• IsInertial: of type bool, same as IsInertial in ManipulationDeltaEventArgs, to be explained in detail later.
• TotalManipulation: similar to CumulativeManipulation in ManipulationDeltaEventArgs, of type ManipulationDelta, corresponds to the whole process of accumulation from the ManipulationStarted event to the ManipulationCompleted event.

A complete demo

Now, let's look into a complete example. I've added a TranslateTransform to the RenderTransform property of the Rectangle element. Now, in the ManipulationDelta event handler of the Rectangle element we work with TranslateTransform, making this rectangle movable on the screen, and at the same time having the obtained information output to the Output window. Let's check out the rewritten code:

In the rectangle_ManipulationDelta method above, I assign the x and y of Translation to the x and y of TranslateTransform, to ensure that the rectangle is free to move. The following is a simulator and a screenshot of the output window:

Figure 7: Move the rectangle to find more result

Figure 8: Part of the related output

Note in the figure above the FinalVelocities value in the ManipulationCompleted event parameter is large; it is because I drag the rectangle very quickly. However, if it is a very slow moving, then this value is changed to 0, and the IsInertial property to False. In addition, if you call the Complete method of the event arguments in the ManipulationDelta event handler, then you can only move the rectangle one time and then another, because the ManipulationDelta event will only be triggered for one time and the FinalVelocities property in the ManipulationCompleted event parameters will throw a NullReferenceException exception.

More about the IsInertial property

In WPF, by setting the ManipulationInertiaStartingEventArgs parameter, the system, when fingers leave the screen, can simulate the effect of inertia using an algorithm, i.e. an extra ManipulationDelta event is triggered. Silverlight for Windows Phone, however, does not support this feature. In WPF, the IsInertial property in the ManipulationDelta and ManipulationCompleted event parameters is used to indicate whether the current event is triggered during the course of inertial effects occurring. In Silverlight for Windows Phone, the IsInertial property in the ManipulationDelta event parameter is always False. Even in the case of fast moving and the IsInertial in the ManipulationCompleted event being True, any inertia effects do not appear, just the process of moving smooth. And also, as soon as the finger lifts the rectangle element immediately stops moving. So, in my opinion, the IsInertial property in the ManipulationDelta and ManipulationCompleted event parameters does not have too much value. As a result, if you want to simulate inertia effects in Silverlight for Windows Phone, then we can achieve this with the help of the FinalVelocities property in the ManipulationCompleted event argument.

Well, we talk so much concerning the high-level touch programming interfaces on Windows Phone 7. Starting from the next section, we will explore the gesture-related touch operations.

GestureListener in Silverlight for Windows Phone Toolkit

Silverlight for Windows Phone Toolkit from CodePlex offers developers a couple of controls for Windows Phone application development, designed to match the rich user experience of the Windows Phone 7. In this article, we will only show interest in the GestureListener control in the toolkit.

Install Silverlight for Windows Phone Toolkit

Follow the steps below to download and install Silverlight for Windows Phone Toolkit:

1. Open your browser and navigate to the url http://silverlight.codeplex.com/releases to download the file Silverlight for Windows Phone Toolkit - Nov 2010.msi (1828KB).

2. Double click the above .msi file to install Silverlight for Windows Phone Toolkit. By default, the install path should be C:\Program Files\Microsoft SDKs\Windows Phone\v7.0\Toolkit\Nov10.

Using the GestureListener Control

1. Add a simple Windows Phone 7 Protrait Page named GestureListenerPage.xaml.

2. Right click the sample project to add reference to the Silverlight for Windows Phone Toolkit assembly, as shown in Figure 9.

Figure 9: Add reference to the Silverlight for Windows Phone Toolkit assembly

3. Open the file GestureListenerPage.xaml and add the following xmlns reference.

4. Inside the Grid control named ContentPanel add the following:

Here GestureService is a helper wrapper around the control GestureListener. In fact, an easy and typical way to add the preceding interested event listeners is clicking the GestureListener control and then double clicking the related items from the Events tab in the Properties dialog. In this case, I've added our interested gesture listeners to the target rectangle control.

Next, let's continue to look at the related Code-Behind event handlers programming.

As is seen, the above events are very easy to use. GestureBegin and GestureCompleted are quite similar to the previous ManipulationStarted and ManipulationCompleted, while the most similar events like Manipulation should be Drag.

Now, let's watch the running result. When you click (Tap) the rectangle, it will change the color randomly. The output is as shown in Figure 10.

Figure 10: Click (Tap) the rectangle related output

When you double click (Double Tap) the rectangle, it will return to its original location, and color changes because a double click includes a single click. The output is as shown in Figure 11.

Figure 11: Double click the rectangle related output

In the state of Press and Hold, a dialog box will pop up. The output is shown in Figure 12.

Figure 12: Press and hold will result in a popup dialog

A Flick operation includes the Drag operation. The output is as shown in Figure 13.

Figure 13: The flick operation corresponding output

On the whole, with the help of the GestureListener object and its related events in Silverlight for Windows Phone Toolkit, we can more easily identify the gesture.

More to Explore

There is also another gesture related solution in the XNA library. This mainly revolves around the TouchPanel class. I want to leave this to the readers to start new exploration.

Summary

In Windows Phone 7 Silverlight Programming, MultiTouch and Manipulation are one of the fundamentals. This article brought you a very elementary tutorial with very simple and even ugly samples. Although you may have a good foundation in the area of C# and Silverlight, there are still much to learn. Anyway, we have enough reasons to say tomorrow of Windows Phone 7 will be brilliant.

About Xianzhong Zhu

I'm a college teacher and also a freelance developer and writer from WeiFang China, with more than fourteen years of experience in design, and development of various kinds of products and applications on Windows platform. My expertise is in Visual C++/Basic/C#, SQL Server 2000/2005/2008, PHP+MyS... This author has published 81 articles on DotNetSlackers. View other articles or the complete profile here.