GesturePak API

The GesturePak API lets you match gestures created with the GesturePak Recorder in real-time in any .NET 4.x WPF app that uses the Microsoft Kinect For Windows v2 SDK.

Requirements and Setup

See the Requirements and Setup section of the GesturePak Recorder documentation.

References

Any app that you build with GesturePak will need to have a reference to the following:

Assembly Location
Microsoft.Kinect.dll
\Program Files\Microsoft SDKs\Kinect\vX.X\Assemblies
GesturePakObjects.dll
Compile GesturePakObjects.csproj in Visual Studio
KinectTools.dll
KinectTools comes with GesturePak v2.

Classes in the GesturePak namespace:

Loading Gestures

Before you can match Gestures you must load the gesture XML files into Gesture objects. 

Create a new Gesture object by passing in the gesture filename to the constructor. The GesturePak Recorder (GPR) saves gesture files in the \Documents\GesturePak folder. You can alternatively use the LoadFile method:

C#

    string gesturefile = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments) + @"\GesturePak\wave.xml";
    var g = new GesturePak.Gesture(gesturefile);

    // or

    var g = new GesturePak.Gesture();
    g.LoadFile(gesturefile);

VB

    Dim gesturefile = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments) + "\GesturePak\wave.xml"
    Dim g = new GesturePak.Gesture(gesturefile)

    '  or

    Dim g = new GesturePak.Gesture
    g.LoadFile(gesturefile)

Using the GestureMatcher API

You can load up multiple gestures into a GestureMatcher object by passing a List<Gesture> into the constructor.
Once you have a Body object, either by using KinectTools (easy) or by using a BodyFrameReader (see the Kinect SDK Sample BodyBasics-WPF) you can pass it into the GestureMatcher and call the GetMatch() method which returns null (or Nothing in VB) if no gesture was matched, or a real Gesture object.

Sample Code

The following sample will load up a gesture, show full color video of you performing the gesture, and will display the name of the gesture that you performed on the screen.

XAML Content:

    <StackPanel>
        <Grid x:Name="MainGrid">
            <!-- Color Video -->
            <Border Background="Black" >
                <Image x:Name="VideoImage" Margin="5" Source="{Binding VideoImageSource}" Stretch="Uniform" />
            </Border>
            <!-- Superimposed Body Video -->
            <Border Background="Transparent"  >
                <Image x:Name="BodyImage" Margin="5" Source="{Binding BodyImageSource}" Stretch="Uniform" />
            </Border>
            <!-- Superimposed Text -->
            <Border Background="Transparent"  >
                <TextBlock x:Name="GestureNameTextBlock" Margin="5" Background="Transparent" FontSize="600" Opacity=".3" FontWeight="Bold" HorizontalAlignment="Center" VerticalAlignment="Center" Foreground="Yellow" />
            </Border>
        </Grid>
    </StackPanel>


C#:

    using GesturePak;
    using Microsoft.Kinect;
    using KinectTools;

    /// <summary>
    /// Interaction logic for MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    {
        // Matcher
        private GestureMatcher matcher;
        // Path to gesture. Point this to any GesturePak created gesture
        private string gesturefile = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments) + "\\GesturePak\\wave.xml";
        // KinectTools ColorAndBodyViewer
        private ColorAndBodyViewer Viewer;

        public MainWindow()
        {
            InitializeComponent();
            this.Loaded += MainWindow_Loaded;
            this.Closing += MainWindow_Closing;
        }

        void MainWindow_Loaded(object sender, RoutedEventArgs e)
        {
            // create a Viewer object
            Viewer = new ColorAndBodyViewer();

            Viewer.ShowLiveVideo = true;
            Viewer.DrawBodies = false;
            Viewer.BodyTracked += Viewer_BodyTracked;

            // Check out the XAML bindings on the Image controls
            this.DataContext = Viewer;

            // create a gesture object
            Gesture g1 = new Gesture(gesturefile);

            // Add it to a gestures collection
            List<Gesture> gestures = new List<Gesture>();
            gestures.Add(g1);

            // Create a new matcher from the gestures
            matcher = new GestureMatcher(gestures);

        }

        void Viewer_BodyTracked(object sender, BodyTrackedEventArgs e)
        {
            matcher.Body = e.Body;
            var matchedGesture = matcher.GetMatch();
            if (matchedGesture != null)
            {
                // We have a match!
                string GestureName = matchedGesture.Name;
                GestureNameTextBlock.Text = GestureName;
            }
        }

        void MainWindow_Closing(object sender, System.ComponentModel.CancelEventArgs e)
        {
            Viewer.Dispose();
        }

    }


VB.NET:

Imports GesturePak
Imports Microsoft.Kinect
Imports KinectTools

Class MainWindow

    ' Matcher
    Private matcher As GestureMatcher
    ' Path to gesture. Point this to any GesturePak created gesture
    Private gesturefile As String = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments) & "\GesturePak\wave.xml"
    ' KinectTools ColorAndBodyViewer
    Private WithEvents Viewer As New ColorAndBodyViewer

    Private Sub MainWindow_Loaded(sender As Object, e As RoutedEventArgs) Handles Me.Loaded

        ' create a Viewer object
        Viewer = New ColorAndBodyViewer()

        Viewer.ShowLiveVideo = True
        Viewer.DrawBodies = False

        DataContext = Viewer

        ' create a gesture object
        Dim g1 = New Gesture(gesturefile)

        ' Add it to a gestures collection
        Dim gestures = New List(Of Gesture)
        gestures.Add(g1)

        ' Create a new matcher from the gestures
        matcher = New GestureMatcher(gestures)

    End Sub

    Private Sub Viewer_BodyTracked(sender As Object, e As BodyTrackedEventArgs) Handles Viewer.BodyTracked
        matcher.Body = e.Body
        Dim matchedGesture = matcher.GetMatch
        If matchedGesture IsNot Nothing Then
            ' We have a match!
            Dim GestureName = matchedGesture.Name
            GestureNameTextBlock.Text = GestureName
        End If
    End Sub


    Private Sub MainWindow_Closing(sender As Object, e As ComponentModel.CancelEventArgs) Handles Me.Closing
        Viewer.Dispose()
    End Sub

End Class