[Home](./README.md) | [Exercise 1 - Kinect Streams](./exo1_subject.md) | [Exercise 2 - Introduction](./exo2_subject.md) | [Exercise 2 part 1 - Postures](./exo2_1_subject.md) | **Exercise 2 part 2 - Gestures** | [Exercise 2 part 3 - Mapping](./exo2_3_subject.md) | [Exercise 3 - Free App](./exo3_subject.md) # Exercise 2: Gesture Recognition part2 (Gestures) # Prerequisites - classes from part 1 - inheritance in C# and abstract classes and methods - polymorphism - static methods and classes - event in C# and Event Standard Pattern - interfaces in C# # Resources - 🔜 event and Event Standard Pattern - 🔜 inheritance in C# - 🔜 interfaces # Requested Work As indicating in the introduction, the goal is to develop and test dynamic gestures. ```mermaid classDiagram class BaseGesture { <> +TestGesture(Body body)* +GestureRecognized : EventHandler~GestureRecognizedEventArgs~ #OnGestureRecognized(Body) +GestureName : string } class Gesture { +IsTesting : bool #MinNbOfFrames : int #MaxNbOfFrames : int -mCurrentFrameCount : int #TestInitialConditions(Body body)* bool #TestPosture(Body body)* bool #TestRunningGesture(Body body)* bool #TestEndConditions(Body body)* bool +TestGesture(Body body)* } BaseGesture <|-- Gesture Gesture <|-- SwipeRightHand Gesture <|-- ClapHands ``` ```Gesture``` represents a *dynamic* gesture and thus, should override the ```TestGesture``` method. The algorithm can be separated in four important pieces: - we start the gesture recognition if **initial conditions** are met (```TestInitialConditions```) - at every frame, we check if the static current posture corresponds to a pose of this gesture (```TestPosture```) (for instance, for a swipe, my hand shouldn't be higher than my head) - at every frame, we check if the pose, in comparison with the previous ones, is valid or not (```TestRunningGesture```). In other terms, we test the *dynamic* of the gesture. For instance, in a swipe, my hand should not go on the left, then on the right, then on the left, then on the right... - at every frame, we check if the ending conditions are met (```TestEndingConditions```). Moreover, we count the number of frames since the initial conditions were met and we check that the number of counted frames is in [```MinNbOfFrames```;```MaxNbOfFrames```] when ending conditions are met, in order to filter too slow or too fast gestures. Next, you will develop 2 or 3 gestures and add them to your *MyGesturesBank* library. At last, you will take care of the factory to ease the use of many gestures. ```mermaid classDiagram class IGestureFactory { +CreateGestures() IEnumerable~BaseGesture~ } <> IGestureFactory class GestureManager { <> +EventHandler~GestureRecognizedEventArgs~ GestureRecognized$ +AddGestures(IGestureFactory: factory)$ +AddGestures(params BaseGesture[])$ +RemoveGesture(BaseGesture)$ +StartAcquiringFrames(KinectManager)$ +StopAcquiringFrame()$ } BaseGesture "*" <-- GestureManager : +KnownGestures KinectManager "1" <-- GestureManager : KinectManager IGestureFactory "1" <.. GestureManager : Factory IGestureFactory <|.. AllGesturesFactory ``` The ```IGestureFactory``` allows creating all at once, by calling ```CreateGestures()```, a collection of gestures. Users can thus implement this interface to create their own collection of gestures. At last, you will update ```GestureManager``` so that it can take a ```IGestureFactory``` as parameter in ```AddGestures``` to fill the collection of gestures. # Steps to reproduce This part is given as a set of advises to realize your application. I use some symbols in it: symbol | signification --- | --- 💡 | be smart: this can obviously be done before practical works and without the kinect sensor! 🎬 | evaluated at the end of all the practical works 🚨 | mandatory if you want at least 10/20 🟢 | difficulty: low 🟡 | difficulty: medium 🔴 | difficulty: high 🔖 | quality (it's always better when it's beautiful 🥹) > Note about difficulty: > It's obviously completely subjective... ## 🚨🟡 Gesture & TestGesture - 💡 Prepare an algorithm based on this subject. It's not that difficult, but it's not that easy neither... - Code the abstract class ```Gesture``` and above all, the ```TestGesture``` method ## 🚨🟡 Gesture - 1st subclass - choose a gesture and code it - test it in a console app > What gesture? > If you do not know what to code, you can begin with simple gestures: swipes, clap hands, hands up. > Avoid more complex gesture made of multiple steps (waves for instance) ## 🔴 Gesture: use body referential instead of kinect referential If you have tested your gesture in a Kinect referential (_ie_ using the X, Y, Z axis of the Kinect), you may have different results if the body is not facing the Kinect. You could use another referential centered on the user body (for instance, use the axis made by the hips, the spin and the neck, etc.). - update the gesture (or code another) using a body centered referential - test it ## 🚨🟢🔖 IGestureFactory - add the ```IGestureFactory``` to the correct class library - implement this interace in the correct class library - update your test ## 🟡 Gesture - 2nd subclass - add a second gesture - test it # Evaluation criteria To earned points, remember that you must validate your knowledge with your teacher, during the practical work. **Signification** symbol | signification --- | --- ☢️ | if not respected => 0/20 🎬 | evaluated at the end of all the practical works 🚨 | mandatory 🟢 | difficulty: low 🟡 | difficulty: medium 🔴 | difficulty: high 🔖 | quality **Criteria** @ | category | description | coeff --- | --- | --- | --- ☢️ | | the repositories must be accessible by the teacher | ☢️ ☢️ | | a .gitignore file must be use at the creation of the repository or at the first push | ☢️ 🎬 | | all *class libraries* and *applications* build | 2 🎬 | | *applications* run without any bug | 2 🚨🟡 | Gesture | TestGesture | 3 🚨🟡 | Gesture | 1st subclass | 2 🔴 | Gesture | use body referential instead of kinect referential | 2 🚨🟢 | Test | Console Test | 2 🚨🟢🔖 | IGestureFactory | interface and implementation | 1 🚨🟢 | Test | Console Test | 1 🟡 | Gesture | 2nd subclass | 2 🟢 | Test | Console Test | 1 🎬🟢 | Documentation | ReadMe, comments, wiki... | 2 > Some samples if you wonder... > - if you do only the 🚨 criteria and your application is building and running, you get => 13/20 > - a coeff 3 criterium => 3 pts > - a coeff 2 criterium => 2 pts > - a coeff 1 criterium => 1 pt (Note: this is just because the sum of coeffs is 20...) > Note : > Coefficients may change and are here only indicative --- [Home](./README.md) | [Exercise 1 - Kinect Streams](./exo1_subject.md) | [Exercise 2 - Introduction](./exo2_subject.md) | [Exercise 2 part 1 - Postures](./exo2_1_subject.md) | **Exercise 2 part 2 - Gestures** | [Exercise 2 part 3 - Mapping](./exo2_3_subject.md) | [Exercise 3 - Free App](./exo3_subject.md) --- Copyright © 2023-2024 Marc Chevaldonné --- While preparing these practical works, I was listening to...

As Is

Bojan Z (2023)

Darkness On The Edge Of Town

Bruce Springsteen & the E Street Band (1978)

Modernistic

Jason Moran (2002)