8.0 KiB

Raw Blame History

Exercise 2: Gesture Recognition part2 (Gestures)

Prerequisites

classes from part 1
inheritance in C# and abstract classes and methods
polymorphism
static methods and classes
event in C# and Event Standard Pattern
interfaces in C#

Resources

🔜 event and Event Standard Pattern
🔜 inheritance in C#
🔜 interfaces

Requested Work

As indicating in the introduction, the goal is to develop and test dynamic gestures.

classDiagram
class BaseGesture {
    <<abstract>>
    +TestGesture(Body body)*
    +GestureRecognized : EventHandler~GestureRecognizedEventArgs~
    #OnGestureRecognized(Body)
    +GestureName : string
}
class Gesture {
    +IsTesting : bool
    #MinNbOfFrames : int
    #MaxNbOfFrames : int
    -mCurrentFrameCount : int
    #TestInitialConditions(Body body)* bool
    #TestPosture(Body body)* bool
    #TestRunningGesture(Body body)* bool
    #TestEndConditions(Body body)* bool
    +TestGesture(Body body)*
}
BaseGesture <|-- Gesture
Gesture <|-- SwipeRightHand
Gesture <|-- ClapHands

Gesture represents a dynamic gesture and thus, should override the TestGesture method.
The algorithm can be separated in four important pieces:

we start the gesture recognition if initial conditions are met (TestInitialConditions)
at every frame, we check if the static current posture corresponds to a pose of this gesture (TestPosture) (for instance, for a swipe, my hand shouldn't be higher than my head)
at every frame, we check if the pose, in comparison with the previous ones, is valid or not (TestRunningGesture). In other terms, we test the dynamic of the gesture. For instance, in a swipe, my hand should not go on the left, then on the right, then on the left, then on the right...
at every frame, we check if the ending conditions are met (TestEndingConditions).

Moreover, we count the number of frames since the initial conditions were met and we check that the number of counted frames is in [MinNbOfFrames;MaxNbOfFrames] when ending conditions are met, in order to filter too slow or too fast gestures.

Next, you will develop 2 or 3 gestures and add them to your MyGesturesBank library.

At last, you will take care of the factory to ease the use of many gestures.

classDiagram
class IGestureFactory {
    +CreateGestures() IEnumerable~BaseGesture~
}
<<interface>> IGestureFactory

class GestureManager {
    <<static>>
    +EventHandler~GestureRecognizedEventArgs~ GestureRecognized$ 
    +AddGestures(IGestureFactory: factory)$
    +AddGestures(params BaseGesture[])$
    +RemoveGesture(BaseGesture)$
    +StartAcquiringFrames(KinectManager)$
    +StopAcquiringFrame()$
}
BaseGesture "*" <-- GestureManager : +KnownGestures
KinectManager "1" <-- GestureManager : KinectManager

IGestureFactory "1" <.. GestureManager : Factory

IGestureFactory <|.. AllGesturesFactory

The IGestureFactory allows creating all at once, by calling CreateGestures(), a collection of gestures.
Users can thus implement this interface to create their own collection of gestures.
At last, you will update GestureManager so that it can take a IGestureFactory as parameter in AddGestures to fill the collection of gestures.

Steps to reproduce

This part is given as a set of advises to realize your application.
I use some symbols in it:

symbol	signification
💡	be smart: this can obviously be done before practical works and without the kinect sensor!
🎬	evaluated at the end of all the practical works
🚨	mandatory if you want at least 10/20
🟢	difficulty: low
🟡	difficulty: medium
🔴	difficulty: high
🔖	quality (it's always better when it's beautiful 🥹)

Note about difficulty:
It's obviously completely subjective...

🚨🟡 Gesture & TestGesture

💡 Prepare an algorithm based on this subject. It's not that difficult, but it's not that easy neither...
Code the abstract class Gesture and above all, the TestGesture method

🚨🟡 Gesture - 1st subclass

choose a gesture and code it
test it in a console app

What gesture?
If you do not know what to code, you can begin with simple gestures: swipes, clap hands, hands up. Avoid more complex gesture made of multiple steps (waves for instance)

🔴 Gesture: use body referential instead of kinect referential

If you have tested your gesture in a Kinect referential (ie using the X, Y, Z axis of the Kinect), you may have different results if the body is not facing the Kinect. You could use another referential centered on the user body (for instance, use the axis made by the hips, the spin and the neck, etc.).

update the gesture (or code another) using a body centered referential
test it

🚨🟢🔖 IGestureFactory

add the IGestureFactory to the correct class library
implement this interace in the correct class library
update your test

🟡 Gesture - 2nd subclass

add a second gesture
test it

Evaluation criteria

To earned points, remember that you must validate your knowledge with your teacher, during the practical work.

Signification

symbol	signification
☢️	if not respected => 0/20
🎬	evaluated at the end of all the practical works
🚨	mandatory
🟢	difficulty: low
🟡	difficulty: medium
🔴	difficulty: high
🔖	quality

Criteria

@	category	description	coeff
☢️		the repositories must be accessible by the teacher	☢️
☢️		a .gitignore file must be use at the creation of the repository or at the first push	☢️
🎬		all class libraries and applications build	2
🎬		applications run without any bug	2
🚨🟡	Gesture	TestGesture	3
🚨🟡	Gesture	1st subclass	2
🔴	Gesture	use body referential instead of kinect referential	2
🚨🟢	Test	Console Test	2
🚨🟢🔖	IGestureFactory	interface and implementation	1
🚨🟢	Test	Console Test	1
🟡	Gesture	2nd subclass	2
🟢	Test	Console Test	1
🎬🟢	Documentation	ReadMe, comments, wiki...	2

Some samples if you wonder...

if you do only the 🚨 criteria and your application is building and running, you get => 13/20

a coeff 3 criterium => 3 pts

a coeff 2 criterium => 2 pts

a coeff 1 criterium => 1 pt (Note: this is just because the sum of coeffs is 20...)

Note :
Coefficients may change and are here only indicative

While preparing these practical works, I was listening to...

As Is

Bojan Z (2023)

Darkness On The Edge Of Town

Bruce Springsteen & the E Street Band (1978)

Modernistic

Jason Moran (2002)

8.0 KiB Raw Blame History