8.6 KiB

Raw Blame History

Exercise 3: Gesture Recognition part3 (Mapping)

Prerequisites

classes from part 1 & 2
inheritance in C# and abstract classes and methods
polymorphism
static methods and classes
event in C# and Event Standard Pattern
generic classes

Resources

🔜 event and Event Standard Pattern
🔜 inheritance in C#
🔜 interfaces
🔜 generics

Requested Work

This third and last part aims at mapping body movements to something else movement (an ellipse, a cat, a nounours, a dinosaur...).

In order to do that, benefit from generic classes, as we do not know in advance what to map with a body.

The BaseMapping<T> generic class is very simple:

its Mapping abstract method has the responsability to return a T value or instance at every frame, based on a Body
the TestMapping will check if Mapping can be called or not.
SubscribeToStartGesture, SubscribeToEndGesture and SubscribeToToggleGesture methods toggle a boolean indicating if the mapping is runnning or not when it receives event from gestures. For instance, every time the posture LeftHandUp is received, it starts the mapping, or every time the gesture SwipeRightHand is received, it stops the mapping... The SubscribeToToggleGesture method is used in order to use the same gesture to start and stop the mapping. So the user will have to either use start gesture and stop gesture, or use toggle gesture. But, in fact, he could also use multiple gestures!
GestureManager will fire an event BodyFrameArrived every time it detects a body in a frame. This event should be catch by instances of BaseMapping. In order to do that, mappings will subscribe (unsubscribe) to the BodyFrameArrived event when the GestureManager.AddMapping method is called (RemoveMapping to unsubscribe).
every time a mapping receives the BodyFrameArrived event, it calls TestMapping and if it returns true, it fires itself its event OnMapping
OnMapping is the event to which you can subscribe in your app to do a mapping

classDiagram
direction TB
class BaseGesture {
    +TestGesture(Body body)*
    +GestureRecognized : EventHandler~GestureRecognizedEventArgs~
    #OnGestureRecognized(Body)
    +GestureName : string
}

class BaseMapping~T~ {
    -running : bool
    +EventHandler<OnMappingEventArgs> : OnMapping
    +SubscribeToStartGesture(gesture: BaseGesture)
    +SubscribeToEndGesture(gesture: BaseGesture)
    +SubscribeToToggleGesture(gesture: BaseGesture)
    #Mapping(Body body)* T
    ~TestMapping(Body body, out T ouput) bool
    ~#OnBodyFrameArrived(object, BodyFrameArrivedEventArgs)
}

class MapLeftHandToMouse {
    #Mapping(Body body)* Point
}

BaseMapping~T~ <|-- MapLeftHandToMouse : T <- Point

BaseGesture <.. BaseMapping~T~

class GestureManager {
    <<static>>
    +EventHandler~GestureRecognizedEventArgs~ : GestureRecognized$ 
    +EventHandler~BodyFrameArrivedEventArgs~ : BodyFrameArrived$ 
    +AddGestures(IGestureFactory: factory)$
    +AddGestures(params BaseGesture[])$
    +RemoveGesture(BaseGesture)$
    +AddMapping~T~(BaseMapping~T~: mapping)$
    +RemoveMapping~T~(BaseMapping~T~: mapping)$
    +StartAcquiringFrames(KinectManager)$
    +StopAcquiringFrame()$
}
BaseGesture "*" <-- GestureManager : +KnownGestures
BaseMapping <.. GestureManager

Steps to reproduce

This part is given as a set of advises to realize your application.
I use some symbols in it:

symbol	signification
💡	be smart: this can obviously be done before practical works and without the kinect sensor!
🎬	evaluated at the end of all the practical works
🚨	mandatory if you want at least 10/20
🟢	difficulty: low
🟡	difficulty: medium
🔴	difficulty: high
🔖	quality (it's always better when it's beautiful 🥹)

Note about difficulty:
It's obviously completely subjective...

🚨🟡 Mapping and TestGesture

create the BaseMapping<T> generic class
manage the running boolean member so that it changes when the events GestureRecognized are received
add the methods Mapping (abstract) and TestMapping calling the former one and return a bool and the result from Mapping.
add constructors with either the start and end gestures, or the toggle gesture

🚨🟡 GestureManager - BodyFrameArrived

add a new event BodyFrameArrived to GestureManager whose arguments give access to bodies in the frame and also the closest one to the kinect sensor
change GestureManager so that it fires this event when it acquires and processes body frames from the kinect sensor

🟢 IGestureFactory & access to gestures

As you may need access to the gestures of a factory, and as we do not have given any public access, add an indexer to get a gesture by its name (if you do not know nor want to use an indexer, use a method...)

🔴 ConcreteMapping

Create a concrete mapping (for instance to the 2D position of an hand).
Test it.

🔴🔖 OnMapping

add and manage the OnMapping event to BaseMapping. Its arguments should give access to the mapped value
fire this event when the mapping is running
update and test your program

Evaluation criteria

To earned points, remember that you must validate your knowledge with your teacher, during the practical work.

Signification

symbol	signification
☢️	if not respected => 0/20
🎬	evaluated at the end of all the practical works
🚨	mandatory
🟢	difficulty: low
🟡	difficulty: medium
🔴	difficulty: high
🔖	quality

Criteria

@	category	description	coeff
☢️		the repositories must be accessible by the teacher	☢️
☢️		a .gitignore file must be use at the creation of the repository or at the first push	☢️
🎬		all class libraries and applications build	1
🎬		applications run without any bug	1
🚨🟡	Mapping	TestGesture	1
🚨🟡	GestureManager	BodyFrameArrived	1
🟢	IGestureFactory	indexer	1
🔴	ConcreteMapping	BodyFrameArrived	1
🚨🟢	Test	Console Test	1
🔴🔖	OnMapping	BodyFrameArrived + OnMapping	1
🟢	Test	Console Test	1
🎬🟢	Documentation	ReadMe, comments, wiki...	1

Some samples if you wonder...

if you do only the 🚨 criteria and your application is building and running, you get => 10/20

Note :
Coefficients may change and are here only indicative

While preparing these practical works, I was listening to...

Hypersuites

Marina Baranova (2016)

Dracula

Pierre Henry (2003)

1935-1936

Fats Waller (1935-1936)

Just Jazz

Aldo Romano (2008)

Carolina Shout

James P. Johnson (1920's - 1930's)

8.6 KiB Raw Blame History