17 KiB

Raw Blame History

Exercise 1: Kinect Sensor and Kinect Streams

Advised duration: 4 hours
Maximum advised duration: 6 hours

Prerequisites

WPF & XAML
- Data binding
- (not mandatory) converters
events
IDisposable
(not mandatory) MVVM and/or MVVM Community Toolkit

Resources

🔜 How to turn a WPF Application (.NET Framework) to a SDK-style csproj?
🔜 Microsoft Kinect Samples
🔜 videos about events
🔜 XAML
🔜 Data Binding
🔜 Converters

Requested Work

The goal of this exercise is to create a WPF Application allowing to get these different Kinect streams (🚨 are mandatory):

🚨 color stream,
depth stream,
infrared stream,
🚨 body stream,
body & color streams (a combination of color and body streams, where the skeleton is drawn above the color image).

The main (and only) window of your application, could look like this:

the upper part indicates if the Kinect sensor is available and running or not. It should display a red dot and a convenient text if the Kinect sensor is unplugged or not running yet. It should display a green dot and a convenient text otherwise.
below this part, stands an horizontal bar of buttons allowing to switch between streams. The available streams are color (mandatory), depth, infrared, body (mandatory), and body and color streams. All this buttons should no be clickable if the Kinect is not available.
at last, the remaining of the window will display the content of the selected stream.

Steps to reproduce

This part is given as a set of advises to realize your application.
I use some symbols in it:

symbol	signification
💡	be smart: this can obviously be done before practical works and without the kinect sensor!
🎬	evaluated at the end of all the practical works
🚨	mandatory if you want at least 10/20
🟢	difficulty: low
🟡	difficulty: medium
🔴	difficulty: high
🔖	quality (it's always better when it's beautiful 🥹)

Note about difficulty:
It's obviously completely subjective...

💡🚨🟢 Create the Window

Create your repository
Create a WPF Application (.NET Framework)
Turn it to SDK-style csproj (especially if you want to use MVVM Community Toolkit, otherwise it is useless)
Create the UI of your window (XAML)

🚨🟢 Kinect connection

💡 Read the Microsoft Kinect Samples (especially the Color Basics-XAML sample) and find how the Kinect sensor is managed
Tip about how to encapsulate your Kinect sensor. You could write a KinectManager class like this one:

classDiagram
class KinectManager {
    +/Status: bool
    +/StatusText: string
    +StartSensor()
    +StopSensor()
    -KinectSensor_IsAvailableChanged(sender, args)
}
class KinectSensor {
    +/IsOpen: bool
    +GetDefault()$ KinectSensor
    +~~event~~IsAvailableChanged
    +Open()
    +Close()
}
KinectSensor <-- KinectManager

This class could be able to start and stop the sensor and listen to the IsAvailableChanged event from the KinectSensor. It would also give access to status information through Status and StatusText.

TIP:
If you like MVVM, consider using it directly to get more points (read next sections below).

Use this class with your app:
- to start the sensor if it is plugged, when the window loads
- to stop the sensor when the window unloads
- use the status feedbacks to update the upper part of the window

🚨🟢 Display Color Image Stream

💡 Read the Microsoft Kinect Samples (especially the Color Basics-XAML sample) and find how to get the color stream
Create your own class ColorImageStream to retrieve and give access to the kinect color stream
Beware of copy and paste! Keep only what's useful!
Consider adding Start and Stop methods
Update the window so that it displays the color stream when you click on the Color button.

🟡🔖 Use MVVM

The two previous mandatory sections can bring more points if you use MVVM (or better, MVVM Community Toolkit).
If you do so, your code will be less cumbersome, simpler to write, read and plug to the window.
If you are not found of MVVM, it will be a nightmare.
Make your choice.

🟢 Display Depth Image Stream

It's exactly the same as for the Color Image Stream. Same difficulties, minor changes.
So it is easy, it does not take a lot of time, but it does not bring more knowledges.
This is why it is not mandatory, and bring less points.

🟢 Display IR Image Stream

🟡🔖 Architecture

If you have prepared the three streams (but you can do it even with only one), you could decide to enhance your architecture by using inheritance.

classDiagram
direction TB
class KinectStream {
    #Sensor : KinectSensor
    +Start()
    +Stop()
}

KinectStream --> "1" KinectManager : #KinectManager

KinectStream <|-- ColorImageStream 
KinectStream <|-- DepthImageStream 
KinectStream <|-- InfraredImageStream

Once it is done, you could make the change between one stream and another easier by using a factory:

classDiagram
direction LR
class KinectStreamsFactory {
    +ctor(kinect: KinectManager)
    -streamFactory : Dictionary~KinectStreams, Func~KinectStream~~ 
    +this[stream: KinectStreams] : KinectStream
}

class KinectStreams {
    <<enum>>
    None
    Color
    Depth
    IR
}

KinectStreamsFactory --> "1" KinectManager
KinectStreamsFactory ..> KinectStreams
KinectStreamsFactory ..> KinectStream
KinectStream --> KinectManager
KinectStream <|-- ColorImageStream
KinectStream <|-- DepthImageStream
KinectStream <|-- InfraredImageStream

The idea here is to provide a collection of method to create and start KinectStreams. These methods are Func<KinectStream> stored in a Dictionary that you could access through an indexer.

If you update now your window with this factory, now your code-behind (or better, your view model) should be very very simple and beautiful (🥹).

🚨🟡 Display Body Stream

💡 Read the Microsoft Kinect Samples (especially the Body Basics-XAML sample) and find how to get the body stream. You will see that their sample is a little bit complex as they have decided to modify the image. I suggest we go in a simpler direction, by providing a canvas with ellipses representing the joints of the body, and lines representing the bones.
Beware of copy and paste! Keep only what's useful!
Update the window so that it displays the body stream when you click on the Body button.

🟡🔖 Update and use the architecture

If you have used the advised architecture, use it to make your new BodyStream class a subclass of the KinectStream abstract class. You may need to change some things in the base class as the body stream uses a canvas.

🔴 Display Body and Color Streams in the meantime

If you still have some time, find a way to display simultaneously color and body streams!

🟡🔖 Update and use the architecture

Obviously, integrate it in your architecture if you have done one.

🟢🔖 Make a class library

As all your classes may be used by other applications, make a class library!

Evaluation criteria

To earned points, remember that you must validate your knowledge with your teacher, during the practical work.

Signification

symbol	signification
☢️	if not respected => 0/20
🎬	evaluated at the end of all the practical works
🚨	mandatory
🟢	difficulty: low
🟡	difficulty: medium
🔴	difficulty: high
🔖	quality

Criteria

@	category	description	coeff
☢️		the repositories must be accessible by the teacher	☢️
☢️		a .gitignore file must be use at the creation of the repository or at the first push	☢️
🎬		all class libraries and applications build	6
🎬		applications run without any bug	6
🚨🟢	XAML of the Window		1
🚨🟢	Kinect Connection	Use of the Kinect sensor	2
🚨🟢	Kinect Connection	Use of the Kinect event	6
🚨🟢🔖	Kinect Connection	Encapsulation	6
🚨🟢	Kinect Connection	Connect when the Window is loaded	4
🚨🟢	Kinect Connection	Disconnect when the Window is unloaded	2
🚨🟢	Kinect Connection	Status feedback in the Window	6
🔴🔖	Kinect Connection	Using MVVM (commands for window loaded and unloaded)	6
🟡🔖	Kinect Connection	Using MVVM (status feedback)	6
🚨🟢🔖	Color Stream	Encapsulation - ColorImageStream class with ONLY necessary members (beware of mindless copy and paste)	6
🚨🟢	Color Stream	retrieving the frame and processing it	6
🚨🟢	Color Stream	start and stop acquisition	6
🚨🟢	Color Stream	updating the window (start when clicking the button)	6
🚨🟢	Color Stream	updating the window (display the color stream)	6
🟡🔖	Color Stream	Using MVVM (button click + commands)	6
🟢🔖	Color Stream	Using MVVM (data bind the color stream to the image)	6
🎬🚨🟢	Color Stream	test runs without any bug	4
🟢🔖	Depth Stream	Encapsulation - DepthImageStream class with ONLY necessary members (beware of mindless copy and paste)	1
🟢	Depth Stream	retrieving the frame and processing it	1
🟢	Depth Stream	start and stop acquisition	1
🟢	Depth Stream	updating the window (start when clicking the button, stop when clicking another button)	1
🟢	Depth Stream	updating the window (display the color stream)	1
🟡🔖	Depth Stream	Using MVVM (button click + commands)	1
🟢🔖	Depth Stream	Using MVVM (data bind the depth stream to the image)	1
🎬🟢	Depth Stream	test runs without any bug	1
🟢🔖	IR Stream	Encapsulation - InfraredImageStream class with ONLY necessary members (beware of mindless copy and paste)	1
🟢	IR Stream	retrieving the frame and processing it	1
🟢	IR Stream	start and stop acquisition	1
🟢	IR Stream	updating the window (start when clicking the button, stop when clicking another button)	1
🟢	IR Stream	updating the window (display the IR stream)	1
🟡🔖	IR Stream	Using MVVM (button click + commands)	1
🟢🔖	IR Stream	Using MVVM (data bind the infrared stream to the image)	1
🎬🟢	IR Stream	test runs without any bug	1
🟡🔖	Architecture	Abstract class to factorize	12
🔴🔖	Architecture	Abstract class to factorize (using MVVM)	6
🔴🔖	Architecture	Factory to allow choosing between one stream or another	12
🎬🔴🔖	Architecture	test runs without any bug using the architecture and MVVM	3
🚨🟢🔖	Body Stream	Encapsulation - BodyStream class with ONLY necessary members (beware of mindless copy and paste)	6
🚨🟢	Body Stream	retrieving the frame and processing it, retrieving the body of the closest user	6
🚨🟢	Body Stream	start and stop acquisition	6
🚨🟡	Body Stream	fill a Canvas with Ellipses and Lines corresponding to the skeleton	6
🚨🟢	Body Stream	updating the window (start when clicking the button, stop when clicking another button)	6
🚨🟡	Body Stream	updating the window (display the body stream)	6
🔴🔖	Body Stream	Using MVVM (button click + commands)	1
🔴🔖	Body Stream	Using MVVM (data bind the body stream to the image)	6
🎬🚨🔴	Body Stream	test runs without any bug	4
🔴🔖	Architecture	Use the Body Stream with the abstract class and the Factory to allow choosing between one stream or another	6
🎬🔴🔖	Architecture	test runs without any bug using the architecture and MVVM	3
🟡🔖	Body and Color Streams	Encapsulation - BodyAndColorStream class (be smart)	4
🟡🔖	Body and Color Streams	updating the window (start when clicking the button, stop when clicking another button)	4
🔴🔖	Body and Color Streams	updating the window (display the body stream)	4
🔴🔖	Body and Color Streams	Using MVVM (button click + commands)	1
🔴🔖	Body and Color Streams	Using MVVM (data bind the body and color streams to the image)	4
🎬🔴🔖	Body and Color Streams	test runs without any bug	2
🎬🔴🔖	Architecture	test runs without any bug using the architecture and MVVM	3
🟢🔖	Architecture	Create a class library	6
🎬🟢	Documentation	ReadMe, comments, wiki...	4

Some samples if you wonder...

if you do only the 🚨 criteria, you get => 10/20

for the KinectManager (without MVVM) => 2,22 pts

for the Color Stream (without MVVM and without considering tests) => 2,56 pts

for depth and IR streams (without MVVM and without considering tests) => 0,85 pts

for body stream (without MVVM and without considering tests) => 3,08 pts

for body & color streams (without MVVM and without considering tests) => 1,03 pt

architecture => 3,08 pts

if you use MVVM (everywhere when indicated) => 4,7 pts

if all your tests pass and if you have done documentation => 2,48 pts