[Home](./README.md) | **Exercise 1 - Kinect Streams** | [Exercise 2 - Introduction](./exo2_subject.md) | [Exercise 2 part 1 - Postures](./exo2_1_subject.md) | [Exercise 2 part 2 - Gestures](./exo2_2_subject.md) | [Exercise 2 part 3 - Mapping](./exo2_3_subject.md) | [Exercise 3 - Free App](./exo3_subject.md) # Exercise 1: Kinect Sensor and Kinect Streams *Advised duration*: 4 hours *Maximum advised duration*: 6 hours # Prerequisites - WPF & XAML - Data binding - (not mandatory) converters - events - IDisposable - (*not mandatory*) MVVM and/or MVVM Community Toolkit # Resources - 🔜 How to turn a **WPF Application (.NET Framework)** to a **SDK-style csproj**? - 🔜 Microsoft Kinect Samples - 🔜 videos about events - 🔜 XAML - 🔜 Data Binding - 🔜 Converters # Requested Work The goal of this exercise is to create a WPF Application allowing to get these different **Kinect** streams (🚨 are mandatory): - 🚨 color stream,

- depth stream,

- infrared stream,

- 🚨 body stream,

- body & color streams (a combination of color and body streams, where the skeleton is drawn above the color image). The main (and only) window of your application, could look like this:

- the upper part indicates if the **Kinect** sensor is available and running or not. It should display a **red dot** and a convenient text if the **Kinect** sensor is unplugged or not running yet. It should display a **green dot** and a convenient text otherwise. - below this part, stands an horizontal bar of buttons allowing to switch between streams. The available streams are **color** (mandatory), **depth**, **infrared**, **body** (mandatory), and **body and color** streams. All this buttons should no be clickable if the **Kinect** is not available. - at last, the remaining of the window will display the content of the selected stream. # Steps to reproduce This part is given as a set of advises to realize your application. I use some symbols in it: symbol | signification --- | --- 💡 | be smart: this can obviously be done before practical works and without the kinect sensor! 🎬 | evaluated at the end of all the practical works 🚨 | mandatory if you want at least 10/20 🟢 | difficulty: low 🟡 | difficulty: medium 🔴 | difficulty: high 🔖 | quality (it's always better when it's beautiful 🥹) > Note about difficulty: > It's obviously completely subjective... ## 💡🚨🟢 Create the Window - Create your repository - Create a **WPF Application (.NET Framework)** - Turn it to *SDK-style csproj* (especially if you want to use *MVVM Community Toolkit*, otherwise it is useless) - Create the UI of your window (XAML) ## 🚨🟢 Kinect connection - 💡 Read the Microsoft Kinect Samples (especially the **Color Basics-XAML** sample) and find how the **Kinect** sensor is managed - Tip about how to encapsulate your **Kinect** sensor. You could write a ```KinectManager``` class like this one: ```mermaid classDiagram class KinectManager { +/Status: bool +/StatusText: string +StartSensor() +StopSensor() -KinectSensor_IsAvailableChanged(sender, args) } class KinectSensor { +/IsOpen: bool +GetDefault()$ KinectSensor +~~event~~IsAvailableChanged +Open() +Close() } KinectSensor <-- KinectManager ``` This class could be able to start and stop the sensor and listen to the IsAvailableChanged event from the ```KinectSensor```. It would also give access to status information through ```Status``` and ```StatusText```. > TIP: > If you like MVVM, consider using it directly to get more points (read next sections below). - Use this class with your app: - to start the sensor if it is plugged, when the window loads - to stop the sensor when the window unloads - use the status feedbacks to update the upper part of the window ## 🚨🟢 Display Color Image Stream - 💡 Read the Microsoft Kinect Samples (especially the **Color Basics-XAML** sample) and find how to get the color stream - Create your own class ```ColorImageStream``` to retrieve and give access to the kinect color stream - Beware of copy and paste! Keep only what's useful! - Consider adding ```Start``` and ```Stop``` methods - Update the window so that it displays the color stream when you click on the **Color** button. ## 🟡🔖 Use MVVM The two previous mandatory sections can bring more points if you use MVVM (or better, **MVVM Community Toolkit**). If you do so, your code will be less cumbersome, simpler to write, read and plug to the window. If you are not found of MVVM, it will be a nightmare. Make your choice. ## 🟢 Display Depth Image Stream It's exactly the same as for the Color Image Stream. Same difficulties, minor changes. So it is easy, it does not take a lot of time, but it does not bring more knowledges. This is why it is not mandatory, and bring less points. ## 🟢 Display IR Image Stream It's exactly the same as for the Color Image Stream. Same difficulties, minor changes. So it is easy, it does not take a lot of time, but it does not bring more knowledges. This is why it is not mandatory, and bring less points. ## 🟡🔖 Architecture - If you have prepared the three streams (but you can do it even with only one), you could decide to enhance your architecture by using inheritance. ```mermaid classDiagram direction TB class KinectStream { #Sensor : KinectSensor +Start() +Stop() } KinectStream --> "1" KinectManager : #KinectManager KinectStream <|-- ColorImageStream KinectStream <|-- DepthImageStream KinectStream <|-- InfraredImageStream ``` - Once it is done, you could make the change between one stream and another easier by using a factory: ```mermaid classDiagram direction LR class KinectStreamsFactory { +ctor(kinect: KinectManager) -streamFactory : Dictionary~KinectStreams, Func~KinectStream~~ +this[stream: KinectStreams] : KinectStream } class KinectStreams { <> None Color Depth IR } KinectStreamsFactory --> "1" KinectManager KinectStreamsFactory ..> KinectStreams KinectStreamsFactory ..> KinectStream KinectStream --> KinectManager KinectStream <|-- ColorImageStream KinectStream <|-- DepthImageStream KinectStream <|-- InfraredImageStream ``` The idea here is to provide a collection of method to create and start ```KinectStream```s. These methods are ```Func``` stored in a ```Dictionary``` that you could access through an indexer. - If you update now your window with this factory, now your code-behind (or better, your view model) should be very very simple and beautiful (🥹). ## 🚨🟡 Display Body Stream - 💡 Read the Microsoft Kinect Samples (especially the **Body Basics-XAML** sample) and find how to get the body stream. You will see that their sample is a little bit complex as they have decided to modify the image. I suggest we go in a simpler direction, by providing a canvas with ellipses representing the joints of the body, and lines representing the bones. - Beware of copy and paste! Keep only what's useful! - Update the window so that it displays the body stream when you click on the **Body** button. ## 🟡🔖 Update and use the architecture If you have used the advised architecture, use it to make your new ```BodyStream``` class a subclass of the ```KinectStream``` abstract class. You may need to change some things in the base class as the body stream uses a canvas. ## 🔴 Display Body and Color Streams in the meantime If you still have some time, find a way to display simultaneously color and body streams! ## 🟡🔖 Update and use the architecture Obviously, integrate it in your architecture if you have done one. ## 🟢🔖 Make a class library As all your classes may be used by other applications, make a class library! # Evaluation criteria To earned points, remember that you must validate your knowledge with your teacher, during the practical work. **Signification** symbol | signification --- | --- ☢️ | if not respected => 0/20 🎬 | evaluated at the end of all the practical works 🚨 | mandatory 🟢 | difficulty: low 🟡 | difficulty: medium 🔴 | difficulty: high 🔖 | quality **Criteria** @ | category | description | coeff --- | --- | --- | --- ☢️ | | the repositories must be accessible by the teacher | ☢️ ☢️ | | a .gitignore file must be use at the creation of the repository or at the first push | ☢️ 🎬 | | all *class libraries* and *applications* build | 6 🎬 | | *applications* run without any bug | 6 🚨🟢 | XAML of the Window | | 1 🚨🟢 | Kinect Connection | Use of the Kinect sensor | 2 🚨🟢 | Kinect Connection | Use of the Kinect event | 6 🚨🟢🔖 | Kinect Connection | Encapsulation | 6 🚨🟢 | Kinect Connection | Connect when the Window is loaded | 4 🚨🟢 | Kinect Connection | Disconnect when the Window is unloaded | 2 🚨🟢 | Kinect Connection | Status feedback in the Window | 6 🔴🔖 | Kinect Connection | Using MVVM (commands for window loaded and unloaded) | 6 🟡🔖 | Kinect Connection | Using MVVM (status feedback) | 6 🚨🟢🔖 | Color Stream | Encapsulation - ColorImageStream class with **ONLY** necessary members (beware of mindless copy and paste) | 6 🚨🟢 | Color Stream | retrieving the frame and processing it | 6 🚨🟢 | Color Stream | start and stop acquisition | 6 🚨🟢 | Color Stream | updating the window (start when clicking the button) | 6 🚨🟢 | Color Stream | updating the window (display the color stream) | 6 🟡🔖 | Color Stream | Using MVVM (button click + commands) | 6 🟢🔖 | Color Stream | Using MVVM (data bind the color stream to the image) | 6 🎬🚨🟢 | Color Stream | *test* runs without any bug | 4 🟢🔖 | Depth Stream | Encapsulation - DepthImageStream class with **ONLY** necessary members (beware of mindless copy and paste) | 1 🟢 | Depth Stream | retrieving the frame and processing it | 1 🟢 | Depth Stream | start and stop acquisition | 1 🟢 | Depth Stream | updating the window (start when clicking the button, stop when clicking another button) | 1 🟢 | Depth Stream | updating the window (display the color stream) | 1 🟡🔖 | Depth Stream | Using MVVM (button click + commands) | 1 🟢🔖 | Depth Stream | Using MVVM (data bind the depth stream to the image) | 1 🎬🟢 | Depth Stream | *test* runs without any bug | 1 🟢🔖 | IR Stream | Encapsulation - InfraredImageStream class with **ONLY** necessary members (beware of mindless copy and paste) | 1 🟢 | IR Stream | retrieving the frame and processing it | 1 🟢 | IR Stream | start and stop acquisition | 1 🟢 | IR Stream | updating the window (start when clicking the button, stop when clicking another button) | 1 🟢 | IR Stream | updating the window (display the IR stream) | 1 🟡🔖 | IR Stream | Using MVVM (button click + commands) | 1 🟢🔖 | IR Stream | Using MVVM (data bind the infrared stream to the image) | 1 🎬🟢 | IR Stream | *test* runs without any bug | 1 🟡🔖 | Architecture | Abstract class to factorize | 12 🔴🔖 | Architecture | Abstract class to factorize (using MVVM) | 6 🔴🔖 | Architecture | Factory to allow choosing between one stream or another | 12 🎬🔴🔖 | Architecture | *test* runs without any bug using the architecture and MVVM | 3 🚨🟢🔖 | Body Stream | Encapsulation - BodyStream class with **ONLY** necessary members (beware of mindless copy and paste) | 6 🚨🟢 | Body Stream | retrieving the frame and processing it, retrieving the body of the closest user | 6 🚨🟢 | Body Stream | start and stop acquisition | 6 🚨🟡 | Body Stream | fill a Canvas with Ellipses and Lines corresponding to the skeleton | 6 🚨🟢 | Body Stream | updating the window (start when clicking the button, stop when clicking another button) | 6 🚨🟡 | Body Stream | updating the window (display the body stream) | 6 🔴🔖 | Body Stream | Using MVVM (button click + commands) | 1 🔴🔖 | Body Stream | Using MVVM (data bind the body stream to the image) | 6 🎬🚨🔴 | Body Stream | *test* runs without any bug | 4 🔴🔖 | Architecture | Use the Body Stream with the abstract class and the Factory to allow choosing between one stream or another | 6 🎬🔴🔖 | Architecture | *test* runs without any bug using the architecture and MVVM | 3 🟡🔖 | Body and Color Streams | Encapsulation - BodyAndColorStream class (be smart) | 4 🟡🔖 | Body and Color Streams | updating the window (start when clicking the button, stop when clicking another button) | 4 🔴🔖 | Body and Color Streams | updating the window (display the body stream) | 4 🔴🔖 | Body and Color Streams | Using MVVM (button click + commands) | 1 🔴🔖 | Body and Color Streams | Using MVVM (data bind the body and color streams to the image) | 4 🎬🔴🔖 | Body and Color Streams | *test* runs without any bug | 2 🎬🔴🔖 | Architecture | *test* runs without any bug using the architecture and MVVM | 3 🟢🔖 | Architecture | Create a class library | 6 🎬🟢 | Documentation | ReadMe, comments, wiki... | 4 > Some samples if you wonder... > - if you do only the 🚨 criteria, you get => 10/20 > - for the ```KinectManager``` (without MVVM) => 2,22 pts > - for the Color Stream (without MVVM and without considering tests) => 2,56 pts > - for depth and IR streams (without MVVM and without considering tests) => 0,85 pts > - for body stream (without MVVM and without considering tests) => 3,08 pts > - for body & color streams (without MVVM and without considering tests) => 1,03 pt > - architecture => 3,08 pts > - if you use MVVM (everywhere when indicated) => 4,7 pts > - if all your tests pass and if you have done documentation => 2,48 pts > Note : > Coefficients may change and are here only indicative --- [Home](./README.md) | **Exercise 1 - Kinect Streams** | [Exercise 2 - Introduction](./exo2_subject.md) | [Exercise 2 part 1 - Postures](./exo2_1_subject.md) | [Exercise 2 part 2 - Gestures](./exo2_2_subject.md) | [Exercise 2 part 3 - Mapping](./exo2_3_subject.md) | [Exercise 3 - Free App](./exo3_subject.md) --- Copyright © 2023-2024 Marc Chevaldonné --- While preparing these practical works, I was listening to...