Kinect-Project/exo1_subject.md

[Home](./README.md)
| **Exercise 1 - Kinect Streams**
| [Exercise 2 - Introduction](./exo2_subject.md) 
| [Exercise 2 part 1 - Postures](./exo2_1_subject.md) 
| [Exercise 2 part 2 - Gestures](./exo2_2_subject.md) 
| [Exercise 2 part 3 - Mapping](./exo2_3_subject.md) 
| [Exercise 3 - Free App](./exo3_subject.md)

# Exercise 1: Kinect Sensor and Kinect Streams

*Advised duration*: 4 hours  
*Maximum advised duration*: 6 hours

# Prerequisites
- WPF & XAML
  - Data binding
  - (not mandatory) converters
- events
- IDisposable
- (*not mandatory*) MVVM and/or MVVM Community Toolkit

# Resources
- 🔜 How to turn a **WPF Application (.NET Framework)** to a **SDK-style csproj**?
- 🔜 Microsoft Kinect Samples
- 🔜 videos about events
- 🔜 XAML
- 🔜 Data Binding
- 🔜 Converters

# Requested Work

The goal of this exercise is to create a WPF Application allowing to get these different **Kinect** streams (🚨 are mandatory):
- 🚨 color stream,  
  <img src="images/forsubjects/color.PNG" height="120px"/>
- depth stream,  
  <img src="images/forsubjects/depth.PNG" height="120px"/>
- infrared stream,  
  <img src="images/forsubjects/IR.PNG" height="120px"/>
- 🚨 body stream,  
  <img src="images/forsubjects/body.PNG" height="120px"/>
- body & color streams (a combination of color and body streams, where the skeleton is drawn above the color image).  

The main (and only) window of your application, could look like this:  
<img src="images/forsubjects/sketch01.png" width="800px"/>  
- the upper part indicates if the **Kinect** sensor is available and running or not. It should display a **red dot** and a convenient text if the **Kinect** sensor is unplugged or not running yet. It should display a **green dot** and a convenient text otherwise.
- below this part, stands an horizontal bar of buttons allowing to switch between streams. The available streams are **color** (mandatory), **depth**, **infrared**, **body** (mandatory), and **body and color** streams. All this buttons should no be clickable if the **Kinect** is not available.  
- at last, the remaining of the window will display the content of the selected stream.  

# Steps to reproduce
This part is given as a set of advises to realize your application.  
I use some symbols in it:  
symbol | signification  
--- | --- 
💡 | be smart: this can obviously be done before practical works and without the kinect sensor!  
🎬 | evaluated at the end of all the practical works  
🚨 | mandatory if you want at least 10/20
🟢 | difficulty: low 
🟡 | difficulty: medium
🔴 | difficulty: high  
🔖 | quality (it's always better when it's beautiful 🥹)
> Note about difficulty:  
> It's obviously completely subjective...  

## 💡🚨🟢 Create the Window
- Create your repository
- Create a **WPF Application (.NET Framework)**
- Turn it to *SDK-style csproj* (especially if you want to use *MVVM Community Toolkit*, otherwise it is useless)
- Create the UI of your window (XAML)
## 🚨🟢 Kinect connection
- 💡 Read the Microsoft Kinect Samples (especially the **Color Basics-XAML** sample) and find how the **Kinect** sensor is managed
- Tip about how to encapsulate your **Kinect** sensor. You could write a ```KinectManager``` class like this one:  
```mermaid
classDiagram
class KinectManager {
    +/Status: bool
    +/StatusText: string
    +StartSensor()
    +StopSensor()
    -KinectSensor_IsAvailableChanged(sender, args)
}
class KinectSensor {
    +/IsOpen: bool
    +GetDefault()$ KinectSensor
    +~~event~~IsAvailableChanged
    +Open()
    +Close()
}
KinectSensor <-- KinectManager
```
This class could be able to start and stop the sensor and listen to the IsAvailableChanged event from the ```KinectSensor```. It would also give access to status information through ```Status``` and ```StatusText```.  
> TIP:  
> If you like MVVM, consider using it directly to get more points (read next sections below).  
- Use this class with your app:
  - to start the sensor if it is plugged, when the window loads
  - to stop the sensor when the window unloads
  - use the status feedbacks to update the upper part of the window
## 🚨🟢 Display Color Image Stream
- 💡 Read the Microsoft Kinect Samples (especially the **Color Basics-XAML** sample) and find how to get the color stream
- Create your own class ```ColorImageStream``` to retrieve and give access to the kinect color stream
- Beware of copy and paste! Keep only what's useful!
- Consider adding ```Start``` and ```Stop``` methods
- Update the window so that it displays the color stream when you click on the **Color** button.
## 🟡🔖 Use MVVM
The two previous mandatory sections can bring more points if you use MVVM (or better, **MVVM Community Toolkit**).  
If you do so, your code will be less cumbersome, simpler to write, read and plug to the window.  
If you are not found of MVVM, it will be a nightmare.  
Make your choice.
## 🟢 Display Depth Image Stream
It's exactly the same as for the Color Image Stream. Same difficulties, minor changes.  
So it is easy, it does not take a lot of time, but it does not bring more knowledges.  
This is why it is not mandatory, and bring less points.
## 🟢 Display IR Image Stream
It's exactly the same as for the Color Image Stream. Same difficulties, minor changes.  
So it is easy, it does not take a lot of time, but it does not bring more knowledges.  
This is why it is not mandatory, and bring less points.
## 🟡🔖 Architecture
- If you have prepared the three streams (but you can do it even with only one), you could decide to enhance your architecture by using inheritance.  
```mermaid
classDiagram
direction TB
class KinectStream {
    #Sensor : KinectSensor
    +Start()
    +Stop()
}

KinectStream --> "1" KinectManager : #KinectManager

KinectStream <|-- ColorImageStream 
KinectStream <|-- DepthImageStream 
KinectStream <|-- InfraredImageStream 
```
- Once it is done, you could make the change between one stream and another easier by using a factory:  
```mermaid
classDiagram
direction LR
class KinectStreamsFactory {
    +ctor(kinect: KinectManager)
    -streamFactory : Dictionary~KinectStreams, Func~KinectStream~~ 
    +this[stream: KinectStreams] : KinectStream
}

class KinectStreams {
    <<enum>>
    None
    Color
    Depth
    IR
}

KinectStreamsFactory --> "1" KinectManager
KinectStreamsFactory ..> KinectStreams
KinectStreamsFactory ..> KinectStream
KinectStream --> KinectManager
KinectStream <|-- ColorImageStream
KinectStream <|-- DepthImageStream
KinectStream <|-- InfraredImageStream
```
The idea here is to provide a collection of method to create and start ```KinectStream```s. These methods are ```Func<KinectStream>``` stored in a ```Dictionary``` that you could access through an indexer.  
- If you update now your window with this factory, now your code-behind (or better, your view model) should be very very simple and beautiful (🥹).  
  
## 🚨🟡 Display Body Stream
- 💡 Read the Microsoft Kinect Samples (especially the **Body Basics-XAML** sample) and find how to get the body stream. You will see that their sample is a little bit complex as they have decided to modify the image. I suggest we go in a simpler direction, by providing a canvas with ellipses representing the joints of the body, and lines representing the bones.  
- Beware of copy and paste! Keep only what's useful!
- Update the window so that it displays the body stream when you click on the **Body** button.
## 🟡🔖 Update and use the architecture
If you have used the advised architecture, use it to make your new ```BodyStream``` class a subclass of the ```KinectStream``` abstract class. You may need to change some things in the base class as the body stream uses a canvas.
## 🔴 Display Body and Color Streams in the meantime
If you still have some time, find a way to display simultaneously color and body streams!
## 🟡🔖 Update and use the architecture
Obviously, integrate it in your architecture if you have done one.
## 🟢🔖 Make a class library
As all your classes may be used by other applications, make a class library!

# Evaluation criteria
To earned points, remember that you must validate your knowledge with your teacher, during the practical work.


**Signification**  
symbol | signification  
--- | --- 
☢️ | if not respected => 0/20
🎬 | evaluated at the end of all the practical works  
🚨 | mandatory
🟢 | difficulty: low
🟡 | difficulty: medium
🔴 | difficulty: high  
🔖 | quality  
  

**Criteria**
@ | category | description | coeff 
--- | --- | --- | --- 
☢️ | | the repositories must be accessible by the teacher | ☢️ 
☢️ | | a .gitignore file must be use at the creation of the repository or at the first push | ☢️
🎬 | | all *class libraries* and *applications* build | 6 
🎬 | | *applications* run without any bug | 6 
🚨🟢 | XAML of the Window | | 1
🚨🟢 | Kinect Connection | Use of the Kinect sensor | 2
🚨🟢 | Kinect Connection | Use of the Kinect event | 6
🚨🟢🔖 | Kinect Connection | Encapsulation | 6
🚨🟢 | Kinect Connection | Connect when the Window is loaded | 4
🚨🟢 | Kinect Connection | Disconnect when the Window is unloaded | 2
🚨🟢 | Kinect Connection | Status feedback in the Window | 6
🔴🔖 | Kinect Connection | Using MVVM (commands for window loaded and unloaded) | 6
🟡🔖 | Kinect Connection | Using MVVM (status feedback) | 6
🚨🟢🔖 | Color Stream | Encapsulation - ColorImageStream class with **ONLY** necessary members (beware of mindless copy and paste) | 6
🚨🟢 | Color Stream | retrieving the frame and processing it | 6
🚨🟢 | Color Stream | start and stop acquisition | 6
🚨🟢 | Color Stream | updating the window (start when clicking the button) | 6
🚨🟢 | Color Stream | updating the window (display the color stream) | 6
🟡🔖 | Color Stream | Using MVVM (button click + commands) | 6
🟢🔖 | Color Stream | Using MVVM (data bind the color stream to the image) | 6
🎬🚨🟢 | Color Stream | *test* runs without any bug | 4 
🟢🔖 | Depth Stream | Encapsulation - DepthImageStream class with **ONLY** necessary members (beware of mindless copy and paste) | 1
🟢 | Depth Stream | retrieving the frame and processing it | 1
🟢 | Depth Stream | start and stop acquisition | 1
🟢 | Depth Stream | updating the window (start when clicking the button, stop when clicking another button) | 1
🟢 | Depth Stream | updating the window (display the color stream) | 1
🟡🔖 | Depth Stream | Using MVVM (button click + commands) | 1
🟢🔖 | Depth Stream | Using MVVM (data bind the depth stream to the image) | 1
🎬🟢 | Depth Stream | *test* runs without any bug | 1 
🟢🔖 | IR Stream | Encapsulation - InfraredImageStream class with **ONLY** necessary members (beware of mindless copy and paste) | 1
🟢 | IR Stream | retrieving the frame and processing it | 1
🟢 | IR Stream | start and stop acquisition | 1
🟢 | IR Stream | updating the window (start when clicking the button, stop when clicking another button) | 1
🟢 | IR Stream | updating the window (display the IR stream) | 1
🟡🔖 | IR Stream | Using MVVM (button click + commands) | 1
🟢🔖 | IR Stream | Using MVVM (data bind the infrared stream to the image) | 1
🎬🟢 | IR Stream | *test* runs without any bug | 1 
🟡🔖 | Architecture | Abstract class to factorize | 12
🔴🔖 | Architecture | Abstract class to factorize (using MVVM) | 6
🔴🔖 | Architecture | Factory to allow choosing between one stream or another | 12
🎬🔴🔖 | Architecture | *test* runs without any bug using the architecture and MVVM | 3 
🚨🟢🔖 | Body Stream | Encapsulation - BodyStream class with **ONLY** necessary members (beware of mindless copy and paste) | 6
🚨🟢 | Body Stream | retrieving the frame and processing it, retrieving the body of the closest user | 6
🚨🟢 | Body Stream | start and stop acquisition | 6
🚨🟡 | Body Stream | fill a Canvas with Ellipses and Lines corresponding to the skeleton | 6
🚨🟢 | Body Stream | updating the window (start when clicking the button, stop when clicking another button) | 6
🚨🟡 | Body Stream | updating the window (display the body stream) | 6
🔴🔖 | Body Stream | Using MVVM (button click + commands) | 1
🔴🔖 | Body Stream | Using MVVM (data bind the body stream to the image) | 6
🎬🚨🔴 | Body Stream | *test* runs without any bug | 4 
🔴🔖 | Architecture | Use the Body Stream with the abstract class and the Factory to allow choosing between one stream or another | 6
🎬🔴🔖 | Architecture | *test* runs without any bug using the architecture and MVVM | 3 
🟡🔖 | Body and Color Streams | Encapsulation - BodyAndColorStream class (be smart) | 4
🟡🔖 | Body and Color Streams | updating the window (start when clicking the button, stop when clicking another button) | 4
🔴🔖 | Body and Color Streams | updating the window (display the body stream) | 4
🔴🔖 | Body and Color Streams | Using MVVM (button click + commands) | 1
🔴🔖 | Body and Color Streams | Using MVVM (data bind the body and color streams to the image) | 4
🎬🔴🔖 | Body and Color Streams | *test* runs without any bug | 2 
🎬🔴🔖 | Architecture | *test* runs without any bug using the architecture and MVVM | 3 
🟢🔖 | Architecture | Create a class library | 6
🎬🟢 | Documentation | ReadMe, comments, wiki... | 4

> Some samples if you wonder...  
> - if you do only the 🚨 criteria, you get => 10/20
> - for the ```KinectManager``` (without MVVM) => 2,22 pts
> - for the Color Stream (without MVVM and without considering tests) => 2,56 pts
> - for depth and IR streams (without MVVM and without considering tests)  => 0,85 pts
> - for body stream (without MVVM and without considering tests) => 3,08 pts
> - for body & color streams (without MVVM and without considering tests) => 1,03 pt
> - architecture => 3,08 pts
> - if you use MVVM (everywhere when indicated) => 4,7 pts
> - if all your tests pass and if you have done documentation => 2,48 pts 

> Note :  
> Coefficients may change and are here only indicative  

---

[Home](./README.md)
| **Exercise 1 - Kinect Streams**
| [Exercise 2 - Introduction](./exo2_subject.md) 
| [Exercise 2 part 1 - Postures](./exo2_1_subject.md) 
| [Exercise 2 part 2 - Gestures](./exo2_2_subject.md) 
| [Exercise 2 part 3 - Mapping](./exo2_3_subject.md) 
| [Exercise 3 - Free App](./exo3_subject.md)

---

Copyright © 2023-2024 Marc Chevaldonné

--- 

While preparing these practical works, I was listening to...  

<table>
    <tr>
        <td>
            <img src="./images/trident.jpg" width="120"/>
        </td>
        <td>
            <div>
                <p><b>Trident</b></p>
                <p><i>McCoy Tyner</i> (1975)</p>
            </div>
        </td>
    </tr>
</table>
<table>
    <tr>
        <td>
            <img src="./images/sweet_return.jpg" width="120"/>
        </td>
        <td>
            <div>
                <p><b>Sweet Return</b></p>
                <p><i>Freddie Hubbard</i> (1983)</p>
            </div>
        </td>
    </tr>
</table>
<table>
    <tr>
        <td>
            <img src="./images/renaissance_man.jpg" width="120"/>
        </td>
        <td>
            <div>
                <p><b>Renaissance Man</b></p>
                <p><i>Jamaaladeen Tacuma</i> (1984)</p>
            </div>
        </td>
    </tr>
</table>
<table>
    <tr>
        <td>
            <img src="./images/back_in_the_usa.jpg" width="120"/>
        </td>
        <td>
            <div>
                <p><b>Back In The USA</b></p>
                <p><i>MC5</i> (1970)</p>
            </div>
        </td>
    </tr>
</table>
<table>
    <tr>
        <td>
            <img src="./images/blues_and_roots.jpg" width="120"/>
        </td>
        <td>
            <div>
                <p><b>Blues & Roots</b></p>
                <p><i>Charlie Mingus</i> (1960)</p>
            </div>
        </td>
    </tr>
</table>
<table>
    <tr>
        <td>
            <img src="./images/the_magic_of_ju-ju.jpg" width="120"/>
        </td>
        <td>
            <div>
                <p><b>The Magic of Ju-Ju</b></p>
                <p><i>Archie Shepp</i> (1967)</p>
            </div>
        </td>
    </tr>
</table>
<table>
    <tr>
        <td>
            <img src="./images/loaded.jpg" width="120"/>
        </td>
        <td>
            <div>
                <p><b>Loaded</b></p>
                <p><i>The Velvet Underground</i> (1970)</p>
            </div>
        </td>
    </tr>
</table>
<table>
    <tr>
        <td>
            <img src="./images/symphonie_fantastique.jpg" width="120"/>
        </td>
        <td>
            <div>
                <p><b>Symphonie Fantastique / Tristia</b></p>
                <p><i>Hector Berlioz - The Cleveland Orchestra & Chrosu - Pierre Boulez</i> (1830 - 1997)</p>
            </div>
        </td>
    </tr>
</table>