---
license: mit
language:
- en
metrics:
- accuracy
library_name: keras
pipeline_tag: video-classification
tags:
- Body-Language
- MediaPipe
- OpenCv
- .tflite
- .pkl
---
# Body-Language-Detection-with-MediaPipe-and-OpenCV

**This Jupyter Notebook (IPython Notebook) provides the code and instructions for implementing body language detection using [MediaPipe](https://github.com/google/mediapipe) and [OpenCV](https://github.com/opencv/opencv). This innovative tool incorporates two distinct models to achieve its functionality, providing users with a comprehensive approach to body language analysis.**
 * [Scikit-Learn (.pkl)](https://huggingface.co./ThisIs-Developer/Body-Language-Detection-with-MediaPipe-and-OpenCV#1-scikit-learn-pkl)
 * [TensorFlow-Keras (.tflite)](https://huggingface.co./ThisIs-Developer/Body-Language-Detection-with-MediaPipe-and-OpenCV#2-tensorflow-keras-tflite)
 *  [Features ⭐](https://huggingface.co./ThisIs-Developer/Body-Language-Detection-with-MediaPipe-and-OpenCV#features-%E2%AD%90)

## 1. Scikit-Learn (.pkl)
The first model is built using **Scikit-Learn** and is stored in a **.pkl (Python Pickle) format**.
1. It employs pipelines to encapsulate preprocessing and modeling steps for multiple algorithms.
    ```python
   pipelines = {
    'lr':make_pipeline(StandardScaler(), LogisticRegression(max_iter=5000)),
    'rc':make_pipeline(StandardScaler(), RidgeClassifier()),
    'rf':make_pipeline(StandardScaler(), RandomForestClassifier()),
    'gb':make_pipeline(StandardScaler(), GradientBoostingClassifier()),
    }
    ```

2. It systematically trains and evaluates different models using accuracy as a metric.
   ```output
    lr 0.995260663507109
    rc 0.985781990521327
    rf 0.9881516587677726
    gb 0.9928909952606635
    ```
3. It saves the best-performing model for later use using pickle.
   ```python
    with open('body_language.pkl', 'wb') as f:
    pickle.dump(fit_models['rf'], f)
   ```
## 2. TensorFlow-Keras (.tflite)

The second model is built using **TensorFlow-Keras** and is stored in a **TensorFlow Lite (.tflite) format**. 
1. It Builds and compiles a neural network model for classification.
```python
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])
```
2. It trains the model with relevant metrics. 
3. It converts and saves the model in TensorFlow Lite format for mobile deployment.
   ```python
    converter = tf.lite.TFLiteConverter.from_keras_model(model)
    tflite_model = converter.convert()
    open("body_language.tflite", "wb").write(tflite_model)
   ```
## Features ⭐
### 1. Create the training dataset using both a **Webcam** and **recording video data (.mp4)**, extracting relevant frames, and annotating those frames with corresponding labels.
#### View Folder: [Video Decoder](https://github.com/ThisIs-Developer/Body-Language-Detection-with-MediaPipe-and-OpenCV/tree/main/Video%20Decoder)
#### Modify the codef for MP4:
```python
class_name = "Happy"
# Replace 'path_to_your_video_file' with the actual path to your video file
cap = cv2.VideoCapture('path_to_your_video_file') 
# Initiate holistic model
with mp_holistic.Holistic(min_detection_confidence=0.5, min_tracking_confidence=0.5) as holistic:
    
    while cap.isOpened():
        ret, frame = cap.read()
```
### 2. Trained models to recognize 10 distinct body language and facial expression categories, enabling the automated recognition of emotions and gestures in videos.
#### Class Labels
1. Happy
2. Sad
3. Angry
4. Surprised
5. Confused
6. Tension
7. Surprised
8. Excited
9. Pain
10. Depressed
### 3. Visual representation of different emotional expressions, with each expression depicted in a separate chart or plot using the Matplotlib library in Python.
#### Pie plot
![image](https://github.com/ThisIs-Developer/Body-Language-Detection-with-MediaPipe-and-OpenCV/assets/109382325/9562c572-fe44-4982-aa8b-9a9e7a241f52)
#### Bar plot
![image](https://github.com/ThisIs-Developer/Body-Language-Detection-with-MediaPipe-and-OpenCV/assets/109382325/b1882a70-f2f8-4f38-ae06-8acfbce5e30f)
#### Horizontal bar plot
![image](https://github.com/ThisIs-Developer/Body-Language-Detection-with-MediaPipe-and-OpenCV/assets/109382325/7585391a-8c02-4c8a-87ea-cc4771f6fc09)
#### Horizontal bar plot in creasing order of sizes
![image](https://github.com/ThisIs-Developer/Body-Language-Detection-with-MediaPipe-and-OpenCV/assets/109382325/a1776f7f-994d-4fca-97c2-22e03a4f4465)
#### Horizontal bar plot with increasing order of sizes
![image](https://github.com/ThisIs-Developer/Body-Language-Detection-with-MediaPipe-and-OpenCV/assets/109382325/4f24a433-daac-4024-90ff-34f00f9c2825)