Update README.md

Changes to README.md

README.md

@@ -1,3 +1,90 @@
 ---
 license: mit
+language:
+- en
 ---
+# CLIP ViT-L/14 in OpenVINO™ format
+
+## Original model details
+
+The CLIP model was developed by OpenAI to learn about what contributes to robustness in computer vision tasks. The model was also developed to test the ability of models to generalize to arbitrary image classification tasks in a zero-shot manner. It was not developed for general model deployment - to deploy models like CLIP, researchers will first need to carefully study their capabilities in relation to the specific context they’re being deployed within.
+
+## Model type
+The model uses a ViT-L/14 Transformer architecture as an image encoder and uses a masked self-attention Transformer as a text encoder. These encoders are trained to maximize the similarity of (image, text) pairs via a contrastive loss.
+
+## OpenVINO™ optimization
+
+To increase the efficiency of the model during inference, we utilized the OpenVINO™ toolkit for optimization. The table below showcases the inference time improvements achieved with OpenVINO™ compared to the original PyTorch implementation:
+
+| Metric             |   PyTorch Inference Time (sec) |   OpenVINO™ Inference Time (sec) |   Similarity |
+|:-------------------|-------------------------------:|---------------------------------:|-------------:|
+| mean               |                           0.52 |                            0.46  |            1 |
+| std                |                           0.11 |                            0.09  |            0 |
+| min                |                           0.39 |                            0.36  |            1 |
+| max                |                           0.70 |                            0.62  |            1 |
+
+OpenVINO offers a 1.12x speedup in inference time compared to PyTorch. It was measured on same image in 100 iterations on Intel(R) Xeon(R) CPU @ 2.20GHz (CPU family: 6, Model: 79).
+
+
+The results indicate that the OpenVINO™ optimization provides a consistent improvement in inference time while maintaining the same level of accuracy (as indicated by the similarity score).
+
+## Usage
+
+You can utilize this optimized model for faster inferences in environments where time is a critical factor. Ensure you have the necessary libraries and dependencies installed to leverage the usage of OpenVINO™.
+
+```bash
+pip install transformers huggingface_hub openvino-dev
+```
+
+Then use it for inference:
+
+```python
+import os
+
+import numpy as np
+from PIL import Image
+from huggingface_hub import snapshot_download
+from openvino.runtime import Core
+from scipy.special import softmax
+from transformers import CLIPProcessor
+
+# Download the OV model
+ov_path = snapshot_download(repo_id="scaleflex/clip-vit-large-patch14-openvino")
+# Load preprocessor for model input
+processor = CLIPProcessor.from_pretrained("scaleflex/clip-vit-large-patch14-openvino")
+ov_model_xml = os.path.join(ov_path, "clip-vit-large-patch14.xml")
+
+image = Image.open("face.png")  # download this example image: http://sample.li/face.png
+input_labels = [
+    "businessman",
+    "dog playing in the garden",
+    "beautiful woman",
+    "big city",
+    "lake in the mountain",
+]
+text_descriptions = [f"This is a photo of a {label}" for label in input_labels]
+inputs = processor(
+    text=text_descriptions, images=[image], return_tensors="pt", padding=True
+)
+
+# Create OpenVINO core object instance
+core = Core()
+
+ov_model = core.read_model(model=ov_model_xml)
+# Compile model for loading on device
+compiled_model = core.compile_model(ov_model)
+# Obtain output tensor for getting predictions
+logits_per_image_out = compiled_model.output(0)
+# Run inference on preprocessed data and get image-text similarity score
+ov_logits_per_image = compiled_model(dict(inputs))[logits_per_image_out]
+# Perform softmax on score
+probs = softmax(ov_logits_per_image, axis=1)
+max_index = np.argmax(probs)
+
+# Use the index to get the corresponding label
+label_with_max_prob = input_labels[max_index]
+print(
+    f"The label with the highest probability is: '{label_with_max_prob}' with a probability of {probs[0][max_index] * 100:.2f}%"
+)
+# The label with the highest probability is: 'beautiful woman' with a probability of 97.87%
+```