TF-Keras
TF Lite
sergiopaniego commited on
Commit
245f24b
·
1 Parent(s): 8279970

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +62 -6
README.md CHANGED
@@ -2,16 +2,17 @@
2
  license: apache-2.0
3
  ---
4
 
5
- # Tensorflow
6
-
7
- * Version: 2.7.0
8
- * TensorRT version: 7.2.2.1
9
- * Docker image: nvcr.io/nvidia/tensorflow:20.12-tf2-py3
10
  * GPU: NVIDIA GeForce 3090
11
  * CUDA: 11.6
12
  * Driver version: 510.54
 
13
 
 
14
 
 
 
 
 
15
 
16
  | Optimization | Model size (MB) | MSE | Inference time (s/frame) | Filename |
17
  | --------------------------------- | ---------------------- | ---------------------- | ---------------------- | --------------------------------- |
@@ -27,7 +28,17 @@ license: apache-2.0
27
  | Pruning preserving Quantization Aware | 1.5446319580078125 | 0.010758002372154884 | 0.0008252830505371093 | 28_04_pilotnet_pqat_model.tflite |
28
  | Sparsity and cluster preserving quantization aware training (PCQAT) | 1.5446319580078125 | 0.008262857163545972 | 0.0008286898136138916 | 28_04_pilotnet_pcqat_model.tflite |
29
 
30
- # TensorRT-Tensorflow
 
 
 
 
 
 
 
 
 
 
31
 
32
  | Optimization | Model size (MB) | MSE | Inference time (s/frame) | Folder |
33
  | --------------------------------- | ---------------------- | ---------------------- | ---------------------- | --------------------------------- |
@@ -35,6 +46,51 @@ license: apache-2.0
35
  | Float16 Quantization | 0.00390625 | 0.010798278900279191 | 0.00042218327522277834 | 24_04_pilotnet_tftrt_fp16 |
36
  | Int8 Quantization | 0.00390625 | 0.04791482252948612 | 0.0003384373188018799 | 14_06_pilotnet_tftrt_fp16 |
37
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
38
 
39
 
40
 
 
2
  license: apache-2.0
3
  ---
4
 
 
 
 
 
 
5
  * GPU: NVIDIA GeForce 3090
6
  * CUDA: 11.6
7
  * Driver version: 510.54
8
+ * Input shape (200,66,3)
9
 
10
+ # Tensorflow
11
 
12
+ * Tensorflow version: 2.7.0
13
+ * TensorRT version: 7.2.2.1
14
+ * Docker image: nvcr.io/nvidia/tensorflow:20.12-tf2-py3
15
+ * nvidia-tensorrt: 7.2.2.1
16
 
17
  | Optimization | Model size (MB) | MSE | Inference time (s/frame) | Filename |
18
  | --------------------------------- | ---------------------- | ---------------------- | ---------------------- | --------------------------------- |
 
28
  | Pruning preserving Quantization Aware | 1.5446319580078125 | 0.010758002372154884 | 0.0008252830505371093 | 28_04_pilotnet_pqat_model.tflite |
29
  | Sparsity and cluster preserving quantization aware training (PCQAT) | 1.5446319580078125 | 0.008262857163545972 | 0.0008286898136138916 | 28_04_pilotnet_pcqat_model.tflite |
30
 
31
+ TensorRT-Tensorflow:
32
+
33
+ To do inference:
34
+
35
+ ```
36
+ pip install nvidia-tensorrt===7.2.2.1
37
+ python3 -c "import tensorrt; print(tensorrt.__version__); assert tensorrt.Builder(tensorrt.Logger())"
38
+ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/python3.8/site-packages/tensorrt
39
+ python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
40
+ ```
41
+
42
 
43
  | Optimization | Model size (MB) | MSE | Inference time (s/frame) | Folder |
44
  | --------------------------------- | ---------------------- | ---------------------- | ---------------------- | --------------------------------- |
 
46
  | Float16 Quantization | 0.00390625 | 0.010798278900279191 | 0.00042218327522277834 | 24_04_pilotnet_tftrt_fp16 |
47
  | Int8 Quantization | 0.00390625 | 0.04791482252948612 | 0.0003384373188018799 | 14_06_pilotnet_tftrt_fp16 |
48
 
49
+ ---
50
+
51
+ # PyTorch
52
+
53
+
54
+ * PyTorch version: 1.13.1+cu116
55
+ * TensorRT version: 8.5.5
56
+ * Docker image: nvcr.io/nvidia/pytorch:22.12-py3
57
+ * torch-tensorrt: 1.3.0
58
+
59
+
60
+ | Optimization | Model size (MB) | MSE | Inference time (s/frame) | Filename |
61
+ | --------------------------------- | ---------------------- | ---------------------- | ---------------------- | --------------------------------- |
62
+ | Dynamic Range Quantization | 1.9493608474731445 | 0.012065857842182075 | 0.001480283498764038 | 28_04_dynamic_quan.pth |
63
+ | Static Quantization | 1.6071176528930664 | 0.012072610909984047 | 0.0007314345836639404 | 28_04_static_quan.pth |
64
+ | Quantization Aware Training | 1.6069536209106445 | 0.01109830549109022 | 0.0011710402965545653 | 28_04_quan_aware.pth |
65
+ | Local Prune | 6.122584342956543 | 0.010850968803449539 | 0.0014387350082397461 | 28_04_local_prune.pth |
66
+ | Global Prune | 6.122775077819824 | 0.010964057565769462 | 0.0014179635047912597 | 28_04_global_prune.pth |
67
+ | Prune + Quantization | 1.6067094802856445 | 0.010949893930274941 | 0.0011728739738464356 | 28_04_prune_quan.pth |
68
+
69
+ TensorRT-PyTorch:
70
+
71
+ To do inference:
72
+
73
+ ```
74
+ pip install torch-tensorrt==1.3.0
75
+ ```
76
+
77
+ | Optimization | Model size (MB) | MSE | Inference time (s/frame) | Filename |
78
+ | --------------------------------- | ---------------------- | ---------------------- | ---------------------- | --------------------------------- |
79
+ | Float32 Quantization | - | - | - | 28_04_trt_mod_float_28_04_float.jit.pt |
80
+ | Float16 Quantization | - | - | - | 28_04_trt_mod_float_28_04_half.jit.pt |
81
+ | Int8 Quantization | - | - | - | 28_04_trt_mod_float_28_04_int8.jit.pt |
82
+
83
+
84
+
85
+
86
+
87
+
88
+
89
+
90
+
91
+
92
+
93
+
94
 
95
 
96