YassineKader commited on
Commit
adf1580
·
1 Parent(s): aa0c612

Upload folder using huggingface_hub

Browse files
Files changed (5) hide show
  1. Readme.md +67 -0
  2. config.json +398 -0
  3. model.bin +3 -0
  4. tokenizer.json +0 -0
  5. vocabulary.json +0 -0
Readme.md ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - ht
4
+ tags:
5
+ - audio
6
+ - automatic-speech-recognition
7
+ license: mit
8
+ library_name: ctranslate2
9
+ ---
10
+
11
+ # Whisper small model for CTranslate2
12
+
13
+ This repository contains the conversion of [YassineKader/whisper-small-haitian](https://huggingface.co/YassineKader/whisper-small-haitian) to the [CTranslate2](https://github.com/OpenNMT/CTranslate2) model format.
14
+
15
+ This model can be used in CTranslate2 or projects based on CTranslate2 such as [faster-whisper](https://github.com/guillaumekln/faster-whisper).
16
+
17
+ ## Example
18
+
19
+ ```python
20
+ import ctranslate2
21
+ import librosa
22
+ import transformers
23
+ from datetime import datetime
24
+ # Load and resample the audio file.
25
+ audio, _ = librosa.load("audio1.wav", sr=16000, mono=True)
26
+ # Compute the features of the first 30 seconds of audio.
27
+ processor = transformers.WhisperProcessor.from_pretrained("YassineKader/whisper-small-haitian")
28
+ inputs = processor(audio, return_tensors="np", sampling_rate=16000)
29
+ features = ctranslate2.StorageView.from_array(inputs.input_features)
30
+ # Load the model on CPU.
31
+ model = ctranslate2.models.Whisper("whisper-small-HT")
32
+ # Detect the language.
33
+ results = model.detect_language(features)
34
+ language, probability = results[0][0]
35
+ print("Detected language %s with probability %f" % (language, probability))
36
+ print(datetime.now())
37
+ # Describe the task in the prompt.
38
+ # See the prompt format in https://github.com/openai/whisper.
39
+ prompt = processor.tokenizer.convert_tokens_to_ids(
40
+ [
41
+ "<|startoftranscript|>",
42
+ language,
43
+ "<|transcribe|>",
44
+ "<|notimestamps|>", # Remove this token to generate timestamps.
45
+ ]
46
+ )
47
+ # Run generation for the 30-second window.
48
+ results = model.generate(features, [prompt])
49
+ transcription = processor.decode(results[0].sequences_ids[0])
50
+
51
+ print(datetime.now())
52
+ print(transcription)
53
+ ```
54
+
55
+ ## Conversion details
56
+
57
+ The original model was converted with the following command:
58
+
59
+ ```
60
+ ct2-transformers-converter --model guillaumekln/faster-whisper-small --output_dir faster-whisper-small-ht --copy_files tokenizer.json --quantization float32
61
+ ```
62
+
63
+ Note that the model weights are saved in FP16. This type can be changed when the model is loaded using the [`compute_type` option in CTranslate2](https://opennmt.net/CTranslate2/quantization.html).
64
+
65
+ ## More information
66
+
67
+ **For more information about the original model, see its [model card](https://huggingface.co/openai/whisper-small).**
config.json ADDED
@@ -0,0 +1,398 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alignment_heads": [
3
+ [
4
+ 6,
5
+ 0
6
+ ],
7
+ [
8
+ 6,
9
+ 1
10
+ ],
11
+ [
12
+ 6,
13
+ 2
14
+ ],
15
+ [
16
+ 6,
17
+ 3
18
+ ],
19
+ [
20
+ 6,
21
+ 4
22
+ ],
23
+ [
24
+ 6,
25
+ 5
26
+ ],
27
+ [
28
+ 6,
29
+ 6
30
+ ],
31
+ [
32
+ 6,
33
+ 7
34
+ ],
35
+ [
36
+ 6,
37
+ 8
38
+ ],
39
+ [
40
+ 6,
41
+ 9
42
+ ],
43
+ [
44
+ 6,
45
+ 10
46
+ ],
47
+ [
48
+ 6,
49
+ 11
50
+ ],
51
+ [
52
+ 7,
53
+ 0
54
+ ],
55
+ [
56
+ 7,
57
+ 1
58
+ ],
59
+ [
60
+ 7,
61
+ 2
62
+ ],
63
+ [
64
+ 7,
65
+ 3
66
+ ],
67
+ [
68
+ 7,
69
+ 4
70
+ ],
71
+ [
72
+ 7,
73
+ 5
74
+ ],
75
+ [
76
+ 7,
77
+ 6
78
+ ],
79
+ [
80
+ 7,
81
+ 7
82
+ ],
83
+ [
84
+ 7,
85
+ 8
86
+ ],
87
+ [
88
+ 7,
89
+ 9
90
+ ],
91
+ [
92
+ 7,
93
+ 10
94
+ ],
95
+ [
96
+ 7,
97
+ 11
98
+ ],
99
+ [
100
+ 8,
101
+ 0
102
+ ],
103
+ [
104
+ 8,
105
+ 1
106
+ ],
107
+ [
108
+ 8,
109
+ 2
110
+ ],
111
+ [
112
+ 8,
113
+ 3
114
+ ],
115
+ [
116
+ 8,
117
+ 4
118
+ ],
119
+ [
120
+ 8,
121
+ 5
122
+ ],
123
+ [
124
+ 8,
125
+ 6
126
+ ],
127
+ [
128
+ 8,
129
+ 7
130
+ ],
131
+ [
132
+ 8,
133
+ 8
134
+ ],
135
+ [
136
+ 8,
137
+ 9
138
+ ],
139
+ [
140
+ 8,
141
+ 10
142
+ ],
143
+ [
144
+ 8,
145
+ 11
146
+ ],
147
+ [
148
+ 9,
149
+ 0
150
+ ],
151
+ [
152
+ 9,
153
+ 1
154
+ ],
155
+ [
156
+ 9,
157
+ 2
158
+ ],
159
+ [
160
+ 9,
161
+ 3
162
+ ],
163
+ [
164
+ 9,
165
+ 4
166
+ ],
167
+ [
168
+ 9,
169
+ 5
170
+ ],
171
+ [
172
+ 9,
173
+ 6
174
+ ],
175
+ [
176
+ 9,
177
+ 7
178
+ ],
179
+ [
180
+ 9,
181
+ 8
182
+ ],
183
+ [
184
+ 9,
185
+ 9
186
+ ],
187
+ [
188
+ 9,
189
+ 10
190
+ ],
191
+ [
192
+ 9,
193
+ 11
194
+ ],
195
+ [
196
+ 10,
197
+ 0
198
+ ],
199
+ [
200
+ 10,
201
+ 1
202
+ ],
203
+ [
204
+ 10,
205
+ 2
206
+ ],
207
+ [
208
+ 10,
209
+ 3
210
+ ],
211
+ [
212
+ 10,
213
+ 4
214
+ ],
215
+ [
216
+ 10,
217
+ 5
218
+ ],
219
+ [
220
+ 10,
221
+ 6
222
+ ],
223
+ [
224
+ 10,
225
+ 7
226
+ ],
227
+ [
228
+ 10,
229
+ 8
230
+ ],
231
+ [
232
+ 10,
233
+ 9
234
+ ],
235
+ [
236
+ 10,
237
+ 10
238
+ ],
239
+ [
240
+ 10,
241
+ 11
242
+ ],
243
+ [
244
+ 11,
245
+ 0
246
+ ],
247
+ [
248
+ 11,
249
+ 1
250
+ ],
251
+ [
252
+ 11,
253
+ 2
254
+ ],
255
+ [
256
+ 11,
257
+ 3
258
+ ],
259
+ [
260
+ 11,
261
+ 4
262
+ ],
263
+ [
264
+ 11,
265
+ 5
266
+ ],
267
+ [
268
+ 11,
269
+ 6
270
+ ],
271
+ [
272
+ 11,
273
+ 7
274
+ ],
275
+ [
276
+ 11,
277
+ 8
278
+ ],
279
+ [
280
+ 11,
281
+ 9
282
+ ],
283
+ [
284
+ 11,
285
+ 10
286
+ ],
287
+ [
288
+ 11,
289
+ 11
290
+ ]
291
+ ],
292
+ "lang_ids": [
293
+ 50259,
294
+ 50260,
295
+ 50261,
296
+ 50262,
297
+ 50263,
298
+ 50264,
299
+ 50265,
300
+ 50266,
301
+ 50267,
302
+ 50268,
303
+ 50269,
304
+ 50270,
305
+ 50271,
306
+ 50272,
307
+ 50273,
308
+ 50274,
309
+ 50275,
310
+ 50276,
311
+ 50277,
312
+ 50278,
313
+ 50279,
314
+ 50280,
315
+ 50281,
316
+ 50282,
317
+ 50283,
318
+ 50284,
319
+ 50285,
320
+ 50286,
321
+ 50287,
322
+ 50288,
323
+ 50289,
324
+ 50290,
325
+ 50291,
326
+ 50292,
327
+ 50293,
328
+ 50294,
329
+ 50295,
330
+ 50296,
331
+ 50297,
332
+ 50298,
333
+ 50299,
334
+ 50300,
335
+ 50301,
336
+ 50302,
337
+ 50303,
338
+ 50304,
339
+ 50305,
340
+ 50306,
341
+ 50307,
342
+ 50308,
343
+ 50309,
344
+ 50310,
345
+ 50311,
346
+ 50312,
347
+ 50313,
348
+ 50314,
349
+ 50315,
350
+ 50316,
351
+ 50317,
352
+ 50318,
353
+ 50319,
354
+ 50320,
355
+ 50321,
356
+ 50322,
357
+ 50323,
358
+ 50324,
359
+ 50325,
360
+ 50326,
361
+ 50327,
362
+ 50328,
363
+ 50329,
364
+ 50330,
365
+ 50331,
366
+ 50332,
367
+ 50333,
368
+ 50334,
369
+ 50335,
370
+ 50336,
371
+ 50337,
372
+ 50338,
373
+ 50339,
374
+ 50340,
375
+ 50341,
376
+ 50342,
377
+ 50343,
378
+ 50344,
379
+ 50345,
380
+ 50346,
381
+ 50347,
382
+ 50348,
383
+ 50349,
384
+ 50350,
385
+ 50351,
386
+ 50352,
387
+ 50353,
388
+ 50354,
389
+ 50355,
390
+ 50356,
391
+ 50357
392
+ ],
393
+ "suppress_ids": [],
394
+ "suppress_ids_begin": [
395
+ 220,
396
+ 50257
397
+ ]
398
+ }
model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d1491f3c41cd8a048ad9c845aae1181aa4ca2a4c8629748fe7eddb793267840c
3
+ size 967072068
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
vocabulary.json ADDED
The diff for this file is too large to render. See raw diff