fix tokenizer save_pretrained method

#32

by katuni4ka - opened May 20

base: refs/heads/main

←

from: refs/pr/32

Discussion Files changed

+175

-1

katuni4ka

May 20

tokenizer.save_pretrained methos failed due to imposibility serialize into json some config values (add_special_tokens, that is represented by method in tokenizer and present in init kwargs)

fix tokenizer save_pretrained method927e4b7c

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Ready to merge

This branch is ready to get merged automatically.

· Sign up or log in to comment