Stupid question: How do I use the model in transfomers.js
I am new to huggingface transformers and I am trying to use gpt-4o:
This is my code:
import { AutoTokenizer, AutoModel } from '@xenova/transformers';
async function generateResponse(inputText) {
const tokenizer = await AutoTokenizer.from_pretrained('Xenova/gpt-4o');
const model = await AutoModel.from_pretrained('Xenova/gpt-4o'); // ERROR
const tokens = tokenizer.encode(inputText);
console.log('Encoded tokens:', tokens);
const input = {
input_ids: tokens,
max_length: 100
};
const output = await model.generate(input);
const response = tokenizer.decode(output[0], { skip_special_tokens: true });
return response;
}
async function main() {
const inputText = 'Hello, how are you today?';
const response = await generateResponse(inputText);
console.log('GPT-4o Response:', response);
}
main();
However, it throws an error on the commented line up top:
file:///Users/TheFatPig/AdonisBrain/node_modules/@xenova/transformers/src/utils/hub.js:241
throw Error(${message}: "${remoteURL}".);
^
Error: Could not locate file: "https://huggingface.co./Xenova/gpt-4o/resolve/main/config.json".
at handleError (file:///Users/TheFatPig/AdonisBrain/node_modules/@xenova/transformers/src/utils/hub.js:241:11)
at getModelFile (file:///Users/TheFatPig/AdonisBrain/node_modules/@xenova/transformers/src/utils/hub.js:474:24)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async getModelJSON (file:///Users/TheFatPig/AdonisBrain/node_modules/@xenova/transformers/src/utils/hub.js:575:18)
at async loadConfig (file:///Users/TheFatPig/AdonisBrain/node_modules/@xenova/transformers/src/configs.js:46:16)
at async PretrainedConfig.from_pretrained (file:///Users/TheFatPig/AdonisBrain/node_modules/@xenova/transformers/src/configs.js:85:30)
at async AutoModel.from_pretrained (file:///Users/TheFatPig/AdonisBrain/node_modules/@xenova/transformers/src/models.js:5475:18)
at async generateResponse (file:///Users/TheFatPig/AdonisBrain/test.js:6:19)
at async main (file:///Users/TheFatPig/AdonisBrain/test.js:24:22)
I know I'm asking a stupid question, but I can't seem to fix this. Thanks in advance!
I understand. Thanks. One last question:
What is the tokenizer used for if the weights aren't available?
The tokenizer is used to perform "tokenization", which is a pre-processing step used to convert text input into numbers which can be understood by the model. See here and here for more information. As shown in the README, here is an example of running the tokenizer:
import { AutoTokenizer } from '@xenova/transformers';
const tokenizer = await AutoTokenizer.from_pretrained('Xenova/gpt-4o');
const tokens = tokenizer.encode('hello world'); // [24912, 2375]
Understood. Again, thank you