Hugging Face's logo

Godot Engine Demos

non-profit
https://godotengine.org/
Activity Feed Request to join this org

AI & ML interests

None defined yet.

Recent Activity

thomwolfย  authored a paper 6 months ago
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
thomwolfย  authored a paper 10 months ago
StarCoder 2 and The Stack v2: The Next Generation
thomwolfย  authored a paper about 1 year ago
The Stack: 3 TB of permissively licensed source code
View all activity

godot-demo's activity

julien-cย 
posted an update 14 days ago
view post
Post
7570
After some heated discussion ๐Ÿ”ฅ, we clarify our intent re. storage limits on the Hub

TL;DR:
- public storage is free, and (unless blatant abuse) unlimited. We do ask that you consider upgrading to PRO and/or Enterprise Hub if possible
- private storage is paid above a significant free tier (1TB if you have a paid account, 100GB otherwise)

docs: https://huggingface.co./docs/hub/storage-limits

We optimize our infrastructure continuously to scale our storage for the coming years of growth in Machine learning, to the benefit of the community ๐Ÿ”ฅ

cc: @reach-vb @pierric @victor and the HF team
ยท
thomwolfย 
posted an update 16 days ago
view post
Post
4324
We are proud to announce HuggingFaceFW/fineweb-2: A sparkling update to HuggingFaceFW/fineweb with 1000s of ๐Ÿ—ฃ๏ธlanguages.

We applied the same data-driven approach that led to SOTA English performance in๐Ÿท FineWeb to thousands of languages.

๐Ÿฅ‚ FineWeb2 has 8TB of compressed text data and outperforms other multilingual datasets in our experiments.

The dataset is released under the permissive ๐Ÿ“œ ODC-By 1.0 license, and the ๐Ÿ’ป code to reproduce it and our evaluations is public.

We will very soon announce a big community project, and are working on a ๐Ÿ“ blogpost walking you through the entire dataset creation process. Stay tuned!

In the mean time come ask us question on our chat place: HuggingFaceFW/discussion

H/t @guipenedo @hynky @lvwerra as well as @vsabolcec Bettina Messmer @negar-foroutan and @mjaggi
  • 2 replies
ยท
thomwolfย 
posted an update 19 days ago
thomwolfย 
posted an update 21 days ago
julien-cย 
posted an update 25 days ago
view post
Post
2184
wow ๐Ÿ˜ฎ

INTELLECT-1 is the first collaboratively trained 10 billion parameter language model trained from scratch on 1 trillion tokens of English text and code.

PrimeIntellect/INTELLECT-1-Instruct
thomwolfย 
posted an update about 1 month ago
thomwolfย 
posted an update about 1 month ago
thomwolfย 
posted an update 2 months ago
view post
Post
4114
Parents in the 1990: Teach the kids to code
Parents now: Teach the kids to fix the code when it starts walking around ๐Ÿค–โœจ
  • 2 replies
ยท
thomwolfย 
authored a paper 6 months ago
thomwolfย 
posted an update 7 months ago
view post
Post
4555
[New crazy blog post alert] We are releasing an extensive blog post on the science of creating high quality web-scale datasets, detailing all the steps and learnings that came in our recent 15 trillion tokens ๐ŸทFineWeb release

Inspired by the distill.pub interactive graphics papers, we settled to write the most extensive, enjoyable and in-depth tech report we could draft on so prepare for a 45-mmin read with interactive graphics and all.

And it's not all, in this article we also introduce ๐Ÿ“šFineWeb-Edu a filtered subset of Common Crawl with 1.3T tokens containing only web pages with very high educational content. Up to our knowledge, FineWeb-Edu out-performs all openly release web-scale datasets by a significant margin on knowledge- and reasoning-intensive benchmarks like MMLU, ARC, and OpenBookQA

We also make a number of surprising observations on the "quality" of the internet it-self which may challenge some of the general assumptions on web data (not saying more, I'll let you draw your conclusions ;)

HuggingFaceFW/blogpost-fineweb-v1
  • 1 reply
ยท
julien-cย 
posted an update 7 months ago
view post
Post
5164
Hey it was good meeting you yesterday @MaziyarPanahi ๐Ÿ”ฅ

thanks @mishig for setting this up

Let's make the Hub as useful as possible for the community โค๏ธ
  • 1 reply
ยท
thomwolfย 
posted an update 8 months ago
view post
Post
4868
Is is time for the open-source AI robots revolution ๐Ÿš€?

With @haixuantao and @Leyo weโ€™ve been playing with a low-cost DJI robot controlled by three local open-source AI models (Whisper, Idefics2, Parler-TTS - all Apache2) and orchestrated by Dora-cs.

Links to find all the hardware/software we used in the demo:
- robot control framework โ€“ dora-rs: https://github.com/dora-rs/dora
- speech-to-text model โ€“ whisper: openai/whisper-base
- vision-text model โ€“ Idefics2: HuggingFaceM4/idefics2-8b-AWQ
- text-to-speech model โ€“ ParlerTTS mini: parler-tts/parler_tts_mini_v0.1
- robot: https://dji.com/robomaster-s1
- code gist: https://gist.github.com/haixuanTao/860e1740245dc2c8dd85b496150a9320
- Larger codebase: dora-rs/dora-idefics2
- laptop/pc: any with a recent GPU card (our has a RTX 4090)

Enjoy!
ยท
julien-cย 
posted an update 9 months ago
view post
Post
6864
text-generation-inference (TGI) is now fully open-source again!

Along with text-embeddings-inference.

We just switched both of those repos' license back to Apache 2. ๐Ÿ”ฅ
thomwolfย 
posted an update 9 months ago
view post
Post
3123
Very interesting model just released by MyShell: jetmoe/jetmoe-8b . It's a 8B-parameters MoE LLM so 2.2B active parameters, really efficient.

Main characteristics:
- impressive performances for its size (beating meta-llama/Llama-2-7b and huggyllama/llama-13b)
- combine Mixture of Attention heads (MoA) and Mixture of MLP Experts (MoE) โ€“ 8 experts with 2 being active for each token
- trained on a rather limited 1.25T tokens from publicly available datasets โ€“ training recipe follows the MiniCPM's two-phases training method => first time I see this for a 2B+ model
- $100k to train
- open weights - open sharing of recipes - open dataset - open code => โ™ก
- still interesting room to improve performances (be it only by training longer)

Links:
- report: https://research.myshell.ai/jetmoe
- model: jetmoe/jetmoe-8b
- code: https://github.com/myshell-ai/JetMoE

Note: I actually detailed all of the MiniCPM schedule, Mixture-of-expert (MoE) and many of the datasets used in this work in my recent little guide to building LLMs in 2024, so feel free to check it out if you want to learn more on these topics: https://www.youtube.com/watch?v=2-SPH9hIKT8
  • 1 reply
ยท
thomwolfย 
posted an update 9 months ago
view post
Post
1993
Little know gem: the Open-source Cookbook

A collection of notebooks for building practical AI applications using open-source tools and models: https://lnkd.in/e6m6Jmwu

Doc: https://lnkd.in/e3FE6TUq

Currently contains 16 notebooks in English (and some in Chinese):
1. Using LLM-as-a-judge ๐Ÿง‘โ€โš–๏ธ for an automated and versatile evaluation
2. Create a legal preference dataset
3. Suggestions for Data Annotation with SetFit in Zero-shot Text Classification
4. Implementing semantic cache to improve a RAG system
5. Building A RAG Ebook โ€œLibrarianโ€ Using LlamaIndex
6. Stable Diffusion Interpolation
7. Building A RAG System with Gemma, MongoDB and Open Source Models
8. Prompt Tuning with PEFT Library
9. Migrating from OpenAI to Open LLMs Using TGIโ€™s Messages API
10. Automatic Embeddings with TEI through Inference Endpoints
11. Simple RAG for GitHub issues using Hugging Face Zephyr and LangChain
12. Embedding multimodal data for similarity search using ๐Ÿค— transformers, ๐Ÿค— datasets and FAISS
13. Fine-tuning a Code LLM on Custom Code on a single GPU
14. RAG Evaluation Using Synthetic data and LLM-As-A-Judge
15. Advanced RAG on HuggingFace documentation using LangChain
16. Detecting Issues in a Text Dataset with Cleanlab
thomwolfย 
posted an update 9 months ago
view post
Post
5039
A Little guide to building Large Language Models in 2024

This is a post-recording of a 75min lecture I gave two weeks ago on how to train a LLM from scratch in 2024. I tried to keep it short and comprehensive โ€“ focusing on concepts that are crucial for training good LLM but often hidden in tech reports.

In the lecture, I introduce the students to all the important concepts/tools/techniques for training good performance LLM:
* finding, preparing and evaluating web scale data
* understanding model parallelism and efficient training
* fine-tuning/aligning models
* fast inference

There is of course many things and details missing and that I should have added to it, don't hesitate to tell me you're most frustrating omission and I'll add it in a future part. In particular I think I'll add more focus on how to filter topics well and extensively and maybe more practical anecdotes and details.

Now that I recorded it I've been thinking this could be part 1 of a two-parts series with a 2nd fully hands-on video on how to run all these steps with some libraries and recipes we've released recently at HF around LLM training (and could be easily adapted to your other framework anyway):
*datatrove for all things web-scale data preparation: https://github.com/huggingface/datatrove
*nanotron for lightweight 4D parallelism LLM training: https://github.com/huggingface/nanotron
*lighteval for in-training fast parallel LLM evaluations: https://github.com/huggingface/lighteval

Here is the link to watch the lecture on Youtube: https://www.youtube.com/watch?v=2-SPH9hIKT8
And here is the link to the Google slides: https://docs.google.com/presentation/d/1IkzESdOwdmwvPxIELYJi8--K3EZ98_cL6c5ZcLKSyVg/edit#slide=id.p

Enjoy and happy to hear feedback on it and what to add, correct, extend in a second part.
  • 2 replies
ยท
julien-cย 
posted an update 9 months ago
view post
Post
3161
Very glad to welcome @josefprusa , pioneer of 3D printing and open source hardware, founder of https://www.prusa3d.com/, to the HF Hub ๐Ÿ‘‹

AI applied to 3D printing could be big.
  • 1 reply
ยท
thomwolfย 
posted an update 9 months ago
julien-cย 
posted an update 10 months ago
julien-cย 
posted an update 10 months ago
view post
Post
What if you could casually access your remote GPU in HF Spaces from the comfort of your local VSCode ๐Ÿคฏ
ยท