19 6 162

Huang Liang Hsun PRO

lianghsun

https://www.lianghsun.dev

AI & ML interests

Focused on applying deep learning in legal and scientific domains, with expertise in NLP and model fine-tuning.

Recent Activity

liked a model 7 days ago

itlwas/Llama-3.2-Taiwan-3B-Instruct-Q4_K_M-GGUF

liked a dataset 8 days ago

instruction-pretrain/ft-instruction-synthesizer-collection

posted an update 9 days ago

🖖 Let me introduce the work I've done over the past three months: 𝗟𝗹𝗮𝗺𝗮-𝟯.𝟮-𝗧𝗮𝗶𝘄𝗮𝗻-𝟯𝗕 and 𝗟𝗹𝗮𝗺𝗮-𝟯.𝟮-𝗧𝗮𝗶𝘄𝗮𝗻-𝟯𝗕-𝗜𝗻𝘀𝘁𝗿𝘂𝗰𝘁, now open-sourced on 🤗 Hugging Face. 𝗹𝗶𝗮𝗻𝗴𝗵𝘀𝘂𝗻/𝗟𝗹𝗮𝗺𝗮-𝟯.𝟮-𝗧𝗮𝗶𝘄𝗮𝗻-𝟯𝗕: This model is built on top of 𝗺𝗲𝘁𝗮-𝗹𝗹𝗮𝗺𝗮/𝗟𝗹𝗮𝗺𝗮-𝟯.𝟮-𝟯𝗕 with continual pretraining. The training dataset consists of a mixture of Traditional Chinese and multilingual texts in specific proportions, including 20B tokens of Traditional Chinese text. 𝗹𝗶𝗮𝗻𝗴𝗵𝘀𝘂𝗻/𝗟𝗹𝗮𝗺𝗮-𝟯.𝟮-𝗧𝗮𝗶𝘄𝗮𝗻-𝟯𝗕-𝗜𝗻𝘀𝘁𝗿𝘂𝗰𝘁: This is a fine-tuned conversational model based on the foundation model. This Llama-3.2-Taiwan open-source project is currently a one-person effort (yes, I did everything from text preparation — so exhausting!). If you're interested, feel free to join the Discord server for discussions. 🅱🅴🅽🅲🅷🅼🅰🆁🅺🅸🅽🅶 The evaluation was conducted using https://huggingface.co./datasets/ikala/tmmluplus, though the README page does not yet reflect the latest results. The performance is close to the previous versions, indicating that further improvements might require adding more specialized knowledge in the datasets. 🅰 🅲🅰🅻🅻 🅵🅾🆁 🆂🆄🅿🅿🅾🆁🆃 If anyone is willing to provide compute resources, it would be greatly appreciated to help this project continue and grow. 💪 --- 🏔️ Foundation model: https://huggingface.co./lianghsun/Llama-3.2-Taiwan-3B 🤖 Instruction model: https://huggingface.co./lianghsun/Llama-3.2-Taiwan-3B-Instruct ⚡ GGUF: https://huggingface.co./lianghsun/Llama-3.2-Taiwan-3B-Instruct-GGUF

View all activity

Organizations

lianghsun's activity

upvoted a collection about 1 month ago

chinese-dataset

Collection

收藏夹 • 9 items • Updated Mar 11, 2024 • 2

upvoted a paper about 2 months ago

Balancing Continuous Pre-Training and Instruction Fine-Tuning: Optimizing Instruction-Following in LLMs

Paper • 2410.10739 • Published Oct 14, 2024 • 2

upvoted a collection 2 months ago

Taide

Collection

Synthetic dataset generated by Taide • 4 items • Updated Aug 30, 2024 • 1

upvoted a collection 3 months ago

MobileLLM

Collection

Optimizing Sub-billion Parameter Language Models for On-Device Use Cases (ICML 2024) https://arxiv.org/abs/2402.14905 • 9 items • Updated Nov 27, 2024 • 104

upvoted 2 collections 4 months ago

Taiwan-Legal-Bench

Collection

This repository offers a dataset for evaluating legal models based on Taiwan’s laws, including legal questions, provisions, and case law. • 4 items • Updated Dec 9, 2024 • 1

Llama-3.2-Taiwan-Legal-SLM

Collection

Based on the lianghsun/Llama-3.2-Taiwan-*B model, the fine-tuning was conducted using datasets related to the laws and judgments of Taiwan. • 3 items • Updated Dec 9, 2024 • 1