beomi commited on
Commit
c4d83c0
โ€ข
1 Parent(s): 4f04d1e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -1
README.md CHANGED
@@ -30,6 +30,11 @@ Llama-3-KoEn-8B-xtuner-llava-preview ๐ŸŒ‹ is Korean based MutliModal based on Ll
30
 
31
  ![Cat walking on frozen Han-River, Seoul](https://cdn-uploads.huggingface.co/production/uploads/5e56829137cb5b49818287ea/NWfoArWI4UPAxpEnolkwT.jpeg)
32
 
 
 
 
 
 
33
  ```python
34
  import requests
35
  from PIL import Image
@@ -43,6 +48,7 @@ model = LlavaForConditionalGeneration.from_pretrained(
43
  model_id,
44
  torch_dtype='auto',
45
  device_map='auto',
 
46
  )
47
 
48
  processor = AutoProcessor.from_pretrained(model_id)
@@ -65,7 +71,7 @@ inputs = processor(prompt, raw_image, return_tensors='pt').to(0, torch.float16)
65
  output = model.generate(**inputs, max_new_tokens=400, do_sample=True, eos_token_id=terminators,)
66
  print(processor.decode(output[0][2:], skip_special_tokens=False))
67
 
68
- # --- Example Output ---
69
  user<|end_header_id|>
70
 
71
  <image>
@@ -74,5 +80,13 @@ user<|end_header_id|>
74
  ์ด ์ด๋ฏธ์ง€์—๋Š” ๊ณ ์–‘์ด ํ•œ ๋งˆ๋ฆฌ๊ฐ€ ๊ฐ•๋ฌผ ์œ„๋ฅผ ๊ฑธ์–ด๊ฐ€๋Š” ๋ชจ์Šต์ด ๋ณด์—ฌ์ง‘๋‹ˆ๋‹ค. ๊ณ ์–‘์ด๋Š” ๊ฐ•๋ฌผ์˜ ์ž”๋ฌผ๊ฒฐ์— ๋ฏธ๋„๋Ÿผ์„ ํƒ€๊ณ  ๊ฐ• ๊ฐ€๋กœ๋ฅผ ์ง€๋‚˜๋Š” ๋ฐ ๋Šฅ์ˆ™ํ•˜๊ฒŒ ๋ณด์ž…๋‹ˆ๋‹ค. ๊ณ ์–‘์ด์˜ ๋ฐœ์€ ๊ฐ•๋ฌผ๋กœ ์ž˜ ๋“ค์–ด๊ฐ€, ๊ทธ๊ฒƒ์„ ์ฆ๊ธฐ๋ฉฐ ๊ฑธ์–ด๊ฐ‘๋‹ˆ๋‹ค.
75
 
76
  ๋˜ํ•œ ์ด ์ด๋ฏธ์ง€๋„ ์Œ์„ฑ ๋…น์Œ์„ ํ•˜๊ฑฐ๋‚˜ ๋…นํ™”๋œ ์ž๋ฃŒ๋กœ ์ œ์ž‘๋˜์—ˆ์œผ๋ฉฐ, ์ฃผ๋กœ ๊ณ ์–‘์ด์˜ ๋ชจ์Šต์„ ๊ฐ•ํ•˜๊ฒŒ ๋ณด์—ฌ์ค๋‹ˆ๋‹ค. ์†Œ๋ฆฌ ํšจ๊ณผ๋„ ์—ฌ๋Ÿฌ ๊ฐ€์ง€๋กœ ์ถ”๊ฐ€ํ•˜์—ฌ ๊ณ ์–‘์ด์˜ ์Šคํ† ๋ฆฌ๋ฅผ ๋‹ค์–‘ํ•˜๊ฒŒ ์ „๋‹ฌํ•ฉ๋‹ˆ๋‹ค. ๊ฐ•๋ฌผ์€ ์ž”๋ฌผ๊ฒฐ์„ ๋‚˜ํƒ€๋‚ด๋ฉฐ ๊ฐ•๋ฌผ ์œ„๋ฅผ ๊ฑท๋Š” ๊ณ ์–‘์ด์˜ ๋ชจ์Šต์„ ๋”์šฑ ๊ฐ•๋ ฌํ•˜๊ฒŒ ๊ฐ•์กฐํ•˜๊ธฐ ์œ„ํ•ด ์ž”๋ฌผ๊ฒฐ์„ ํ†ตํ•ด ๋” ๋””ํ…Œ์ผํ•œ ์žฅ๋ฉด์„ ๋ณด์—ฌ์ค๋‹ˆ๋‹ค.<|eot_id|>
 
 
 
 
 
 
 
 
77
  ```
78
 
 
30
 
31
  ![Cat walking on frozen Han-River, Seoul](https://cdn-uploads.huggingface.co/production/uploads/5e56829137cb5b49818287ea/NWfoArWI4UPAxpEnolkwT.jpeg)
32
 
33
+ > Two version recommended
34
+ >
35
+ > v1. `revision='a38aac3'`: Basic ChatVector
36
+ > v2. `revision='4f04d1e'`: Model diff based merging(ref. https://huggingface.co/blog/maywell/llm-feature-transfer)
37
+
38
  ```python
39
  import requests
40
  from PIL import Image
 
48
  model_id,
49
  torch_dtype='auto',
50
  device_map='auto',
51
+ revision='a38aac3', # 'a38aac3' for basic ChatVector, '4f04d1e' for Model diff based merging(ref. https://huggingface.co/blog/maywell/llm-feature-transfer)
52
  )
53
 
54
  processor = AutoProcessor.from_pretrained(model_id)
 
71
  output = model.generate(**inputs, max_new_tokens=400, do_sample=True, eos_token_id=terminators,)
72
  print(processor.decode(output[0][2:], skip_special_tokens=False))
73
 
74
+ # --- Example Output [Chat Vector] ---
75
  user<|end_header_id|>
76
 
77
  <image>
 
80
  ์ด ์ด๋ฏธ์ง€์—๋Š” ๊ณ ์–‘์ด ํ•œ ๋งˆ๋ฆฌ๊ฐ€ ๊ฐ•๋ฌผ ์œ„๋ฅผ ๊ฑธ์–ด๊ฐ€๋Š” ๋ชจ์Šต์ด ๋ณด์—ฌ์ง‘๋‹ˆ๋‹ค. ๊ณ ์–‘์ด๋Š” ๊ฐ•๋ฌผ์˜ ์ž”๋ฌผ๊ฒฐ์— ๋ฏธ๋„๋Ÿผ์„ ํƒ€๊ณ  ๊ฐ• ๊ฐ€๋กœ๋ฅผ ์ง€๋‚˜๋Š” ๋ฐ ๋Šฅ์ˆ™ํ•˜๊ฒŒ ๋ณด์ž…๋‹ˆ๋‹ค. ๊ณ ์–‘์ด์˜ ๋ฐœ์€ ๊ฐ•๋ฌผ๋กœ ์ž˜ ๋“ค์–ด๊ฐ€, ๊ทธ๊ฒƒ์„ ์ฆ๊ธฐ๋ฉฐ ๊ฑธ์–ด๊ฐ‘๋‹ˆ๋‹ค.
81
 
82
  ๋˜ํ•œ ์ด ์ด๋ฏธ์ง€๋„ ์Œ์„ฑ ๋…น์Œ์„ ํ•˜๊ฑฐ๋‚˜ ๋…นํ™”๋œ ์ž๋ฃŒ๋กœ ์ œ์ž‘๋˜์—ˆ์œผ๋ฉฐ, ์ฃผ๋กœ ๊ณ ์–‘์ด์˜ ๋ชจ์Šต์„ ๊ฐ•ํ•˜๊ฒŒ ๋ณด์—ฌ์ค๋‹ˆ๋‹ค. ์†Œ๋ฆฌ ํšจ๊ณผ๋„ ์—ฌ๋Ÿฌ ๊ฐ€์ง€๋กœ ์ถ”๊ฐ€ํ•˜์—ฌ ๊ณ ์–‘์ด์˜ ์Šคํ† ๋ฆฌ๋ฅผ ๋‹ค์–‘ํ•˜๊ฒŒ ์ „๋‹ฌํ•ฉ๋‹ˆ๋‹ค. ๊ฐ•๋ฌผ์€ ์ž”๋ฌผ๊ฒฐ์„ ๋‚˜ํƒ€๋‚ด๋ฉฐ ๊ฐ•๋ฌผ ์œ„๋ฅผ ๊ฑท๋Š” ๊ณ ์–‘์ด์˜ ๋ชจ์Šต์„ ๋”์šฑ ๊ฐ•๋ ฌํ•˜๊ฒŒ ๊ฐ•์กฐํ•˜๊ธฐ ์œ„ํ•ด ์ž”๋ฌผ๊ฒฐ์„ ํ†ตํ•ด ๋” ๋””ํ…Œ์ผํ•œ ์žฅ๋ฉด์„ ๋ณด์—ฌ์ค๋‹ˆ๋‹ค.<|eot_id|>
83
+
84
+ # --- Example Output [Model diff based merging] ---
85
+ user<|end_header_id|>
86
+
87
+ <image>
88
+ ์ด ์ด๋ฏธ์ง€์— ๋Œ€ํ•ด์„œ ์„ค๋ช…ํ•ด์ฃผ์„ธ์š”.<|eot_id|><|start_header_id|>assistant<|end_header_id|>
89
+
90
+ ์ด ์ด๋ฏธ์ง€์—๋Š” ํ•œ๊ตญ์–ด ์ž๋ง‰๊ณผ ํ•จ๊ป˜ ๊ณ ์–‘์ด๊ฐ€ ๋ฌผ์— ๋ฐœ์„ ๋””๋””๊ณ  ๊ฑท๋Š” ๋ชจ์Šต์ด ๋‹ด๊ฒจ ์žˆ์Šต๋‹ˆ๋‹ค. ๊ณ ์–‘์ด๋Š” ์˜ค๋ฅธ์ชฝ ๋ฐœ์„ ๋ฌผ์— ๋‹ด๊ทธ๊ณ  ๊ฑท๋Š” ์ค‘์ด๋ฉฐ, ํ•œ๊ตญ์–ด ์ž๋ง‰์€ "๊ณ ์–‘์ด๋Š” ๋ฌผ์„ ์ข‹์•„ํ•ฉ๋‹ˆ๋‹ค"๋ผ๋Š” ๋ฌธ์žฅ์„ ํฌํ•จํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ์ด ์ž๋ง‰์€ ๊ณ ์–‘์ด๊ฐ€ ๋ฌผ์„ ์ข‹์•„ํ•˜๋Š” ๊ฒƒ์„ ๊ฐ•์กฐํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.<|eot_id|>
91
  ```
92