Tonic commited on
Commit
68fc76e
ยท
unverified ยท
1 Parent(s): eee3bd2

add instructions

Browse files
Files changed (1) hide show
  1. main.py +163 -0
main.py CHANGED
@@ -27,6 +27,166 @@ On ๐ŸŒGithub: [Tonic-AI](https://github.com/tonic-ai) & contribute to๐ŸŒŸ [Mult
27
  ๐Ÿค—Big thanks to Yuvi Sharma and all the folks at huggingface for the community grant ๐Ÿค—
28
  """
29
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
  hf_token = os.environ.get('HFTOKEN')
31
  if hf_token:
32
  login(token=hf_token)
@@ -275,6 +435,9 @@ with gr.Blocks() as demo:
275
  with gr.Column():
276
  with gr.Group():
277
  gr.Markdown(join_us)
 
 
 
278
 
279
  with gr.Row():
280
  with gr.Column():
 
27
  ๐Ÿค—Big thanks to Yuvi Sharma and all the folks at huggingface for the community grant ๐Ÿค—
28
  """
29
 
30
+ useage_instructions = """
31
+ ## Overview
32
+ JASCO is a powerful text-to-music generation system that allows you to create music using text descriptions and various controls including chords, drums, and melody. This guide explains how to use each feature of the interface.
33
+
34
+ ## Model Selection
35
+ Four different models are available:
36
+ 1. `facebook/jasco-chords-drums-400M` - Basic model with chord and drum support (400M parameters)
37
+ 2. `facebook/jasco-chords-drums-1B` - Enhanced model with chord and drum support (1B parameters)
38
+ 3. `facebook/jasco-chords-drums-melody-400M` - Model with melody support (400M parameters)
39
+ 4. `facebook/jasco-chords-drums-melody-1B` - Full-featured model with melody support (1B parameters)
40
+
41
+ ## Input Controls
42
+
43
+ ### 1. Text Description
44
+ - Enter a descriptive text about the music you want to generate
45
+ - Examples:
46
+ - "80s pop with groovy synth bass and electric piano"
47
+ - "Strings, woodwind, orchestral, symphony"
48
+ - "Jazz quartet with walking bass and smooth piano"
49
+
50
+ ### 2. Chord Progression
51
+ Format: `(Chord, Time), (Chord, Time), ...`
52
+ - Time is in seconds (0-10 seconds range)
53
+ - Example: `(C, 0.0), (D, 2.0), (F, 4.0), (Ab, 6.0), (Bb, 7.0), (C, 8.0)`
54
+
55
+ Supported chord types:
56
+ ```python
57
+ Basic Chords: C, D, E, F, G, A, B
58
+ Minor Chords: Cm, Dm, Em, Fm, Gm, Am, Bm
59
+ Seventh Chords: C7, D7, E7, F7, G7, A7, B7
60
+ Major Seventh: Cmaj7, Dmaj7, Emaj7, Fmaj7, Gmaj7, Amaj7, Bmaj7
61
+ Minor Seventh: Cm7, Dm7, Em7, Fm7, Gm7, Am7, Bm7
62
+ Flat Chords: Ab, Bb (and their variations)
63
+ Special: N (No chord/silence)
64
+ ```
65
+
66
+ ### 3. Drums Input
67
+ Two options for adding drums:
68
+ 1. File Upload:
69
+ - Select "file" in Drums Input Source
70
+ - Upload a WAV file containing drum patterns
71
+ - Recommended length: 2-4 bars
72
+
73
+ 2. Microphone Recording:
74
+ - Select "mic" in Drums Input Source
75
+ - Record drum patterns using your microphone
76
+ - Keep recordings short and rhythmic
77
+
78
+ ### 4. Melody Input
79
+ - Upload a melody salience matrix as a PyTorch tensor
80
+ - Format: Shape [n_melody_bins, T]
81
+ - File should be saved using `torch.save()`
82
+
83
+ ### 5. Generation Parameters
84
+
85
+ #### Classifier Free Guidance (CFG) Controls:
86
+ - CFG ALL: Controls overall adherence to input conditions (default: 1.25)
87
+ - Range: 1.0-3.0
88
+ - Higher values = stronger conditioning
89
+
90
+ - CFG TEXT: Controls text conditioning strength (default: 2.5)
91
+ - Range: 1.0-4.0
92
+ - Higher values = closer match to text description
93
+
94
+ #### ODE Parameters:
95
+ - ODE Solver: Choose between 'euler' and 'dopri5'
96
+ - euler: Faster, less accurate
97
+ - dopri5: Slower, more accurate
98
+
99
+ - ODE Tolerance: Numerical precision (default: 1e-4)
100
+ - Lower values = higher precision, slower generation
101
+
102
+ - Euler Steps: Number of steps for euler solver (default: 10)
103
+ - Higher values = more accurate, slower generation
104
+
105
+ ## Generation Process
106
+
107
+ 1. Select a model based on your needs:
108
+ - Use 400M models for faster generation
109
+ - Use 1B models for higher quality
110
+ - Choose melody-enabled models if using melody input
111
+
112
+ 2. Enter your text description
113
+
114
+ 3. Input chord progression:
115
+ ```
116
+ Example:
117
+ (C, 0.0), (Am, 2.5), (F, 5.0), (G, 7.5)
118
+ ```
119
+
120
+ 4. (Optional) Add drums via file upload or microphone
121
+
122
+ 5. (Optional) Upload melody matrix
123
+
124
+ 6. Adjust generation parameters or use defaults
125
+
126
+ 7. Click "Make Musix"
127
+
128
+ ## Output
129
+ - The system generates two variations of your music
130
+ - Each generation is 10 seconds long
131
+ - Output is provided as WAV files
132
+ - You can download or play directly in the interface
133
+
134
+ ## Tips for Best Results
135
+
136
+ 1. Text Descriptions:
137
+ - Be specific about instruments
138
+ - Include genre information
139
+ - Mention desired mood or style
140
+
141
+ 2. Chord Progressions:
142
+ - Use common progressions for better results
143
+ - Space chords evenly
144
+ - Include resolution points
145
+
146
+ 3. Drums:
147
+ - Use clean, clear drum patterns
148
+ - Avoid complex patterns for better results
149
+ - Keep volume levels consistent
150
+
151
+ 4. Memory Management:
152
+ - The interface caches models after first use
153
+ - Switch models only when necessary
154
+ - Clear browser cache if experiencing issues
155
+
156
+ ## Example Usage
157
+
158
+ ```python
159
+ # Example 1: Pop Music
160
+ Text: "Upbeat pop song with electric piano and synthesizer"
161
+ Chords: (C, 0.0), (Am, 2.5), (F, 5.0), (G, 7.5)
162
+ Model: facebook/jasco-chords-drums-400M
163
+
164
+ # Example 2: Orchestral
165
+ Text: "Epic orchestral piece with strong strings and brass"
166
+ Chords: (Cm, 0.0), (G, 3.0), (Bb, 6.0), (Cm, 8.0)
167
+ Model: facebook/jasco-chords-drums-melody-1B
168
+
169
+ # Example 3: Jazz
170
+ Text: "Smooth jazz quartet with walking bass and piano"
171
+ Chords: (Dmaj7, 0.0), (Em7, 2.5), (A7, 5.0), (Dmaj7, 7.5)
172
+ Model: facebook/jasco-chords-drums-1B
173
+ ```
174
+
175
+ ## Error Handling
176
+ - If generation fails, try:
177
+ 1. Simplifying chord progressions
178
+ 2. Reducing CFG values
179
+ 3. Using simpler text descriptions
180
+ 4. Checking input format compliance
181
+ 5. Refreshing the page
182
+
183
+ ## Performance Considerations
184
+ - First generation may be slower due to model loading
185
+ - Subsequent generations with same model are faster
186
+ - Higher parameter models (1B) require more memory
187
+ - Melody-enabled models may be slower
188
+ """
189
+
190
  hf_token = os.environ.get('HFTOKEN')
191
  if hf_token:
192
  login(token=hf_token)
 
435
  with gr.Column():
436
  with gr.Group():
437
  gr.Markdown(join_us)
438
+ with gr.Row():
439
+ with gr.Accordion(open=False, label="Useage Instructions"):
440
+ gr.Markdown(useage_instructions)
441
 
442
  with gr.Row():
443
  with gr.Column():