jspr

/

miqurelian-120b

like 2

Have you noticed any problems due to mixing the rope scales:

  "rope_scaling": null,
  "rope_theta": 1000000,

vs:

 "rope_scaling": {
    "factor": 8.0,
    "type": "linear"
  },
  "rope_theta": 10000.0,

How does this compare to stock aurelian-v0.5-70b-rope8-32K and/or Goliath-120b for writing?