I haven't pushed a new version to HF yet, but i'll do that in a min
https://github.com/Ktiseos-Nyx/Dataset-Tools
Oh <_< that
I gotta fix that issue i have that too
For now, I've rewritten it so that it only works with HfApi. The speed should have increased. I'll make more detailed improvements tomorrow.๐ช
https://github.com/John6666cat/sdwebui-hfbackup/tree/Ver0.4
Feel free to merge request to ver0.4 when you're ready - i dont know how to add you to do the repo for this but you're welcome to be added when i figure it the EFF out XD
Oh sweet!
OH! Got the extension code cleaned up a bit, i don't run A1111 in HF, but if someone wants to sponsor this to be installed and TESTED - because it uses HF hub, people are MORE than welcome to use it in their on HF installations. I dont know if it'l work right because of coding, but y'know - i haven't been on A1111 to see if it works yet today.
https://huggingface.co./Duskfallcrew/sdwebui-hfbackup
https://github.com/duskfallcrew/sdwebui-hfbackup/tree/Ver0.4
Also, like honestly i'm OK with people poking holes in my code, i don't program, i tinker XD
Oh you're welcome to check out the code, and suggest improvements.
This is my running tool i've been using, i started doing an a111 via claude, but the thing is i don't program - I know a TINY BIT AROUND LOGIC because of web design and like early learning of programming before python existed.. .but i'm a designer jim, (star trek reference) not a programmer.
I used google to fix it up, the new gemini one, it's ... actually QUITE easier than gpt or claude for programming, and it's got a FAIRLY GOOD wide "freebie" range, like you get 1million lines of code or text with it, and it's REALLY good.
one of those things that, Llama isn't BAD but it's more for like general use, and i know that sadly gemini isn't really open source, but right now, someone like me -- needs something THAT BEEFY :D
The colab one SHOULD WORK because i actually used it, uploaeded an AI song of mine to one of my personal repositories, i made that tool go on personal rather than E&D -- just cause it's on personal rather than the E&D organization on github.
"...YNTEC was largely doing backups of diffusers, which is valid in itself, but sometimes that lead to clashes with model creators on Civitai..."
I agree for the part on the CIVITAI.COM. But Yntec also really care about author license, as we discussed on that subject.
I disagree for the part on was largely doing backups of diffusers. Because Yntec always fine-tuned them with a default resolution at 768px instead of 512px, and either VAE baked-in/either LoRAs fusions, I need to say Yntec models are more creative than the original are. Yntec is a passionate person, I mean that he care attention to precise details on each of his models (either conversions from LoRAs either conversions from Diffusers), just try his models, then try originals. With DEFAULT settings not in WebUI, and you will see a great improvements over each original models.
Try testing LadyNostalgia or even Gacha from Yntec, you will certainly be more than pleased to the output result!Edit : typo fix on my orthograph xD
Yea the hard part is a lot of people didn't agree, i don't mind - i think i'd found some of mine backed up, and converting to diffusers meant other sites had access that a LOT OF US DIDNT KNOW about - but i do know diffuesers saves people time and money and i'm ok with that.
Now if anyone catches CalypsoCrunchies ..... i know nothing
Yea i'll DEF consider if my budget can handle it do Enterprise for at least E&D - right now i'm juggling several small income things (none of which can really be classified as a business) and being that i don't really have a job it'll be depending on how i juggle stuff.
@Yntec will perhaps never come back again... given the way he put all his models offline... ๐ต. And it's a huge lost of precious models !
By the way, I'm already at 25GB used on my 500 GB of free storage... Just with 8 models lol. ๐... 500 GB is really not a sustainable value for hardware capacity when it comes to store on HF these GB models!
I think, 1TB for free storage and unlimited for PRO would be great!
(if one day, it come to unlimited storage for PRO, I sign immediately for a PRO account each month ๐).If not... What is the point then.
But.. @Yntec took down his 449 models.. He was really upset. Hey come back lovely @Yntec ๐.
YNTEC was largely doing backups of diffusers, which is valid in itself, but sometimes that lead to clashes with model creators on Civitai...
Public storage- y'all ... HF are you nuts? LOL - Thank you for the effort, starting to wonder if i should be kind and at least pay for enterprise if it ain't expensive on E&D -- I dont know if you can classify half the stuff i do as research -
But the idea that public storage for organizations at least is good.
@reach-vb Hey, totally valid that the staff here DO comment and come back at us with updates. Somedays i feel like making models and throwing them at the wall like playdough isn't really "RESEARCH" but i guess the idea of creation is sort of the other half of research? It's not that i DONT do that stuff, it's that honestly - i have a github full of random python toys i'm making and none of them end up here lol
LOL, 'im not terribly worried, i'm acutally a paid creator on civitai, i just can't gen locally so i HAVE to upload my models up here.
The thing is datasets are also semi a copyright issue, they're not just looking at the $$$ but the stupidity of the lawyers who will look at a tshirt and call it porn. (I say this because amazon's rekognition tagger loves to tell me a clothed human is explicit nudity)
Really weird, because i'm sure i'm entirely OVER that with my organization lol.
I haven't uploaded to my flippin main account in ages, i always do it to my org.
I don't have much of an issue, but i'm not uploading 6tb in a day so XD