[ b / kemono / coomer ]

/kemono/ - kemono.party

Kemono Development and Discussion
Name
Email
Subject
Comment
Verification
File
Password (For file deletion.)

File: 1732504211680.jpg (275.5 KB, 1080x1080, 467153641_1076029120976481….jpg)

 No.49039

hello, I'm using gallery-dl to archive creators from kemono and coomer, I seem to have accidentally over run bandwidth and been restricted, getting a 443 error. It seems to have gone away but I don't want this to happen again and want to take steps to make sure I avoid it in the future.

I can also answer questions about my gallery-dl process if you want to know.

 No.49045

this fork (abandoned unfortunately) downloads 1 file at a time i never got banned
https://github.com/elvis972602/Kemono-scraper
it's kind of shitty but works
>I can also answer questions about my gallery-dl process if you want to know.
what's your command line of choice to download?

 No.49060

the ratelimiting is automatic, it will never be a permanent ban.

 No.49067

>>49039
Just change your gallerydl config file like this to add a timeout between each download
"extractor":
{
"kemonoparty":
{
"sleep": 10
}
}

See https://gdl-org.github.io/docs/configuration.html

 No.49070

>>49045
I use git bash usually,but lately I've been splitting off the command
I SSH into my file server and "split off" the command and send its output to a log

./master_bash.sh >> output.log 2>&1 &

I store all the creators I follow in a database and build a bash script with all of the downloads.

I may experiment with the kemono scraper, the gallery-dl one doesn't re-download everything but causes a ton of network requests, which can (apparently) get you limited

 No.49075

>>49067
I've started adding a sleep request,

"sleep-request": [2.0, 4.8],

I may change it to sleep, but both seem to work fine, do you use 10 seconds? it would take forever with all I'm trying to download.



[Return][Go to top] [Catalog] [Post a Reply]
Delete Post [ ]
[ b / kemono / coomer ]