[ b / kemono / coomer ]

/kemono/ - kemono.party

Kemono Development and Discussion
Name
Email
Subject
Comment
Verification
File
Password (For file deletion.)

File: 1674598193224.png (1.21 MB, 1674x1080, my_kin.png)

 No.27299

Hi, i fear loss of information, i love the effort put into kemono and the kind community feeding it.

I would like to know if anyone knows a way to clone the entirety of data archived in this site at some given moment in a way that's not literally just scraping as i believe this is neither efficient nor realistic considering the volume and number of operations (so probably not nice to the network?)

Something like an API that i missed, torrents of certain previous states of archives, anything.

If content is only available through the website as is, then too bad i guess, but i really want to get a local archive if possible :).

strongly wishing for this stuff to remain available to diggers for as long as possible

 No.27301

Scraping is the only way. Grab a partial dump https://kemono.party/dumps/, grab hash+ext from files table and fetch them like https://kemono.party/data/ha/sh/hash.ext

 No.27340

^^ or try gallery-dl
Hydrus is also an option but will require an config that is found in telegram.

If you are experiencing 403 errors simply email me your ip and ill whitelist it.

 No.27358

>>27340
Wait, there's a config on Telegram for downloading the entire site?

 No.27377

>>27301
Can you give some rough numbers/order of magnitude about the size of the site?

 No.27378

>>27377
22.5M, 120TB+

 No.27418

>>27378
>120tb inside active server racks
holy shit how much power does that thing eat every night? is it self hosted? i can surely tell its one of those LTTstore sponsored NAS drive bays too keep things budget priced.
i dont even know any VPS that has THAT much storage for only one user. i bet the networking costs are in the thousands every month though its a real shame bahnhof cucked out to the feds you could use that one if youre on europe

>how much storage do i have

a hundred times smaller just about about 2TB in total excluding my old but full HDDs
also what are some good things to do with my mini pc? any practical uses for my new file server? (its too weak for hosting VMs and gaming)
it came with a slow but legit 1tb nvme and why the hell is it partitioned to 120/850 GB by default? and i need a decent light distro for the second 256 gb m2 disk

 No.27421

>>27418
Don't forget about the Petabyte backup server too



[Return][Go to top] [Catalog] [Post a Reply]
Delete Post [ ]
[ b / kemono / coomer ]