[ b / kemono / coomer / memoryhole / requests ]

/kemono/ - kemono.party

Kemono Development and Discussion
Name
Email
Subject
Comment
Verification
File
Password (For file deletion.)

File: 1628187059027.png (962.99 KB, 699x700, bcheatsenabled true.png)

 No.78[View All]

Post information about websites or software that people deserve to know about.

gallery-dl (https://github.com/iori-hub/gallery-dl)
A program that, once installed, lets you run a command in command prompt to download all images from a given link. It's compatible with pretty much every site I've tried it on, too. (Twitter, FurAffinity, e621, Inkbunny, Newgrounds, e-hentai, Pixiv, and surprisingly even Kemono.)
The main downside is that it's a python script run from command prompt, so to install it you need to install python (and in my case that didn't work so I needed to install it through Chocolatey, which is another python script that does… something?). Once it's installed, though, all you need to do is feed it a link and it just works.

Twitter Media Downloader (https://addons.mozilla.org/en-US/firefox/addon/tw-media-downloader)
Firefox extension. Essentially an in-browser version of gallery-dl for just twitter specifically. Easier to install, and it has an actual HUD instead of using command-line. It also lets you download a user's LIKES too, not just their media or retweets.

r34dl (https://github.com/r34dlnew/Rule34.xxx-Downloader)
rule34.xxx is one of the few sites gallery-dl doesn't work on, so I use this instead. Lets you download the entirety of the results of a specified search.

Awesome Duplicate Photo Finder (http://www.duplicate-finder.com/photo.html)
This program scans a given folder (including subfolders) for duplicates, then lists them and gives you the option to move or delete them. This is really useful for doing a first-pass over a mass-downloaded gallery before sorting it "properly" afterwards.
90 posts and 3 image replies omitted. Click reply to view.

 No.13190

Given an artist's ID number and the service they're using, is there a way within the API to see when they were most recently updated or imported? I know that if I use kemono.party/api/$service/user/$ID I can get the date of the most recent post that has been imported, but that's not the same as the import date.
Currently I periodically scrape artists with gallery-dl but I'd like to add a filter to my script to skip artists that haven't been updated since the previous round.

 No.14188

>>11981
Gallery-dl probably needs updating…though I can't say I know how to do that.

Don't see any way to do that when using it, so do I just download it again from github?

 No.14189

Actually, it DEFINITELY needs to be updated, but again, I've no real experience or understanding of coding.

If it's done any other way than redownloading it off github and replacing the original's files, I'm at a loss.

 No.14194

File: 1640593546520.png (1.38 MB, 1155x1263, E6_kcLOXoAAdeHG.png)

Thank you so fucking much Twitter Media Downloader is exactly the tool I was looking for, I have like 6k+ likes on Twitter that I wanted to download

 No.14346

>>14194
>>78
Does Twitter Media Downloader also handle retweets (RTs) and tweets beyond the 3200-or-so timeline limit? I don't give a shit about who liked what or pictures of fucking food but I do give a shit about links even if they are said to be dead.\n\nSocialfags sometimes have decent ties though they mostly retweet themselves and inflate their tweet count which in turn destroys the api search feature. Not sure when twitter stopped the ability to check for non-native RTs.

 No.14767

>>11981
Same anon as>>14188 again

I figured out how to update gallery-dl, and have done so via 'py -m pip install –upgrade gallery-dl' in Powershell, yet viewing the version, or trying to download from KP still gives an error.
[config][warning] Could not parse 'C:\Users\-Username-\gallery-dl.conf': Expecting ':' delimiter: line 317 column 17 (char 7124)

What I have in the config file is this: "kemonoparty":
{
"cookies":{ "C:/Users/-Username-/Desktop/galtest/KP Cookies.txt"
"__ddg1": "OEChcCo9s8pMoSaqz3wc",
"__ddg2": "FTBYHNpGitMKfuIL",
"session": "eyJfcGVybWFuZW50Ijp0cnVlLCJhY2NvdW50X2lkIjo1NzcyfQ.YaRcEg.8swGkcJWSyRQxUq7riYp_2IAUJU"
}
}


Is it asking for a colon to be put before the "__ddg1": or something? That doesn't make sense.

 No.14768

Continuation of >>14767
I have updated this entry to be the exact same as the way it was set up here: https://github.com/mikf/gallery-dl/issues/1716

So what I have is this:
"kemonoparty":
{
"cookies": "C:/Users/-Username-/Desktop/galtest/KP Cookies.txt"
{
"__ddg1": "OEChcCo9s8pMoSaqz3wc",
"__ddg2": "FTBYHNpGitMKfuIL",
"session": "eyJfcGVybWFuZW50Ijp0cnVlLCJhY2NvdW50X2lkIjo1NzcyfQ.YaRcEg.8swGkcJWSyRQxUq7riYp_2IAUJU"
}
},


Now it's giving an error saying it's expecting ',' delimiter…so that the first two lines would read as
"kemonoparty":
, {

This, of course, doesn't work, and just gives an entirely new error about expecting something else. I'm using the default config file from the git page, so I'm utterly at a loss as to what's going on.

 No.14777

>>78
If no one has said (https://bionus.github.io/imgbrd-grabber/)
It's a good image scraper with presets
Great for us who don't know too much about tech shenanigans.

 No.16966

Sorry for long absence but I do notice something wrong with yiff.bat and I really wish I have updated long ago,

It failed for Kemono Patreon because it tried to grab avatars from Patreon server then proceed to Kemono Patreon

it worked because I was signed in on Patreon and provided Yiff.bat Patreon cookies, really oversight of me

Kemono.party's handling of files seems to be different today so I'll need to update yiff.bat accordingly, I haven't touched it for a while so I ran into several issues, seems there's lot to do!

 No.16967

>>16966
Indeed it is different… Very VERY different. Basically a full rewrite would probably be needed since we now have 8 cache servers, of which are selected by the load balancer. You'd have to hit the load balancer to be able to download, or risk being IP banned from Kemono.

 No.16978

>>78
GalleryDL doesn't work for Rule34.xxx because the sdite's API is complete shit. Intentionally. You have to make a request for every fucking tag individually if you want any level of metadata about them. I brought it up to the dickhead that runs the site once, and all he said was "Frontend scraping is against the TOS, deal with the API."

 No.16980

Duplicate photo finder for android?

 No.16989

>>16966
Why not use gallery-dl?

 No.16998

>>1972
You could use the –exec argument. I am unsure how this works on windows (as the commands will be different. but on GNU/linux you can do)

gallery-dl FILENAME –exec 'num=0 ; while [ -e {}."$num" ] ; do num=$(($num+1)) ; done ; mv {} {}."$num" ; echo {} saved as {}."$num"'

This will create file duplicates if you run the gallery-dl command twice though.

 No.16999

>>1972
>>16998
instead of FILENAME do LINK.

 No.17159

Hey, does anyone have a way to decrypt or get the password out of a .zip file?

 No.17413

gallery-dl downloads images in a completely scrambled order for some reason. is there any way to force it to download things in an orderly fashion?

 No.17432

>>17413
What version of Gallery-dl are you using? I use gallery-dl on my server and it downloads into recursive folders: Platform>ID>Post#
I can't remember if there is an option in the config for that or if it is set by default. if it's not, there's probably a setting somewhere. Check ther github and see if something about it is on there if you can't find anything otherwise

 No.17505

uh anyone know a site or tools for twitter image source locator?

I downloaded images from twitter long time ago and want to know the artist or the source again. I used Saucenao, google reverse, and yandex. But no one gives an accurate result, esp if the image is not popular and only with a few RT

thanks

 No.17714

Fuchsatchi.bat (formerly yiff.bat, you can find an obvious link to it) received significant changes with a bunch of new features. I haven't tested thoroughly so expect to run into issues (issues can be coming from kemono give or take).

I wanted to drop the name "yiff" for something else new because I'm getting tired of it and because it was sorta named after Yiff.party and Yiff.party is pretty much dead. While I'm still exploring for a new name for yiff.bat, I'm brainstorming with random but relevant words and then translate them to another language for German accent.

Fox -> Fuchs (there's hardly any place for "furry" in another language but German had a good one for "fox")
Eight -> Acht (yiff.bat has 8 available slots for parallel downloading)

Fuchsacht…

There's also Japanese translate for 8: Hachi

Fuchshachi but that sounds wrong.

I revised it to be Fuchsatchi, there's "Satchii" from Dennou Coil that sounds like a cool name and might help with pronouncing the last bit better. But does it sound too strange? Kinda want new cool names in German accent!

Existing Yiff folder should be named symmetrically with the batch file name (you can also rename it whatever you want it to be).

 No.17715

My another project is https://github.com/Rukario/Schande, its filename is also German. ¯\_(Schande for shame ツ)_/¯

I wanted to kill off yiff.bat for this but couldn't, there are some remaining features I can see appeal to using yiff.bat. More simplicity, different directories, per-artist filename rejection list, its checkpoint "ender" file, etc… so I gave yiff.bat a new name with a bit of German accent. It was hard to brainstorm for good script names so pardon me making weird names. ¯\_(ツ)_/¯

Schande.bat and Fuchsatchi.bat share many similar codes. Schande.bat is DIY oriented, you can write special scraping instruction in text file to add website support, I've tested with downloading video off from YouTube, creating alert system or scraping files off from some unlikely places, such as 4chan and Best Buy.

Another major feature is duplicate image finder, it will build HTML as our comparison UI with tools for naked eyes. Schande.bat has shortcut to delete files as you see on browser but it's clunky atm. There's a better way to it so I'm working on it. Many other image deduplicating features I'd love to add, but maybe that's for later.

 No.17719

File: 1645920321245.png (19.84 KB, 1644x679, ser.png)

Anyone know what's wrong with my config here?

I'm trying to have the FurAffinity directory put all "main gallery" files from an artist in one folder, with Scraps and Favorites as subfolders of that folder. but this current code just puts everything in one big folder called "subcategory == 'gallery'".

 No.17790

Anyone know how do use the –filter parameter in gallery-dl to exclude posts that contain a certain string in their title?

I want to download an artist's page but they make a lot of WIP and half-resolution posts I'd like to exclude. And unfortunately I don't know regular expressions.

 No.17798

>>17790790
Denpending on the size of the WIPs and half-res images you could filter them using the "–filesize-min" parameter

 No.17804

>>17798
That would technically work, but I feel like it would be both easier and result in less false-positives to just filter out posts with "WIP" or "low-res" in their title.

 No.17878

Hey

 No.17900

>>17714
>>17715

Thanks for updating this! I like the new name; way better than yiff. And yeah, the simplicity of Fuchsatchi.bat is really nice.

 No.18128

What's best way to handle artists posting links in kemono?

gallery-dl doesn't support downloading mega/dropbox/drive/other random archive sites, so is there any way to configure gallery-dl to write the kemono post url or the link itself in a textfile if it finds an unsupported link?

I thought about manually going through all kemono profiles and searching manually, but I couldn't figure out a single search term to find all links in an kemono profile, searching https returns all posts with attachments, and neither the 'OR' nor the '-' operator work when searching in a kemono profile, so you can't do a search like "https://drive OR https://mega OR https://www.dropbox" or "https:// -https://kemono" in profiles, those searches only works in 'Recent Posts'

 No.18176

Crimson here.
Anyone knows how to configure gallery-dl or Hydrus to only download from specific channels in a Kemono-imported Discord server? Or is there's any other ways?
Some artists of my faves. got too much irrelevant channels and I only want stuffs from ones with his arts.

 No.18214

>>18128
I use this to download all of the letters in each post as a txt file.

>"postprocessors":

>[
>{
>"name": "metadata",
>"event": "post",
>"filename": "{id}.txt",
>"mode": "custom",
>"format": "{content}\n{embed[url]:?/\n/}"
>}
>]

 No.18252

Gallery-dl is no longer working on KP for me. I get a 403 Forbidden error, even though I have my login credentials in the .conf file.

 No.18254

>>18252
Investigating..

 No.18304

File: 1647737367469.png (14.16 KB, 256x256, Me Fuchsatchi.png)

>>17900
I like the name too, I think it's going to be named as Fuchastchi for foreseeable future.

Updated Schande.bat for abilities to move files to trash-esque folder from browser after hitting "Schande!" button beneath each picture. Nothing will happen until you press D twice on CLI. Use native file explorer to move the trash folder to recycle bin or perma delete it. Fuchsatchi.bat also gained this new feature as well as abilities to update blacklist so they don't get to download again.

I've made lot of changes to Fuchsatchi.bat. More things will make use of parallel. For example, the last few remaining artists with long pages will start to propagate on parallel so they can finish up faster.

It appears Fanbox changed their API some time ago, updated Fuchsatchi.bat to reflect this. More data fetching but luckily the new parallel algo already got this covered.

Another change is the HTML will be created before downloading files so you don't need to wait for Fuchsatchi.bat to finish downloading files for it.

Reverted incremental file naming system for Fantia and Fanbox, I'm not proud of it. I want to accommodate new files of edited posts and the incremental name will overlap with wrong files. From now on, these files with incremental file names will become stray files.

New key listener: Press G to rebuild HTML from partition.json of each scannable artist. Especially useful for adding new stray aka "files not on server" files and/or updated blacklist.

 No.18305

>>18128
>>18214
Gallery-dl and Hydrus are why Schande.bat and Fuchsatchi.bat are way underappreciated, they build HTMLs with "Links in this HTML" in the top for these. If you're good at viewing page source, maybe you can train Schande.bat to go a little further and download files from those unsupported websites.

 No.18710

Does anyone know of a way of making new Patreon accounts without them getting instantly removed for 'suspicious activity'. Anything I use gets deleted instantly, and I just need a way of getting in and out so I can add to this site.

 No.18777

Last minute fixes to Fuchsatchi.bat for scraping Patreon with its new antibot detection

Press F on error page and it will load up Firefox over Selenium to defuse any possible antibot detection.

 No.18859

Some artists only release overly hi-res, overly bloated filesize versions of their pics and I can't commit the HDD for 7mb per pic when I have 10,000+ pics.

What I normally do is just batch convert all the "absurdres" PNG files into JPG's of around 50% or lower res. BUT, when you're working with a folder containing pics of all different res, this isn't enough.

EG 50% of 8000px is still way too big, but 50% of 3000px is usually too small. Because of this I usually end up batching loads of small batches (which takes AGES) instead of "everything at once" (which would take 9 secs).

I just wonder if (in Irfanviews batch dialogue) theres a better way to do it anyone knows of?

 No.18863

>>18859
Maybe consider switching over to Python with module Pillow, by writing good code I'm sure it will do just as exactly what you want it to do.

 No.18874


 No.18881

Hello. I have a question about scrapper Schande.bat. After the entering the Patreon creator's URL as an input I get this error: "Thread 0 was killed because there's no suitable name asset for folder creation. Check folder pickers and try again". How do I solve it? Thank you.

 No.18901

>>18881
Sorry it's not being clear. Your url was supposed to end with "/posts" but it can also mean you're logged out so "/posts" will be necessary for free posts. Schande is looking for a pattern in page for folder name. When the match is not found you will get this error.

 No.19152

Guys, can you recommend software to reduce the size of videos (either by lowering bitrate or resolution)? Maybe something simple like ffmpeg should suffice?

 No.19212

>>19152
Try Video Editor on 123apps.com.

 No.20405

>>18214
Thank you!!
It's been a week since I start using gallery-dl and was trying to download the text of each post.
Gallery-DL's Doc is incomprehensible but your code works perfectly.

 No.20906

Are there any good translation add-ons for Firefox or Chrome to translate Japanese manga?

 No.20919

>>20906
call me a dinosaur but that sounds pretty advanced for a plugin/extension. Does something like that even exist (reading & specially processing text from image files)?

Hopefully I'm about to find out just how behind the times my technology estimates are, because that would be a cool extension.

 No.20923

Tell him(https://www.patreon.com)(/user/creators?u=377212) to import creators session ids that he bought to Kemono Party.

 No.20948

Does anyone know any ways to unpixeal any art cus I'm want to buy some pack but it's from Japan and all pixealated.

 No.20951

>>20919
Strange times we live in. I looked into this when jewgle wouldn't translate the Bible for me. There isn't a plugin persay but you can use https://github.com/tesseract-ocr/tesseract in the form of a greasemonkey script https://github.com/naptha/tesseract.js . After it turns your image into text you can use https://github.com/huzheng001/stardict-3 on your computer locally to translate kanji or the likes, or just put it through google's translate plugin via right click https://chrome.google.com/webstore/detail/right-click-translate/piojkjkndneggfaibepkmabjnkjlhlmk .

Its a complicated and slow process depending on how browser dependent you want it. But just using tesseractjs and right click translate would be "simple" while online.

 No.20973

>>20951
thanks for the heads up. That does sound quite interesting and handy. It sounds like it'd do the job even if its a bit fiddly, so I'll be sure to give this a try when I next save some Doujin pages or something. See if I can get any speech bubble text copied into usable text then translate it externally using Stardict-3 or G-Translate. Afterwards I can just "white-out" the original Kanji and paste the English ontop in PS.



[Return][Go to top] [Catalog] [Post a Reply]
Delete Post [ ]
[ b / kemono / coomer / memoryhole / requests ]