No.78[Last 50 Posts]
Post information about websites or software that people deserve to know about.
gallery-dl (
https://github.com/iori-hub/gallery-dl)
A program that, once installed, lets you run a command in command prompt to download all images from a given link. It's compatible with pretty much every site I've tried it on, too. (Twitter, FurAffinity, e621, Inkbunny, Newgrounds, e-hentai, Pixiv, and surprisingly even Kemono.)
The main downside is that it's a python script run from command prompt, so to install it you need to install python (and in my case that didn't work so I needed to install it through Chocolatey, which is another python script that does… something?). Once it's installed, though, all you need to do is feed it a link and it just works.
Twitter Media Downloader (
https://addons.mozilla.org/en-US/firefox/addon/tw-media-downloader)
Firefox extension. Essentially an in-browser version of gallery-dl for just twitter specifically. Easier to install, and it has an actual HUD instead of using command-line. It also lets you download a user's LIKES too, not just their media or retweets.
r34dl (
https://github.com/r34dlnew/Rule34.xxx-Downloader)
rule34.xxx is one of the few sites gallery-dl doesn't work on, so I use this instead. Lets you download the entirety of the results of a specified search.
Awesome Duplicate Photo Finder (
http://www.duplicate-finder.com/photo.html)
This program scans a given folder (including subfolders) for duplicates, then lists them and gives you the option to move or delete them. This is really useful for doing a first-pass over a mass-downloaded gallery before sorting it "properly" afterwards.
No.79
Photopea (www.photopea.com)
Essentially a knockoff browser version of Photoshop. Extremely useful if you've downloaded some .psd files and want to look at them or export them as something else.
whotwi (en.whotwi.com)
Gives a list of statistics for a given Twitter user. More importantly, you can search that user's tweets more thoroughly than with Twitter's default search function. Useful if, say, an artist has an OC you like and you want to search for them by name.
Tumbex (www.tumbex.com)
An archive of old Tumblr posts. If you want to find an artist's content from before the NSFW purge, this might have it. Emphasis on MIGHT, though; it doesn't have EVERY blog, and if the user deleted all the posts on the blog before it was archived then you're screwed.
No.132
Is Awesome Duplicate Photo Finder better than Visipics, which I have been using for probably a decade?
No.143
>>132Absolutely no idea, this is the first i've heard of VisiPics. ADPF works fine, although it's not very robust in terms of settings.
No.144
>>132>>143Update, I've just installed VisiPics. Based on about five minutes of preliminary usage, here's a basic comparison:
VISIPICS
* Groups together groups of 3 or more similar pictures
* Allows for a sliding scale of whether to search for exact duplicates or any similar images (in ADPF this is just an on/off toggle and not a scale)
* Folder-browsing interface is faster and more intuitive
* Able to scan for images rotated 90 degrees
* Able to browse duplicates during the scan, instead of only when it's been completed
ADPF
* Lists exact percentage of similarity (this is very loose, though, given that two copies of the same image with different resolutions can have very low similarity)
* Probably "lighter" in terms of disk/RAM usage
So VisiPics does seem like the better option. Like I said, i've only been using this for five minutes, but so far it seems to just flat-out have more features.
No.217
>>79>not pirating PhotoshopCome on, we're already pirates anyway
No.229
Hydrus Network (
https://hydrusnetwork.github.io/hydrus/) is a great image organizer/viewer/downloader program.
No.243
>>144>>78I've been using Czkawka (
https://github.com/qarmin/czkawka) for image deduplication.
No.257
>>217Yeah, but I'm not going to go to the effort to find and install a cracked version of Photoshop just to view .psd files when Photopea is right there. I'm unsure if it's better or worse as an actual art program (it can definitely be really slow sometimes though) but it's a lot more convenient for the purpose of just viewing .psds quickly.
No.295
can someone post an faq for dumbasses who can't code on how to configure gallery-dl to work with kemono? I've tried pointing it to a cookies file exported with the program they say on github and I tried putting the cookies directly in gallery-dl.conf in the exact way they demonstrate (shown below) but everything still 403s
"kemono":
{
"cookies":
"cfduid": "(VALUE)"
"ddg2": "(VALUE)"
"ddg1": "(VALUE)"
"ddgid": "(VALUE)"
"session": "(VALUE)"
"ddgmark": "(VALUE)"
},
No.309
>>295You have to label the site as "kemonoparty", not "kemono". That's probably the problem.
No.360
thanks for all who helped the github help thread linked solved it the –cookies command is a lifesaver!
No.449
Some tips for gallery-dl I've found in my experience:
* Cookies in the .cfg file can be specified either by writing them down directly or by pointing to a Netscape HTTP Cookie File. A Firefox extension that can export such files can be found at
https://github.com/rotemdan/ExportCookies* The commands "gallery-dl -h" or "gallery-dl –help" can be used to give a list of different comandline options to use. Useful commands include:
-d/–dest (specify where to download)
–cookies (load a Netscape HTTP Cookie File manually instead of through the .cfg file)
-r/–limit-rate (specify a maximum download speed)
–filesize-min/–filesize-max (specify minimum/maximum filesizes for downloads)
–no-skip (instead of skipping files that have already been downloaded through gallery-dl, overwrite the existing copies)
-u/–username and -p/–password (input username/password manually instead of through .cfg file)
–range (specify which specific files to download)
–zip (store downloaded files in a .zip archive).
* When downloading from e-hentai, you need to log into an account or downloaded images will be capped at 1280px.
No.450
>>449(cont.)
* Downloads from Kemono specifically tend to have a lot of duplicate images. I think this is because imported posts on Kemono have both the original post file and all attachments, and the post file is often used by the creator to show a preview of the attachments. More concerningly, if the post file has the exact same filename as one of the attachments, it will actually overwrite the attachment; in a couple of cases I've noticed this leading to having two copies of an image in the post and no copies of a different image in the same post. Fortunately, there's a way to check to see if this has happened; you just need to look for two images imported from the same post (which means their filenames start with the same ID) that are completely identical but have different filenames (e.g, one is called "cool image of lady.png" and the other is called "boring image of house.png" but they're both identical pictures of the same house). Duplicate-image-finding software can help with this.
* Sometimes Kemono posts themselves will also just actually have multiple copies of the same attachment, with the filenames being like "[filename].jpg", "[filename]_1.jpg", "[filename]_2.jpg", etc. Kemono will download all of these duplicates. Again, good to have duplicate-finders here. Don't just delete any files with a filename that's "[filename]_[number]", because sometimes there isn't an "original" version of the image without the underscore and number at the end.
No.466
gallery-dl config bits for tor access via a local tor daemon proxy (optional). Note the double underscore in front of the DDOS-Guard cookie names, in case those break
{
"extractor":
{
"proxy": "socks5h://localhost:9050",
"postprocessors": [
{
"name": "metadata",
"mode": "json",
"directory": ".meta"
}
],
"kemonoparty":
{
"cookies":
{
"__ddg1": "make these the same as your regular browser",
"__ddg2": "in tor browser, hit f12, and they're on the Storage tab"
},
"directory": [
"{category}",
"{service}",
"{user}",
"{date!s:.10} {title} (#{id})"
],
"filename": "{date!s:.10} {filename} (#{num:>02}).{extension}",
"wait-min": 5.0,
"wait-max": 15.0,
"skip": "abort:30"
}
}
No.677
Anyone have any luck with the new yiff.bat update? Can't seem to get it to work.
yiff.bat (
https://www.mediafire.com/file/d4x9l1fdxa8pzug/yiff.bat/file)
No.705
>>677First time I'm hearing about this program. What does it do?
No.745
>>705is like gallery-dl but only supports kemono and is more user friendly, but also lacks some features compared to gallery-dl, also seems like the latest update is broken
No.747
>>705You put artists of your choosing into the separate yiff.txt (######.artistname) and it'll download the images/zips/files from Kemono. It requires python to run and it was a lot easier to use than gallery-dl for me. Can create HTMLs and doesn't have issues with duplicates. The dev was pretty active on telegram but was working on something new.
No.748
For PC users, there are actually a standalone executable files for Windows and Linux, and it doesn't require you to install all mandatory things (such as Python scripts) and ran through CMD. I'm actually using it too.
BTW, I want a help from you guys
I've been searching the .conf file all around the directories stated in the github site, but I can't find it. What should I do? As I stated above, I'm using the standalone .exe file.
No.759
Going to be honest, I'd love to use gallery-dl, but I have no idea how to run it, let alone put in code to do so. I've installed Python, and Chocolatey, and…
nothing. Nada. I don't even know how to get whatever thing I need to put the code into up. How the heck do I do any of that?
No.760
>>759Same writer, I just figured I'd specify for better assistance.
I did get gallery-dl running via the Windows command prompt, but trying to download from KP just gives me 403s.
I have no idea how to download/find cookies for it to go off of, or to add multiple lines of code to Windows' Command Prompt without it just running it, or what code I should be copying to bypass it without a downloaded cookie.
I would really appreciate a bit of help on this, because I am, well, completely code illiterate, and this is really confusing to me.
No.787
to get the cookies you need an extension for your web browser
on the github page of gallery-dl theres a default config file, download it and put it on your user folder/home folder
(windows: *drive letter*:\Users\*username*, most linux distros: /home/*username*/)
then open it with a text editor (in windows i recomend using notepad++)bellow this part:
"booru":
{
"tags": false,
"notes": false
}
add a "," to the last } and add this text:
"kemonoparty":
{
"cookies":{
"__ddg1": "",
"__ddg2": ""
}
}
between those "" you need to put your extracted ddg1 and ddg2 cookies, then save
No.814
>>787To expand on this, it's worth noting that you also need to put the relevant cookies in the .cfg file for some other sites, like FurAffinity and Pixiv.
Aside from that, the .cfg file can also be used for several other changes, such as changing the default download directory or changing the maximum number of retries. There's also site-specific settings too, like whether or not to download an artist's Scraps folder from FurAffinity or the order in which to download images from an Inkbunny gallery. All the settings should be listed somewhere on the GitHub page.
No.853
Anyone knows of a program where I can just dump all psd files I have and this automatically export all possible combinations of layers?
Sometimes artists just upload 2-3 variants of the picture and provide the psd file which has lot of variants.
No.880
>>787Somehow I managed to make it work with the help of this thread and github, but I am getting lots of '502 Bad Gateway' and 'HTTPSConnectionPool(host='data2.kemono.party', port=443): Read timed out.'
No.916
>>880Those are issues on Kemono's end. It might help to increase the maximum number of retries from 5 to something higher in the .cfg file.
No.919
>>916Thanks. Will give it a try.
No.1026
For those using gallery-dl on this site I would recommend including –filter "extension != 'psd'" or –filter "extension != 'psd' and extension != 'clip'" to your download query.
.Psd's and .clip's are project files that tend to be fairly heavy, and can't be viewed easily, unless you are interested in learning how to draw, you should skip downloading those files types
No.1027
>>1026*
–filter "extension != 'psd' and extension != 'clip'" No.1039
>>1026>>1027This is why I reccomended Photopea earlier in the thread. You can drop .psd files in there to view and edit them, which is useful if you want to mess with layers to make alt versions or if you want to create a version with a transparent background.
Unfortunately, Photopea doesn't work with .csp files (to my knowledge).
No.1130
I wish there was a tool that could DL or scrape from Tw@ter without needing an account. The site is so painful to browse & navigate and I feel like I'm so "anti Twitter" that I can't even bring myself to make an account there, but I realise I'm just being eccentric lol. I don't think I've experienced a site that makes "viewing all an artists pics" so difficult & longwinded. Its so frustrating it sometimes makes you laugh in that unhinged "about to have a psychotic episode" way :P
No.1212
Isn't there's any programs that allows bulk downloads fo U18Chan?
No.1280
>>880Any chance you could write a quick guide for it? As I said, I just get 403 errors when trying to download stuff.
I don't know how to put in multiple lines of code into Windows Command Prompt, as copying and pasting them just…does it like each line's its' own individual instruction.
No.1282
>>1280https://github.com/iori-hub/gallery-dl#configurationThis links to the section of the github readme with all the information you need. A quick overview:
* Create a file called "config.json" in one of the filepaths listed in the readme on github. You can use Notepad or similar apps to edit this config file.
* The github readme also links to two other github docs: "gallery-dl.conf" and "gallery-dl-example.conf". "gallery-dl.conf" is a config file with the the default settings, while "gallery-dl-example.conf" is just a list of unfilled variables you can check for reference on how the config file should generally look. Paste the contents of "gallery-dl.conf" into "config.json".
* You can now change information in "config.json" to change gallery-dl's settings. 403 errors from kemono are caused by gallery-dl not being able to bypass DDOS-Guard, so to avoid that you'll need to specify some cookies in the config file: namely, "__ddg1" and "__ddg2". You can find these cookies via your browser's debug tools or the Firefox extension linked in
>>449. Once you have the cookies, type them into the .cfg file like in
>>787.
* You can also use the .cfg file to specify cookies for other sites that require it. In addition, you can save usernames and passwords for sites there, so you don't need to use the –username or –password options every time you download from a site. Keep in mind that commandline options always override the settings in the .cfg file if applicable.
No.1350
>>78Gallery-dl won't let me put any cache files, neither the normal app nor the one installed in PowerShell with Chocolatey XD
No.1362
A few browser extensions that I think could be useful:
-Hydrus companion
https://gitgud.io/prkc/hydrus-companionBrowser integration extension for the
hydrus network client, allows you to "send" tabs and cookies directly to hydrus for downloading, can also be used as a standalone cookie downloader.
-Twitter view original images
Simple extension that allows you download images in their original resolution from twitter easily, instead of the compressed versions that are normally served.
-Referer control:
Allows you to control the Http referer for any individual site, this is necessary for acessing vimeo videos embbeded in patreon posts.
No.1815
Anyone know how to use the –filter option in gallery-dl? I mainly want to use it to include/exclude certain filetypes from a download.
No.1924
>>1825Ah, okay, so it's *single* quotes around the filetype. Thanks.
No.1972
Anyone know how to add the number of the download to the start/end of a filename in gallery-dl? I'm running into an issue where Kemono pages that have different files with the same filename end up only downloading one of the images. My theory is that, due to the shared filename, they're overwriting each other, so I figure if the program adds a number or something to the end of the filename it'll not overwrite like that.
No.1979
>>1972But you'll download both copies when you can –auto-skip them prior by grabbing the filesize information where available. If you don't care about extra hammering then just use an automatic rename flag and dedupe afterwards.
The command flag you are looking for is something like –batch or –auto-rename.
I don't use gallery-dl since wget already has everything you need and it's included in any *nix distro or cygwin if you're on some shitdows machine.
For help on wget, man wget.
No.2534
>>677I don't know what to tell you, but I downloaded it and python yesterday night and have gotten it working. It's been downloading dagasi for the past 17 hours and it's not even halfway I think, but that's to be expected from kemono I guess
It took me a while to figure out how it even worked, but the download you posted runs fine
No.3082
How do I use the –filter argument in galltey-dl to exclude a file from download based on part of its name? An artist on kemono uploads both 1080p and 720p versions of videos and I'd like to filter out the 720p ones, and since they all have "720" in the post titles and filenames that feels like the best place to start.
No.3210
Is there a way to make gallery-dl work on Kemono Discord imports?
As far as I can tell, imports of Discord servers don't import the images directly to Kemono's servers, but just link them from Discord's direct-image URLs. (That feels like it should be changed.) I don't know anything about making gallery-dl importers, but in theory you could have gallery-dl scan all of the image links, open them, and then download their contents, right?
No.3212
>>3210gallery-dl can work with any import, not just discord. It's much easier for discord because of how image attachments are handled on the discord cdn and there is no risk of session leakage if done right.
No.3686
Does anyone know of a program that could let me scrape the download links on a paywalled patreon? Text, even, or page metadata like inspect element. Basically the person I keep requesting never gets added so I'm willing to learn how to do it myself if it's possible.
No.3809
>>3696
I probably used it wrong, all I got were forbidden 403s. I managed to scrape everything of the user patreon page itself but none of the links were useful, just images.
No.3815
>>3686>Basically the person I keep requesting never gets added so I'm willing to learn how to do it myself if it's possible.To be clear, you aren't going to be able to get around paying for locked work. Any sort of "cracks" don't exist outside of private zero-day vulns you are never going to be able to afford. Kemono revolves around shares of content people paid for.
If you are interested in ripping FREE posts, gallery-dl should suffice.
No.3823
>>3686Pay, then use gallery-dl.
Ez.
No.3914
it's still highly useful for us data hoarders. Especially the high volume of artist that just yeet all there content from shitty personality to social anxiety
No.4237
Anyone having issues with gallery-dl and image tagging? I've set "tags": "translated" but it doesn't apply
No.4238
>>4237Forgot to add that i'm downloading from pixiv
No.4682
Not sure if this has already been posted, but Spot-dl is great for downloading music from Spotify. Can go off of your own playlist, or a premade one. Also downloads all the metadata and implements them within the files.
https://github.com/spotDL/spotify-downloader No.4842
>>4682As long as we're mentioning stealing from music streaming services: Deemix (
https://deemix.app/) is a good way to download from Deezer.
As for actually browsing and playing a local music library, my personal choice is MusicBee. Works well and has all of the trappings you might want (import/export tools, playlist creation, easy way to edit song metadata and artwork, etc)
No.4851
>>4842You need a premium account to download anything more than 128kpbs, which is bleh
No.4943
I have managed to get gallery-dl up and running despite being shockingly inept at programming and I would like to express my exhilaration in this thread.
No.4945
So, me and one of the site devs have come up with an easy to use rsync script that allows you to download any creator on the site into their own directories. This downloads the whole contents of the creator.
Initially, there were multiple scripts that I wrote out of shear boredom and to learn a bit of bash, but the dev decided to take it upon himself and overhaul the scripts, adding all functions into one easy to use script. You can go in and edit the different parameters in an editor like Vim or just a normal text editor.
https://github.com/jojo935/KRsync-A-Kemono-rsync-script No.5183
Hydrus is the ultimate coomer. Tags, downloads, subscriptiona, duplicate checking, blacklist, site login scripts, support for custom downloaders, and FOSS
https://hydrusnetwork.github.io/hydrus/ No.5261
>>5183Meus shilling again…
No.5303
szurubooru (
https://github.com/rr-/szurubooru ) if you're willing to host a webserver is probably the easiest option if you want to self-host an imagebooru. supports most image formats and can be put up w/ docker.
No.7117
I figured I'd ask here first instead of making a new thread..
Is there any software to open .cdp and .dlst files and extract the images from them? Print screen obviously doesn't work for these and snipping tool does gives you black screen.
No.7267
>>7208
Ah, right. I think I came across that name when I was looking for info.
I'd rather not give him my digiket/dlsite account info.
I don't get why he needs to be so secretive about it.
In any case, thanks for the reply.
No.7281
>>1362>>229Hydrus is the sh*t but their UI is a bit raw
No.9622
>>78gallery-dl did its job perfectly for months, but all of a sudden it stopped working and it gives me this error each time it tries to download anything:
"failed to establish a new connection errno 11001 getaddrinfo failed"
Any help?
No.9672
>>9622Did you edit the gallery-dl config ? what is your command line query ?
No.9707
If you want to download entire posts from kemono here's a downloader I made.
https://github.com/AplhaSlayer1964/kemono-dlFeel free to make feature requests or suggestions on the github.
No.9735
>>9707How do you set that up? I try to run the requirements text but get syntax errors
No.9805
>>9672Oh, deleting and redownloading it fixed the issue.
No.9809
>>9805Glad you figured out the problem is you have any issues it'd be better to make a github issue instead of posting here.
No.10416
Is there a way to use gallery-dl to comb through posts and only download from certain URLs? Like artists who only upload thumbnails on Fanbox and then link to the full thing on Catbox.
No.10421
>>10416Yeah, just use the post's URL instead of the main artist URL.
An example being:
gallery-dl
https://kemono.party/patreon/user/14529677/post/56485244 No.10500
>>10421That still just downloads the previews though, not the external links.
For example:
https://kemono.party/patreon/user/7741707/post/58079700Is there a way to get it to look through posts and download the discord links?
No.10515
>>10500Kek no. Literally impossible.
No.10526
are you able to use gallery-dl on sites like pateron?
No.10581
>>10578
Gallery-dl's manual is fucking shit tho
No.10603
>>10581I feel your pain. I just got through blindly stumbling around and getting things through that.
No.11059
>Anonymous 10/17/21 (Sun) 16:27:40 No.7208>7117 .dlst is for the DLsite exclusives protected by CypherGuard right? If so you could ask genl on sadpanda.>Make sure you keep your purchase session token because if you don't you'll need to provide your credentials to authenticate the account.why is this baleeted? trying to hide something?
>>217 (inb4 i hope your VM gets infected next) (why does no one add vmware on the /cumg/ thread on 4chan)
STOP djvu ransomware increased cases last year (and now it skyrocketed during the second wave clowndemic)
>toon boom harmony crack reddit (trying to help a friend out by using anydesk since my other VM is mostly unused)>see furry Aria Ryder model sheet on plebbit (EWWW gross but on the other hand it looks really good)>getintopc crack by countryboy (works after 30 minutes of patching and agreeing not to support unscrupulous lowlifes)good thing i used VM poor fliggas i wish i could help them decrypt their files (i saw it on facebook)
even the phillipines has a dedicated site for malware resources and information (kek they never mentioned VMs)
wtf i love some ordinary gamers even more (now turns out im not the only one with a VM fetish and he drops the F bomb veery often)
grr im so jealous my PC will never be as powerful as his (now patrecucks will never find out that i pirated their shit)
https://www.youtube.com/watch?v=L1JCCdo1bG4 pay attention fgts this is important (even more important than spaceinvaderone)
No.11060
>>9707thanks saved (gonna repurpose my ubuntu VM)
>>10581elaborate? i never used this software before but i wanna try
>>5261who is meus (i see this software being shilled on porn threads in some obscure chan) (maybe it was egoistic 8kun /fur/ board)
>>5303will no-ip work how do i hide my webserver location (i dont want to be raided by du30 for hosting furry loli shit)
holy fuck dx11 works on VMware now i can play KSP and slime rancher to my hears content without worrying about unity analytics
on the other hand use Vbox for cpu only software/tasks like .py scrappers (fear the IDM keygen no more and it syncs my network drive)
>nobody mentioned put it in sandboxie or VMware for the infected shared files games (yes furryVNE runs with minimal lag why did you ask)man i hope stupid furfags like ^(you)^ get some kind of harmless wacom pen and photoshop/SAI history keylogger recorder thingy
and your smutty paywalled drawings leaked to your friends and the public unless you shout I LOVE VIRTUAL MACHINES on twitter
TL;DR use pirated software in a secure container to avoid getting malware or worse get caught using furry apps (especially on online classes)
how do i hide/rename the SVGA 3d adapter it show up on every output_log.txt (i don't wanna get caught for games that phone home to patreon)
No.11072
I don't know many creators who do this but, in case you want to see vimeo content embedded into a post and the vimeo link doesn't work, try this:
>>3910
No.11118
Is there any kind of software that can rename multiple images numerically in an existing folder?
No.11435
>>787Could you, or someone else, by chance link to the config file? I'm not sure if you mean this one here, or another one entirely:
https://github.com/mikf/gallery-dl/blob/master/setup.cfg Is this still the way to use gallery-dl on KP, too? I never was able to get it to work on KP.
No.11438
Same anon as
>>11435, I set up the json file as explained here
>>1282, only for it to give…this:
[downloader.http][warning] HTTPSConnectionPool(host='data.kemono.party', port=443): Max retries exceeded with url: /74/b0/74b00ac20c5da385063a2f8ecce4395c39ead61b51c13a7e1fff27bd09e84f02.png (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x03AF1A50>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))
I have no idea how to fix this, or what this means.
No.11980
>>11438Similar problem as this anon, I keep getting [downloader.http][warning] HTTPSConnectionPool(host='data3.kemono.party', port=443): Read timed out and some [downloader.http][warning] '404 Not Found' on Kemono no matter how much I try, I don't know what causes it but it seems like it only happens with some users, as I downloaded an entire fantia user page without problem just some hours ago.
No.11981
>>11438Sounds like the balancer isn;t doing what it should or gallery-dl needs updating, since it should put you on one pf the data servers. data.kemono.party isn't valid last I checked. data1-5 should be what it uses. Might be a gallery-dl issue. sudo apt update
No.12124
How do I specify the following file structure for gallery-dl?
Folder with name "Username (user ID)" > File with name "Post number - Original file name"
Is there an idiot's guide to specifying a custom layout? I've tried reading gallery-dl's configuration.rst which seems to have relevant info but I can't fathom how to go about doing it.
No.12127
How do I configure gallery-dl to download private messages, and download posts from favorite users (rather than from input.txt)?
No.13190
Given an artist's ID number and the service they're using, is there a way within the API to see when they were most recently updated or imported? I know that if I use kemono.party/api/$service/user/$ID I can get the date of the most recent post that has been imported, but that's not the same as the import date.
Currently I periodically scrape artists with gallery-dl but I'd like to add a filter to my script to skip artists that haven't been updated since the previous round.
No.14188
>>11981Gallery-dl probably needs updating…though I can't say I know how to do that.
Don't see any way to do that when using it, so do I just download it again from github?
No.14189
Actually, it DEFINITELY needs to be updated, but again, I've no real experience or understanding of coding.
If it's done any other way than redownloading it off github and replacing the original's files, I'm at a loss.
No.14346
>>14194>>78Does Twitter Media Downloader also handle retweets (RTs) and tweets beyond the 3200-or-so timeline limit? I don't give a shit about who liked what or pictures of fucking food but I do give a shit about links even if they are said to be dead.\n\nSocialfags sometimes have decent ties though they mostly retweet themselves and inflate their tweet count which in turn destroys the api search feature. Not sure when twitter stopped the ability to check for non-native RTs.
No.14767
>>11981Same anon as>>14188 again
I figured out how to update gallery-dl, and have done so via 'py -m pip install –upgrade gallery-dl' in Powershell, yet viewing the version, or trying to download from KP still gives an error.
[config][warning] Could not parse 'C:\Users\-Username-\gallery-dl.conf': Expecting ':' delimiter: line 317 column 17 (char 7124)
What I have in the config file is this: "kemonoparty":
{
"cookies":{ "C:/Users/-Username-/Desktop/galtest/KP Cookies.txt"
"__ddg1": "OEChcCo9s8pMoSaqz3wc",
"__ddg2": "FTBYHNpGitMKfuIL",
"session": "eyJfcGVybWFuZW50Ijp0cnVlLCJhY2NvdW50X2lkIjo1NzcyfQ.YaRcEg.8swGkcJWSyRQxUq7riYp_2IAUJU"
}
}
Is it asking for a colon to be put before the "__ddg1": or something? That doesn't make sense.
No.14768
Continuation of
>>14767I have updated this entry to be the exact same as the way it was set up here:
https://github.com/mikf/gallery-dl/issues/1716So what I have is this:
"kemonoparty":
{
"cookies": "C:/Users/-Username-/Desktop/galtest/KP Cookies.txt"
{
"__ddg1": "OEChcCo9s8pMoSaqz3wc",
"__ddg2": "FTBYHNpGitMKfuIL",
"session": "eyJfcGVybWFuZW50Ijp0cnVlLCJhY2NvdW50X2lkIjo1NzcyfQ.YaRcEg.8swGkcJWSyRQxUq7riYp_2IAUJU"
}
},
Now it's giving an error saying it's expecting ',' delimiter…so that the first two lines would read as
"kemonoparty":
, {
This, of course, doesn't work, and just gives an entirely new error about expecting something else. I'm using the default config file from the git page, so I'm utterly at a loss as to what's going on.
No.14777
>>78If no one has said (
https://bionus.github.io/imgbrd-grabber/)
It's a good image scraper with presets
Great for us who don't know too much about tech shenanigans.
No.16966
Sorry for long absence but I do notice something wrong with yiff.bat and I really wish I have updated long ago,
It failed for Kemono Patreon because it tried to grab avatars from Patreon server then proceed to Kemono Patreon
it worked because I was signed in on Patreon and provided Yiff.bat Patreon cookies, really oversight of me
Kemono.party's handling of files seems to be different today so I'll need to update yiff.bat accordingly, I haven't touched it for a while so I ran into several issues, seems there's lot to do!
No.16967
>>16966Indeed it is different… Very VERY different. Basically a full rewrite would probably be needed since we now have 8 cache servers, of which are selected by the load balancer. You'd have to hit the load balancer to be able to download, or risk being IP banned from Kemono.
No.16978
>>78GalleryDL doesn't work for Rule34.xxx because the sdite's API is complete shit. Intentionally. You have to make a request for every fucking tag individually if you want any level of metadata about them. I brought it up to the dickhead that runs the site once, and all he said was "Frontend scraping is against the TOS, deal with the API."
No.16980
Duplicate photo finder for android?
No.16989
>>16966Why not use gallery-dl?
No.16998
>>1972You could use the –exec argument. I am unsure how this works on windows (as the commands will be different. but on GNU/linux you can do)
gallery-dl FILENAME –exec 'num=0 ; while [ -e {}."$num" ] ; do num=$(($num+1)) ; done ; mv {} {}."$num" ; echo {} saved as {}."$num"'
This will create file duplicates if you run the gallery-dl command twice though.
No.16999
>>1972>>16998instead of FILENAME do LINK.
No.17159
Hey, does anyone have a way to decrypt or get the password out of a .zip file?
No.17413
gallery-dl downloads images in a completely scrambled order for some reason. is there any way to force it to download things in an orderly fashion?
No.17432
>>17413What version of Gallery-dl are you using? I use gallery-dl on my server and it downloads into recursive folders: Platform>ID>Post#
I can't remember if there is an option in the config for that or if it is set by default. if it's not, there's probably a setting somewhere. Check ther github and see if something about it is on there if you can't find anything otherwise
No.17505
uh anyone know a site or tools for twitter image source locator?
I downloaded images from twitter long time ago and want to know the artist or the source again. I used Saucenao, google reverse, and yandex. But no one gives an accurate result, esp if the image is not popular and only with a few RT
thanks
No.17714
Fuchsatchi.bat (formerly yiff.bat, you can find an obvious link to it) received significant changes with a bunch of new features. I haven't tested thoroughly so expect to run into issues (issues can be coming from kemono give or take).
I wanted to drop the name "yiff" for something else new because I'm getting tired of it and because it was sorta named after Yiff.party and Yiff.party is pretty much dead. While I'm still exploring for a new name for yiff.bat, I'm brainstorming with random but relevant words and then translate them to another language for German accent.
Fox -> Fuchs (there's hardly any place for "furry" in another language but German had a good one for "fox")
Eight -> Acht (yiff.bat has 8 available slots for parallel downloading)
Fuchsacht…
There's also Japanese translate for 8: Hachi
Fuchshachi but that sounds wrong.
I revised it to be Fuchsatchi, there's "Satchii" from Dennou Coil that sounds like a cool name and might help with pronouncing the last bit better. But does it sound too strange? Kinda want new cool names in German accent!
Existing Yiff folder should be named symmetrically with the batch file name (you can also rename it whatever you want it to be).
No.17715
My another project is
https://github.com/Rukario/Schande, its filename is also German. ¯\_(Schande for shame ツ)_/¯
I wanted to kill off yiff.bat for this but couldn't, there are some remaining features I can see appeal to using yiff.bat. More simplicity, different directories, per-artist filename rejection list, its checkpoint "ender" file, etc… so I gave yiff.bat a new name with a bit of German accent. It was hard to brainstorm for good script names so pardon me making weird names. ¯\_(ツ)_/¯
Schande.bat and Fuchsatchi.bat share many similar codes. Schande.bat is DIY oriented, you can write special scraping instruction in text file to add website support, I've tested with downloading video off from YouTube, creating alert system or scraping files off from some unlikely places, such as 4chan and Best Buy.
Another major feature is duplicate image finder, it will build HTML as our comparison UI with tools for naked eyes. Schande.bat has shortcut to delete files as you see on browser but it's clunky atm. There's a better way to it so I'm working on it. Many other image deduplicating features I'd love to add, but maybe that's for later.
No.17790
Anyone know how do use the –filter parameter in gallery-dl to exclude posts that contain a certain string in their title?
I want to download an artist's page but they make a lot of WIP and half-resolution posts I'd like to exclude. And unfortunately I don't know regular expressions.
No.17798
>>17790790
Denpending on the size of the WIPs and half-res images you could filter them using the "–filesize-min" parameter
No.17804
>>17798That would technically work, but I feel like it would be both easier and result in less false-positives to just filter out posts with "WIP" or "low-res" in their title.
No.17878
Hey
No.17900
>>17714>>17715Thanks for updating this! I like the new name; way better than yiff. And yeah, the simplicity of Fuchsatchi.bat is really nice.
No.18128
What's best way to handle artists posting links in kemono?
gallery-dl doesn't support downloading mega/dropbox/drive/other random archive sites, so is there any way to configure gallery-dl to write the kemono post url or the link itself in a textfile if it finds an unsupported link?
I thought about manually going through all kemono profiles and searching manually, but I couldn't figure out a single search term to find all links in an kemono profile, searching https returns all posts with attachments, and neither the 'OR' nor the '-' operator work when searching in a kemono profile, so you can't do a search like "
https://drive OR
https://mega OR
https://www.dropbox" or "https:// -
https://kemono" in profiles, those searches only works in 'Recent Posts'
No.18176
Crimson here.
Anyone knows how to configure gallery-dl or Hydrus to only download from specific channels in a Kemono-imported Discord server? Or is there's any other ways?
Some artists of my faves. got too much irrelevant channels and I only want stuffs from ones with his arts.
No.18214
>>18128I use this to download all of the letters in each post as a txt file.
>"postprocessors": >[>{>"name": "metadata",>"event": "post",>"filename": "{id}.txt",>"mode": "custom",>"format": "{content}\n{embed[url]:?/\n/}">}>] No.18252
Gallery-dl is no longer working on KP for me. I get a 403 Forbidden error, even though I have my login credentials in the .conf file.
No.18304
>>17900I like the name too, I think it's going to be named as Fuchastchi for foreseeable future.
Updated Schande.bat for abilities to move files to trash-esque folder from browser after hitting "Schande!" button beneath each picture. Nothing will happen until you press D twice on CLI. Use native file explorer to move the trash folder to recycle bin or perma delete it. Fuchsatchi.bat also gained this new feature as well as abilities to update blacklist so they don't get to download again.
I've made lot of changes to Fuchsatchi.bat. More things will make use of parallel. For example, the last few remaining artists with long pages will start to propagate on parallel so they can finish up faster.
It appears Fanbox changed their API some time ago, updated Fuchsatchi.bat to reflect this. More data fetching but luckily the new parallel algo already got this covered.
Another change is the HTML will be created before downloading files so you don't need to wait for Fuchsatchi.bat to finish downloading files for it.
Reverted incremental file naming system for Fantia and Fanbox, I'm not proud of it. I want to accommodate new files of edited posts and the incremental name will overlap with wrong files. From now on, these files with incremental file names will become stray files.
New key listener: Press G to rebuild HTML from partition.json of each scannable artist. Especially useful for adding new stray aka "files not on server" files and/or updated blacklist.
No.18305
>>18128>>18214Gallery-dl and Hydrus are why Schande.bat and Fuchsatchi.bat are way underappreciated, they build HTMLs with "Links in this HTML" in the top for these. If you're good at viewing page source, maybe you can train Schande.bat to go a little further and download files from those unsupported websites.
No.18710
Does anyone know of a way of making new Patreon accounts without them getting instantly removed for 'suspicious activity'. Anything I use gets deleted instantly, and I just need a way of getting in and out so I can add to this site.
No.18777
Last minute fixes to Fuchsatchi.bat for scraping Patreon with its new antibot detection
Press F on error page and it will load up Firefox over Selenium to defuse any possible antibot detection.
No.18859
Some artists only release overly hi-res, overly bloated filesize versions of their pics and I can't commit the HDD for 7mb per pic when I have 10,000+ pics.
What I normally do is just batch convert all the "absurdres" PNG files into JPG's of around 50% or lower res. BUT, when you're working with a folder containing pics of all different res, this isn't enough.
EG 50% of 8000px is still way too big, but 50% of 3000px is usually too small. Because of this I usually end up batching loads of small batches (which takes AGES) instead of "everything at once" (which would take 9 secs).
I just wonder if (in Irfanviews batch dialogue) theres a better way to do it anyone knows of?
No.18863
>>18859Maybe consider switching over to Python with module Pillow, by writing good code I'm sure it will do just as exactly what you want it to do.
No.18881
Hello. I have a question about scrapper Schande.bat. After the entering the Patreon creator's URL as an input I get this error: "Thread 0 was killed because there's no suitable name asset for folder creation. Check folder pickers and try again". How do I solve it? Thank you.
No.18901
>>18881Sorry it's not being clear. Your url was supposed to end with "/posts" but it can also mean you're logged out so "/posts" will be necessary for free posts. Schande is looking for a pattern in page for folder name. When the match is not found you will get this error.
No.19152
Guys, can you recommend software to reduce the size of videos (either by lowering bitrate or resolution)? Maybe something simple like ffmpeg should suffice?
No.19212
>>19152Try Video Editor on 123apps.com.
No.20405
>>18214Thank you!!
It's been a week since I start using gallery-dl and was trying to download the text of each post.
Gallery-DL's Doc is incomprehensible but your code works perfectly.
No.20906
Are there any good translation add-ons for Firefox or Chrome to translate Japanese manga?
No.20919
>>20906call me a dinosaur but that sounds pretty advanced for a plugin/extension. Does something like that even exist (reading & specially processing text from image files)?
Hopefully I'm about to find out just how behind the times my technology estimates are, because that would be a cool extension.
No.20923
Tell him(
https://www.patreon.com)(/user/creators?u=377212) to import creators session ids that he bought to Kemono Party.
No.20948
Does anyone know any ways to unpixeal any art cus I'm want to buy some pack but it's from Japan and all pixealated.
No.20951
>>20919Strange times we live in. I looked into this when jewgle wouldn't translate the Bible for me. There isn't a plugin persay but you can use
https://github.com/tesseract-ocr/tesseract in the form of a greasemonkey script
https://github.com/naptha/tesseract.js . After it turns your image into text you can use
https://github.com/huzheng001/stardict-3 on your computer locally to translate kanji or the likes, or just put it through google's translate plugin via right click
https://chrome.google.com/webstore/detail/right-click-translate/piojkjkndneggfaibepkmabjnkjlhlmk .
Its a complicated and slow process depending on how browser dependent you want it. But just using tesseractjs and right click translate would be "simple" while online.
No.20973
>>20951thanks for the heads up. That does sound quite interesting and handy. It sounds like it'd do the job even if its a bit fiddly, so I'll be sure to give this a try when I next save some Doujin pages or something. See if I can get any speech bubble text copied into usable text then translate it externally using Stardict-3 or G-Translate. Afterwards I can just "white-out" the original Kanji and paste the English ontop in PS.