/hydrus/ - Hydrus Network

I figured out how to make Ebola-chan's symbol in unicode this week:

☜🌢☞

Please spread it as appropriate.

Also here is a leaner version of the rifle that has been floating around:

︻╤╦━╾╴

just going to make a recommendation for the simple downloader.

Personally, the vast majority of what I download will run through all linked files through images in page, and if it doesn't work though that, it's going to have to go through a gallery downloader.

Im sure in the future there will be a great use case for it but I can't see it right now.

would there be an easy way to put a checkbox on it to lock out any parser change other than what you have as default/last used?

It would largely be piece of mind more than anything else. I just have the feeling i'm going to accidently a large 20K image download at some point and not have the links to re feed the downloader.

that said, thanks for the update and getting the old functionality back, albe to go though some of a backlogged threadlist now.

>>8584

Based dev.

I never thought I'd see the day that our glorious dev would grace us with his baneposting abilities. Thanks for all you hard work my man!

>>8590

>>Devbro sticking to his weekly output well into the 2030's

i love you

appreciated as always

one thing i have noticed is that exif rotation metadata is correctly adhered when rendering thumbnails, but not in the image viewer itself. any chance of getting this little one fixed up?

>>8598

> we all still have problems with data hoarding

Also dev PLS unpin the old update threads it's making my autistic checking of /hydrus/ a lot harder.

Please add the ability to set a shortcut key to the "set all selected as alternates" option. I've been setting alternates manually as I import images to avoid unnecessary work with the dupe finder, but I just imported 2000+ images I have to go though and each image has several alts. The right-click menu is way too slow, a shortcut key would be very useful.

>>8600

>to avoid unnecessary work with the dupe finder

dude what. just use the dupe finder, its amazing since he added that shortcut menu to the side.

i ran over 2000 duplicate pairs in a couple of days

is there any way to download art from twitter?

i cannot understand why people use twitter, of all fucking social media available, to upload their fucking art, when its not designed for such a purpose, but they do. and there's a couple artists twitters that i've been eyeing

>>8602

Not built in. Hydrus_dev is working on various overhauls that may allow him [or third parties like "us"] to eventually add more downloaders in an efficient fashion.

I'm going to guess that if third parties can actually pick this up easily, we're probably going to see reddit/imgur and twitter and instagram etc. downloader script integrations for it. Most of these already have some downloaders out there on github and so on anyhow.

For now, you simply want to use an external application and just import the resulting folder.

If you also need twitter's tags, I guess you're going to have to find out how to regex them from the filename or wrangle them into a companion .txt file. Hydrus can be made to use either of these to record tags on import.

>>8603

eeeh i can do tags on my own, at least some general tags like artist name, character name, and whats going on in the picture. but i have no idea how to bulk-download from twatter because of their shitty weird interface.

>>8604

> eeeh i can do tags on my own

Ok.

> i have no idea how to bulk-download from twatter because of their shitty weird interface

Well, obviously twitter themselves do not provide this?

They want your attention on ads and behavior data they can sell; not host easy media downloads in bulk.

Anyhow, you can still use software to get what you want. Use one of the multiple twitter downloader tools published on aforementioned github. They're typically fairly trivially easy to use from a command line.

deleted >>8605 >>8606 because im retarded i accidentally un-ticket "auto-add tags parents" and didnt realize until now

>>8583

>imgur image album

i was working on imgur album parsing but i stopped due to new job

each album/gallery has a chunk of json in the html with all the images in it

every script tag
regex
(?s)widgetFactory\.mergeConfig\('gallery', {(.+?)}\);

string manip regex
(?s).+image\s+:\s(.+?),\n.+

should output json that you can further parse to get all imgur pics from album

i ran out of time to actually going through and parsing the json though

note: is if the image is a gif then you can check if prefer_video is true and return .mp4 instead.

Sorry if this is already reported, but after updating (from like 29x) new parent tags do no longer work until i reboot hydrus - they then work just like old ones.

>>8583

This is a pretty massive feature request but I think it would fit naturally in with hydrus. How about adding some tensorflow script that scans all your tags and learns to identify the patterns that correlate with each tag in each image? Considering we all have huge libraries of manually curated, unique images, it seems like a machine leaning auto tagger that we could opt to train on our personal libraries is a natural fit. Like it could do all its training then the user could approve each image's added tags individually or all at once.

Hydrus is phenomenal btw, thanks for making it.

>>8612

>The new 'simple downloader' has a simple 'twitter image' parser that'll do single posts for now. The new gallery overhaul will allow you to bolt a 'put in username, get image tweets back' search component to the same basic system and add tags and all that.

>The 'drag-and-drop URLs on the client' stuff should be a half-step towards this, so you'll be able to drop twitter URLs on the client soon enough.

>Twitter gif/video seemed a little trickier from my recent looks at it. It gets embedded dynamically by javascript somehow. I'll give it another look when it comes time to write a more complicated parser, but we might have to look for some aggregator feed or something that does the twitter-end API legwork for us.

amazing. hydev you are a true hero.

>>8612

I wish I could answer your questions about ml. I only do programming recreationally and have hardly scratched the surface of tensorflow, but from what I've seen image recognition isn't hard to implement at all. The question of scaling across a variety of commercial hardware is an interesting one. Does hydrus have an api?

In regards to the new sankaku limit, would it be possible for the downloader to grab the list of files, but instead of grabbing the files from sankaku directly, instead try to download them using ipfs?

>>8612

with the persistent lock, instead of deleting them would it be possible to add a disable? there are ones like twitter or imgur that I will likely end up using in time and dont really want to delete because what if.

though I will go through with deleting them if I do end up fucking an import up.

>>8618

I'm trying to get a hold on those downloader formulae things, can you explain a bit more about how that website works and what pages you want scraped? I'll see if I can take a look at getting a downloader formulae working for it.

>>8620

i have no idea how it works, yiff.party is not specifically aimed at furry, disregard the name, but at any and all patreon content.

users use the sites functions to "rip" posts from a patreon page and upload them to the site. The developer of it refuses to add a "download all" button to the site because of bandwidth. you also need a cookie to be allowed access to the site. You'd have better luck going to the yiff.party thread and asking there directly, the dev should be browsing casually. I'm just a user reaping the benefits tbh.

>>8612

For what it's worth, once I get the cash to fix my good desktop I'm planning to see if it can sustain a bit of HyperGAN so I can try to auto-generate more niche porn using part of my Hydrus collection.

I'll get back to you if it yields any good results.

>>8616

How the hell would that even work? Why would you assume that someone has those files lying around and has them uploaded on the network?

im having a problem with pixiv, or maybe its a well known current-issue?

when i try to download pictures that are in multiple pages the download fails. i've tried through Subscriptions and through the raw URL downloader, no luck. any help??

>>8612

>Our meme corpus is far in advance of theirs, as is our electronic sentience, and their… ethical inflexibility has allowed us to make progress in areas they refuse to consider.

Is that The Algorithm from Alpha Centauri?

>>8636

Why contain it?