/hydrus/ - Hydrus Network

>>6790

Thanks for your work!

>colour update

Hydrus Dev is European?

>>6792

He always seems to post in the dead of the night. So its possible.

>>6793

maybe for your time zone. He seems to post afternoon central us time

Ok, just got done with migrating to the nvme, shit took entirely to long to move less than 20gb worth of files.

So the images are now s:/hydrus and the db + thumbs are in c:/hydrus/thumbnails and c:/hydrus/database respectively.

as for booting the database of cluster fuck I have, was the fastest it ever opened

scrolling down the images are there so fast its not even annoying

Now, before I updated, I moved 2 directories into the database for testing, one sorted through images at around 100 images a minute, and the other, the g art folder, went at around 21 images a minute, putting it just about at what it was doing before, once the h folder imports ill get times on that, and then i'll import a smaller folder to see if that goes faster.

Other then that, it would be time to get off my ass and close out some of the bigger tabs I have been meaning to get to.

The file import options dialog starts off fully maximized for some reason. Just lettin ya know.

>>6790

Hey, I've been out of the loop for a few months so I missed quite a few versions.

Just tried the duplicate checker, cool stuff.

I wanted to know if there's yet a way to scrape Sankaku again – I think you removed it temporarily because it didn't work anymore because of the Cloudflare protection they introduced, right?

Looking over the release logs it seems like you've completely changed the way downloading works.

So I've tried re-adding Sankaku manually using these settings: https://github.com/hydrusnetwork/hydrus/blob/1cf48e5a2585b14bf6b702af55d097e55aff12fc/include/ClientDefaults.py#L458

However, I'm still getting 503's. So I guess this still isn't fixed – or is there any workaround we can use?

Funny thing, I can use curl to fetch stuff no problem at all. E.g., curl -I https://chan.sankakucomplex.com/?tags=rating%3Asafe

I have no idea how this is possible when Cloudflare uses JS challenges that need to be solved to get a 200 response.

Another question: is there a way to mass-tag a file from the clipboard? Let's say I have 10 tags, one tag per line, in my clipboard. Now when I paste it, the line feeds are removed and I end up with a single tag that's composed of all of the 10 tags.

Thanks for your work!

Is there a way to update from the program? If not, why isnt there and can we expect that to be added?

>>6805

Im just going to assume complexity involved, and at best the program would download the installer for you to run, the most the program itself could probably do at this point would tell you there is an update, but I would much rather see dev time spent in other areas than things of convenience that are near painless to do manually.

>>6799

Ok, just restarted the program a bit ago with a blank page, imported the j folder, and it went at around 238-251 images a minute.

A definite improvement over when I have all the images loaded, but its a bit weird how I am given wiggle room from 100 images a minute down to 21 700 images later,

either way i think im going to get the importing of the massive fuck off folders done this way so I can evaluate what came in and close them out

Going to keep a saved session of what is there now for future testing if this gets worked on more.

>>6806

>>Im just going to assume complexity involved, and at best the program would download the installer for you to run, the most the program itself could probably do at this point would tell you there is an update, but I would much rather see dev time spent in other areas than things of convenience that are near painless to do manually.

Seconded. This is one of the last things I would like to see the dev working on.

So, in part of importing my archives, I found that one of the archive folders was fucked completely to torrent program never finishing.

Not willing to just let it sit at this, I decided to hit the boorus and download everything with the same name

YOZORA

for those that don't know, this is pretty much the giant cock futa possibly loli vampire artist, i'm sure everyone knows an image by this person.

So I hit all the boorus but then it came to me… why am I hitting them all manually?

Now I can't imagine using this for something like "anus" but i'm sure we have all liked an artist online, and then found out they found jesus and scrubbed fucking everything and the only way to build an archive is either find a torrent of said artist, or hit the image boards hoping someone is posting or posted their archive.

But with this and going to every working one, as sankakuchannel doesn't work, has likely rebuilt the entire archive that would have been lost if I hadden had the artist name.

It would be nice to have an 'all the usual suspects' option to just check one site, when it scrapes everything, go to the next, so on so forth till its done. possibly checking it all and adding them at the same time, because if the downloader is set to be kind to sites, then going to a different ot check and download would allow the first to rest a bit more.

>>6807

It really shouldn't take more than an afternoon to include an auto-updater.

It can have options that prompts the user that an update is available, download the update only or download+install. The update pop up can include the change log.(or even just link to it)

Installation would be as simple as downloading the zip from github and extracting it in the main hydrus folder.

I personally don't mind manually updating since I always check the threads for any bugs that may arise before updating,but I figure it's a basic function any program has.

>>6809

literally every wednesday the program updates, I picked this version up I think 2 minutes after it was uploaded, not linked to here.

Best to work on this last.

but if you really wanted, something like a popup telling you 'you may want to check for update now' may be easily added, possibly a subscription to watch a url and it checks the url every 4 hours for updates, that way when one comes, it notifies in the corner with an image that had 'Update XXX now available'

>>6810

this isn't really about me. its more about the philosophy behind including it, the big picture. you're only thinking of yourself.

I've already got 30+ programs in which I have to visit a forum or site to update weekly, I really dont want more devs following that advice. even a simple batch/bash script would do.

if it's last then its guaranteed it won't get touched, especially when you look at the current todo list. i'm a bit surprised it hasn't been included yet considering how often the dev updates, and especially considering he pushes the news to twitter tumblr and 8ch.

but that's just my 2 cents.

>>6812

Dayum… look at that consistency.

Was he sick in February?

>>6815

Yes. Or sometime there abouts anyway

>>6812

Looking at that consistency, I believe even more in a subscription that you can opt out of called "important hydrus" It just watches a page and either only puts a message up when something major happens, as in 'a new system is fully finished, time to update' or 'massive performance boosts, update at convenience' or 'sick till further notice'

I know when I started dicking around with this program was nearly 1 year to 2 years ago and I didn't actively move my archive over to it till somewhere around 7-8 months ago.

It was somewhere around me finally giving up pretending I would ever use folders to accurately sort everything, and realised that even my best methods were clusterfucks. It also doesn't hurt that I came into a large amount of storage space for relatively cheap so it allowed me to dump 200gb into the archive, and is allowing me to unzip and import all the artist archives I have collected over the years.

during that period where I was iffy about should I or should I not, It would have been nice to be told what updates were important due to finishing implementing systems, or that updates were available at all as its only been 7 months or so I updated weekly.

Oh, and hydrus dev, as I have said, 400-600gb is how big this archive will be when alls said and done, but I don't have a hdd for backup, nor do I really want to buy one just for backup either.

Now, I had an idea.

there are effectively 3 parts to hydrus to backup.

1) all the info, the mappings, the tags, the ratings, all the user time and effort shit that if you lost all the images, and then re imported them, you would have all the info back

2)The thumbnails

3) the images themselves.

Now personally, I dont have the space to backup images easily, but I do have thumbdrives that I can back up the shit in number 1 fairly painlessly, as for number 2, is it necessary to back up at all or would if shit hits the fan a rebuild of thumbnails be viable?

but 3…

Is there any way that you could implement a system that would store the images out in sizes that are burnable to dvd or bluray? This at least for me is the cheapest and best method, as its less prone to breaking, easier to store ect ect.

Its less for a working backup, and more of a rebuild blackup.

with this, remembering what disc what images were on in case a reburn is necessary, or keeping track of how many new images are in the archive so it knows when you could either burn a dvd or a blu ray off and make it worthwhile.

Then have the important shit on a normal backup method where it writes to wherever you dump it the new info that was added or what needs to be changed.

as for how it would do this? thats hard.

You could have it make a folder where you just dump a folder that says

hydrus\disc_XXX

and you burn it off like that, as even though I have enough space to double my archives size, I cant say that is an indefinite thing. or you could have the program make an iso file for you to burn off.

This does not need to be any more complicated then a non functioning version of the archive that you just dump into the program when you rebuild.

I don't know how important you consider backup options outside of full working archive ones, but it's something I think you could consider as an option as I think there are a lot of us who have multiple smaller areas we could dump files then singular large ones.

>>6820

A compromise between the two would likely be one big file per folder, or many smaller yet bigger files, just big enough to better fit in a block

but honestly, the size difference isn't that bad, but the taking something like 6-12 hours, I forget how long I slept, was fairly bad, but it should be a one time thing.

>>6821

I would honestly just make an opt in subscription kind of like 'i don't know what i'm doing' option is that you just toss semi-fairly important notices in.

it keeps it out of peoples faces for those who don't want it

it notifies important new updates for people who don't do weekly ones.

on the backup side, all my drives together are somewhere around 20tb of storage, some of its redundant, some not.

I try to keep everything fairly important mirrored on the nvme but doubling the cost of my storage for no active benefit is kind of hard to swallow.

could probably get a used laptop drive and connected it via usb type c and do it,

>>6823

Thank you for this report. This should be fixed for v275. Please let me know if it gives you any more trouble.

>>6820

>>That's unusual! Can you tell me your OS and what settings you have under options->gui for the 'regular_dialog' window entry? I think file import options falls back to that default. Mine has False for remember size/position and maximised and fullscreen, but maybe yours is different?

Tried to reproduce it just now and the problems gone. Strange

so i'm importing things, getting the image archive up to that 1 million image mark, or at least doing my best to get there and I come across an overwatch image by ottofoxhole

but right next to it is a clothed to censor version, well… it may not be to censor but the clothing was added so shittily that it seems like it was.

Now I noticed on right click I have a see similar images, and find duplicates…

I'm not sure when they came in, I thought they were there but then went away but they are now back so I thought I would test it just to see the results.

exact match, nothing

wasn't expecting anything, exact only picks up small edits as exacts

very similar… nothing again don't have a frame of reference for what this picks up

then similar, this one picked up the clothing edit… cool

then just for fun, lets see what speculative picks up. and it came up with a really shitty futa edit.

Now curious, I want to see if I can find an image that picks up correctly across all 4 of the presets,

So I go looking for an image that I know I will have many copies of and will have edits.

now, its an exact match that is white and blue, now there are many different file sizes, however it has a futa version, and a color version that I would say aren't even fucking close in the exact match category…

looking through some more, im finding some weird things.

I have to ask, how does the duplicate finder find duplicates, because some of these differences under exact are a bit… to big to be exact anymore.

Came across some images on importing that made me wonder if this was a viable option

would there be a way to add 2 hotkeys to viewing images that changed the background color? Personally, when everything works correctly, I prefer a black background to images, however transparency is a bit of an issue, so the hotkeys would switch the background from black to grey and white either while pressed going back to black when released, or a toggle

Potentially some form of data associated with an image that will change the background or just impart a different color to the transparency in the image borders though i'm not sure how viable an option the latter is.

>>6835

>>6828

Got it, thanks.

>>6835

with the deleting the program obviously remember something of the image, so there is data it was removed, that's where a deleted file being able to be sorted into different deleted folders when it tries to re import is comes from.

I have no idea how hard this would be, and more on this a bit further down, but a flag that sticks with an images hash (im assuming this is how the image data is retained sans image) so that when it tries to re import, it sees

1) the image was delated

It then looks for the reason it was deleted, it sees duplicate filter was what took it out

2) inside the deleted folder it sorts it into the reason folder

Now, I personally use 200x200 thumbnails, so I could make a fair amount of decisions without even opening images, so lets say I select 10 or so files and I want them removed because they are crap

I hit delete and it wants to move them to the trash, instead of ok and cancel, again this is just for my case as I am thinking of user assignable reasons, it gives me 4 options

[crap] [prefer alternate] [to big] [cancel]

3 of them sends it to the trash with a tag for what folder to sort it to if it gets re imported, the other is kind of obvious. for now I have the space to eat the potential 150000 duplicates so I can wait till either something happens in program or I figure something out, but I would love to see duplicates get hit with some kind of tag just so its something I can do.

Now, on the note of an issue that may crop up with this, when I was importing some of the artist archive, I had saved quite a few of the images from other sources, so I had an already in archive folder of 15000 images, I like this mostly so I know all the images were imported, and like seeing what was already inside. However, 15000 images on a hdd, in one folder… takes a bit of time to fully load, so I wanted to see if I could just delete the folder and hydrus would just make a new one to replace it. needless to say that tossed an error at me.

I cant really say how often this would be used, but hydrus being able to make its own folders when one is missing would be nice.

and on the topic of duplicate detectors, I want to advise you to just add multiple versions of the dup detector alongside each other. I cant speak for everyone, but it the way I deal with my archive got based around the habits of the current detector, and then it changed potentially for the better but broke the function it was serving, that would be frustrating.

>>6839

If I am not mistaken you could give the file a local tag before deleting it, Then after importing a number of files you could sort by that tag and delete them again.

Ok, got a question

Something appears to be gone

files -> right click -> filter -> custom filter

I don't see a way to do this anymore. there are a few artists I imported where I want to go through every file and hit them with all the ratings and doing that was infinitely easier using my macro pad over the mouse to hit ratings.

Ao am I doing something stupid or were the filter shortcuts removed?

>>6844

I have replaced this with 'custom' shortcut settings. Please go file->shortcuts and then create a new shortcut set. I believe your old custom filters will have been transferred over, although I do not remember well.

Custom shortcuts are activated at any time in a media viewer through the small keyboard icon on the top hover window.

>>6846

Thank you for this report. I have tried your situation, and I agree that it is annoying. Lining things up both ways so specifically isn't trivial, but I will see about not scrolling if say at least 75% of the thumb is in view. We'll see if that makes the workflow cleaner or whether we need to do more.

You can hit middle-click to launch a file in the media viewer with one click, btw.