/hydrus/ - Hydrus Network

>>12863

I'm alive!

Something is wrong with the duplicate system.

I set a collection of files as alternates manually but the dupe filter still asks me about them.

When I used the "reset potential duplicates" option because I wanted to go from a higher search distance to a lower, I think it reset a bunch of decisions I already made because the potential pairs number jumped up and now it's asking about the same files again. Including some dupes I made decisions about last week.

Hey folks, I am sorry if you are getting any 'couldn't open a page for that URL, maybe too many pages open?' popup errors while copying URLs that aren't set up to be caught by the new clipboard watcher–the popup is a false positive. It is accidentally broadcasting that text when it doesn't know what to do with a URL. The unwanted URL is being correctly ignored, just the text is wrong. If it is very annoying to your workflow, please turn off the clipboard watcher for now. I'll fix it up for 356.

>>12866

Thanks, I will make sure to give that popup a kill timer.

Here's my current plan for adding videos to the dupe filter: >>12846

>>12867

Thank you for this report. This business with the 'are alternates transitive or what?' logic I've been hammering out the past couple of weeks has made the interaction of potentials and alternates more complicated. For the moment, if potentials are reset, alternate files will be requeued in the potentials system (while remaining alternates). Basically, the client isn't yet confident that an alternate association alone is sufficient to prove the files aren't dupes, so if potentials are reset, it won't discount them from being re-added.

For 356 I will move same/better/worse relations to the new structure, and after that I will move potentials. To deal with the situation where two duplicate files are alternated to the same third file and so transitively alternated with each other, I now expect to improve the potential system to better track how 'integrated' a file is into its alternate group. It'll all work the same way as the current duplicate filter, but it will remember better what it needs to. I'll also brush up the manual setting-alternates command so it skips this verification step.

I am happy with the improvements so far and to come, but the duplicates system is in logic limbo in some areas right now. Please bear with any redundancies for another month or so. If you want to skip redundant alternates in your processing queue in the meantime, please try adding 'system:num duplicate relationships = 0 alternates' to your dupe filter search.

>>12873

That is the eventual hope. Alternates have no proper workflow or media viewer display yet, but I would like to get to the point where they have some sort of labelling and indexing capability, and to cache that UI-side and display it in the media viewer as a 'perpendicular' file queue to browse through, so there will be some sort of notification in the UI that the file is 'Number 3 in a WIP series' or whatever and present you quick up/down browsing to jump between its alts.

I do not feel I can start work on a proper alternates-processing workflow until the underlying duplicates database storage is improved, which is my current job. My first stab at this system is all pair-based, which is fine for groups of two or three dupes, but for larger alternate groups (like 150 game cgs), it is a hell of rats' nests. This new system I am moving to now is group-based and requires less storage and human processing to work with. Instead of 700,000 decisions, it'll be more like 7,000-70,000. I will not have alternate processing working in this iteration, but this is a popular topic, so I expect it will be queued up in the next big job poll or two.

For now, files that are set 'alternate' are just dumped into a group together with no labelling. They are in a holding pattern, waiting for a workflow to be made to process them properly.

>>12862

You are definitely getting there, your supporters love your work

And I definitely love your work as well, but I am mere mortal

https://kk.org/thetechnium/1000-true-fans/

https://yaro.blog/8745/how-to-build-a-fan-base/

https://www.copyblogger.com/20-steps-1000-fans/

Hydrus is definitely not a business, but it has great potential for the community.

Someone may find this funny, I mistook the image drive for my steam drive, so I fucked myself, instead of 32gb and being in a good spot for lacking, I had 2gb and wouldn't make it a week. have to buckle down and get shit done, no more slacking till everything giver and video over 4mb is dealt with unless approved.

>>12861

Hey dev, thanks for the jpeg quality thing even as experimental, sometimes I have to sort high-res dupes on a low-res screen so I appreciate it, zooming in lags for me and without it it can be hard to detect artifacting.

>>12913

use a third party program for zooming in, most images, with exceptions for extreme resolutions to cover the artifacts, show artifacts at a far lower resolution, its just VERY hard to see as we are talking about small details. personally even on 4k 55 inch its hard to see with naked eye, its there, but hard to see, my second monitor has a 8x-12x zoom on it at all times and wherever my mouse is I see it enlarged, I can easily see this even on lower end monitors if the opened image was 1/2 the screen and the zoom was the other half

How did you find the E3 this year, hydrus dev? Better or worse than last year?

>>12909

Hey, I am very sorry, but I do not understand your post's question. Can you rephrase it a bit? Are some of those shortcuts not applying somewhere, or are you asking more about what is a 'good' way to set up a processing workflow?

BTW G13 is great, I have one too. I just cleaned it the other week when I found out they stopped making it.

>>12913

Yeah, thanks, I am quite pleased with it. It uses some shit code, but overall it seems basically accurate. I have some bugs to clean up with it, and then I'll show it for all users.

macOS, Version 355:

Navigation keys and UI in media viewer no longer work, causing the user to get stuck.

The media viewer UI (bar at the top of the media viewer, tags list, and URL list) only begins to work after right-clicking, but the navigation keys do not.

Even if the user escapes fullscreen from the right-click menu or top bar, the navigation keys do not work.

Sorry if this belongs in the bugs thread.

Ran into a minor problem that took me about 5 minutes to figure out.

so I just imported 10gb of images, of which only 500 were shown and only 100 were new, fantastic, I get a good chunk of hdd space back.

Now going through the images I imported to see what's worth keeping I ran into a few images that were deleted. everything is going perfectly fine till I decide to find out why one was deleted, and i click show file in new page… on a deleted file…

so hydrus has its normal error image come up, everything is working as intended, however, I didn't know it was the wrong file, I thought it was one above it and something on the db side was fucked.

So with this kind of screw up in mind, would there be a way to have hydrus return the normal hydrus logo with a 'deleted' or something image for the thumbnail?

>>12926

they stopped making it? well fuck me, hope mine doesn't break as I use it as a hardware monitor.

As for my question, I can't remember the circumstance but I have 2 sets of hotkeys, one for the mediaviewer (fullscreen) that I am going to use for sorting files, and one for thumbnail view, the issue was in fullscreen the ones in thumbnail were still applying to fullscreen mode rather then just the ones for the fullscreen.

ill try to write this better if I remember what went wrong, or if the conflict comes up again, but for the time being, I just made 2 more sets of potential hotkey buttons so it likely wont conflict for a while.

>>12936

I'm having the same problem on osx after updating to 355. The keys no longer work for navigation or shortcuts until right clicking, and clicking on an image will sometimes open it in a new window which I'll have to close manually, instead of escaping from, in order to get back to the browser. Also if I select now is a good time for maintenance upon shutdown the maintenance will get stuck and I'll have to force quit.

Ok, have one more thing I probably asked about a while ago but didn't know there was already kind of an in program way to do this. would it be possible to have a floating ratings window on the thumbnails? a good portion of sorting for me takes place on thumbnails, is it possible to get a floating ratings bar so I can see what things are rated?

Ran into a duplicate that I don't want to deal with

long story short, I want to deal with these ones in the threads I got them from through watchers.

Is it possible to have a filter to hold potential dups and not show them again till you flush the filter?

>>12939

>>12936

Thank you for these reports. I am sorry for the inconvenience. This is fixed in today's release. The new menubar open/close detection code was misfiring on OS X, initialising to 'on' for some reason, and a legacy 'don't process shortcuts while a menu is open' test was blocking anything from being processed in OS X until you open and close the right kind of menu and reset the system.

>>12937

Thank you for this report. When you say 'deleted', was the file in the trash or was it completely physically deleted? If so, do you remember how you got a thumbnail for it in the first place to right-click->show in a new page on? Was that original thumbnail correct, or also the hydrus 'error' icon that shows on 'remote' files?

Typically, hydrus will dismiss physically deleted files from view as they leave the file system. Were you by chance looking at an 'all known files' view or anything, to make them show, or was this from a right-click menu on the importer somewhere?

I think a thumb knows that it was previously deleted, even if remote. I think it makes sense to have an icon in this case, even on the 'hydrus' thumbnail.

>>12938

Yeah, they are about $200 on Ebay now. Only legit alternatives are Razer products, which I don't want to move to since I don't want to have a cloud account just to make my buttons work. There are screws underneath and the whole thing comes in half pretty reasonably. The buttons at the top have additional delicate inserts you will want to be careful don't snap off and fall down a crack, but the main thing is a couple of plastic/silicone sheets that do the buttons. I cleaned those and the button front carefully, let it all dry, and put it back together and it was better. The thumb button below the stick is apparently always the first to break–it just has bad plastic in or something.

For the shortcuts, I think both the media viewer and thumbnails consume shortcuts in the 'media' set, so anything in there will apply to both, but the different kings of media viewer will pull from media_viewer, media_viewer_browser, or the two _filter sets.

I think it makes sense to have a separate set for thumbnails available. I won't do it today, but next week I'll set something up with appropriate shortcut commands.

>>12942

That is a nice idea, but also a decent bit of work. Maybe something like a popup window in the regular bottom-right toaster that says '1 file rated x' that then dies after a second? That might be shit to look at, but I could add it quickly.

>>12943

Unfortunately, having downloaders know about deletions after the fact is tricky. Completely downloader items only know file hash and whether it was new/already in/deleted at time of import. They aren't plugged into the content update pipeline and only cause additional file metadata to be loaded in the thumbgrid when thumbs are called to be displayed.

Are those 154k/139k counts are related to this duplicate search domain question? And the ~150k count seems low compared to 4M? Assuming you have done your full search at 0 distance or whatever, that sounds correct–it is only searching the files that have potential matches found in the duplicates system. Having 1/15 to 1/10 duplicate files is typical, and since you have 2.6M eligible files, it sounds correct to me.

There's no good way to do a complete wipe. The old system had some hacked-together commands, but it never worked well. This new system, once it is complete in a few weeks, will have logically consistent 'dissolve' commands that will be able to nuke everything. You can do this if you like, but I suspect your duplicates are fine.

Let me know if I misunderstood. And please bear in mind I am actively overhauling this system right now. Potentials will move over in a couple of weeks, going from pairs to groups, which may fundamentally change how this queue is presented in ui anyway.

>>12945

Not yet. You can hit the 'skip' button, which I think is up arrow shortcut by default or you can hit the double-right arrow icon button, but the same pair is likely to come up in a future filter.

Best solution for now to put it off indefinitely is to exclude it in the duplicate file search domain. You could try adding a 'don't dupe filter' local tag or something and exclude that, or through a system tag if that makes sense.

>>12967

Thank you.

I appreciate the frequent updates.

>>12977

Thanks for clarifying. Yeah, I'll stick the trash icon on 'remote' files that were known to once be in the client. I think thumbnails know this already, so I can just draw the icon on.

My thing with these game controllers is having the side joystick. Feels so good to WASD with just my thumb, I can't see myself going back to a regular keyboard for vidya. I may bite the bullet of a Razer. I did some searching a while ago, and I think you need to 'initialise' your Razer controller with the cloud account, then you set your shortcuts, and can then set it to 'offline' mode. You can't edit the shortcuts completely locally and it is seen as a default device until you do it the first time. When my G13 finally packs up, I'll just have to bite the bullet. I'm ok with having a cloud bullshit driver if it is only ten minutes before I firewall it, even if I find it atomically stupid.

I understand your count issue better. I am not sure about the million files. Normally, a handful of files will be discarded if they have no useful search data–such as completely black/single-colour images–but that does not account for so many. As I do more of the current duplicates work, I will be integrating the 'phash regen' that that maintenance stuff refers to to the new file maintenance manager. If your client has somehow forgotten or missed that big chunk of files, it will be easier at that point to requeue them for generation. I'll have some UI for it under the database menu. At that point, I recommend you do something like 'regen missing search data for: system:filetype=png/jpg'.

Adding skip is a bit tricky just because it adds complexity in storage, workflow, and UI. I haven't reworked the potential data storage yet so I can't yet say cleverly if this would be ok or an utter pain in the neck. I will keep it in mind, but I suspect I will not add it.