/hydrus/ - Hydrus Network

>>7742

>- fixed an important grammar typo in the help

top kek

I haven't been following hydrus for a while now. Does anyone know if sankaku login has been implemented yet?

so, trying to use the thumbnails to add ratings, not able to…

There an option I have to tick and im not seeing?

also at least as far as feedback is concerned I have an idea

Right now, if I select multiple images, and some of them are tagged, It shows the tags with a (####)

Now here, when things are selected 1 at a time, you could have a total image selected counter above the tag window, and force sort by incidents when selecting images, this would kick in after 2 images are selected. this would show both how many images are selected and how many of them share the same tag. You could even go a step further and make any tag they all share bold.

Now also in the thumbnail view, you could also add a ratings viewer, have it act much the same way as it does in the media viewer, possibly have it work like tags and just be text, where it tells you in my case

safe/suggestive/explicit - 0

safe/suggestive/explicit - 1

safe/suggestive/explicit - 2

safe/suggestive/explicit - 3

0 being unrated and 3 being highest, may not be the prettiest way to do it but its something

then have

Total selected - 97

———————————————–

safe/suggestive/explicit - 0 (23)

safe/suggestive/explicit - 1 (2)

safe/suggestive/explicit - 2 (63)

safe/suggestive/explicit - 3 (9)

and lets say I want to change them all to safe so I hit what 1 would be

Total selected - 97

———————————————–

safe/suggestive/explicit - 1 (97)

and it bolds itself

>>7746

Not really an answer to your question, but I just made it so that certain tags implied the tag "lewd" so that I could search for/filter them. Could help apply those ratings I guess.

>>7742

For some reason, Hydrus doesn't recognize when I press buttons that aren't alphanumeric, backspace, enter, tab, escape, or modifiers. I noticed there were binidngs for - + = in the media viewer but they didn't work, and when I try to assign a key to a binding only those I mentioned work. They do work when I'm entering into a box like a tag box or search box. I'm on Linux using version 287 because the AUR hasn't updated in a while, so sorry if this has been fixed since them. Speaking of, wonder how hard/feasable it would be to make a hydrus-git package.

>7708 here, still the same error.

After commenting out locale.setlocale() it works however

2018/01/18 21:21:21: Traceback (most recent call last):
  File "client.py", line 82, in <module>
    controller.Run()
  File "include\ClientController.py", line 1020, in Run
    self.frame_icon = wx.Icon( os.path.join( HC.STATIC_DIR, 'hydrus_32_non-transparent.png' ), wx.BITMAP_TYPE_PNG )
wxAssertionError: C++ assertion "strcmp(setlocale(LC_ALL, NULL), "C") == 0" failed at ..\..\src\common\intl.cpp(1579) in wxLocale::GetInfo(): You probably called setlocale() directly instead of using wxLocale and now there is a mismatch between C/C++ and Windows locale.
Things are going to break, please only change locale by creating wxLocale objects to avoid this!

>>7742

>If you still get crashes in this version, please let me know the details of your client and your usage, including whether any behaviour tends to cause or avoid the crash.

Had a crash during an import last night. For the record, I'm the Win 7 guy who's Hydrus locks up the system/drive (typically for 3-10 minutes) every so often with the HDD light blinking on and off (for about a 1/2 second each). It hasn't happened quite so much lately, although previously it did seem to happen more often if I did a manual vacuum (I don't do these much anymore in hopes of preventing this lockup) and then shut down Hydrus immediately following. Usually waiting after the client shuts down for 3-5 minutes prevents any issue from happening; if I try to open taskmanager or recycle bin to clear out imported files, the system will basically be locked up (mouse changes to the spinning "loading" icon) but does not affect music already playing back, including if I pause/unpause Foobar.

Anyway, import was business as usual. Had a total of about 800 files, 300 of which were text files (tags). Hydrus got roughly 10 files into the import before locking up as described above (this is unusual, it does not typically happen while the client is running and when it does, it is during an import but the duration is extremely low, perhaps a minute or 2. However, import will sometimes appear to stop working for a minute or 2 even under normal circumstances; system is not locked up during this time, although Hydrus usually is). It stayed that way for ~5 minutes before I attempted to pop open taskmanager to see if the client was even still working. As usual, it wasn't responsive at first and once it finally popped up when things went back to normal (no blinking HDD light, etc. This is typical for this issue.). However, once taskmanager had popped up and things had gone back to normal, I got the standard notice that "client.exe has stopped working" and had to close.

Restarted and continued my import; worked as normal, albeit slower than usual. The ~500 files I had to imported worked out to being ~350 once duplicates already in db matched themselves. Nothing about the import itself was unusual, nothing about my particular locking up issue was truly that unusual (other than the issue itself and me seemingly being the only one with this issue), yet it crashed which IS unusual.

I think I'll try having resource monitor going during the lockup; it might give some insight on if the HDD is actually being read/written/accessed or even if it's actually the client doing all the writing. I vaguely recall debug mode not catching anything out of the ordinary so unless you've updated how/what that reports, I'm not sure it would catch anything (particularly since the lockup after shutdown happens after client.exe has cleft taskmanager's list of processes).

Got this error by doing the following: Open manage tag window, add a bunch of tags, pressed esc and then "yes". At first it seemed like nothing was wrong but the program crashed a few minutes later.

Haven't been able to reproduce this though, so not sure what's going on.

RuntimeError
wrapped C/C++ object of type ListBoxTagsSuggestionsFavourites has been deleted
  File "site-packages\wx\core.py", line 3189, in <lambda>
  File "include\ClientGUITagSuggestions.py", line 302, in _SetTags
    self._tags.SetTags( tags )
  File "include\ClientGUIListBoxes.py", line 1900, in SetTags
    self._RecalcTags()
  File "include\ClientGUIListBoxes.py", line 1876, in _RecalcTags
    self._RefreshTexts()
  File "include\ClientGUIListBoxes.py", line 691, in _RefreshTexts
    self._SetDirty()
  File "include\ClientGUIListBoxes.py", line 732, in _SetDirty
    self.Refresh()

>>7742

>>7750

>If you still get crashes in this version, please let me know the details of your client and your usage, including whether any behaviour tends to cause or avoid the crash.

Also still getting rando crashes during regular operation. Dunno if it's specifically tied to clearing/loading sessions but that's what I've been doing. I've gotten crashes thus far from:

1. Loading session, minimizing the client and it crashing upon return

2. Loading session, doing some similar searches and it crashing in the middle of media viewer use

3. Loading session, doing some similar searches, overwriting session and then it crashing when I try to clear all closed pages

Would setting debug mode be useful for catching more info on these?

>>7742

The macOS build seems broken, at least under 10.13.2. When I open hydrus I just get the spinner (with no startup splash showing) and the application hangs, have to force quit via Activity Manager.

I've tried updating from 289 and using it as a standalone (no upgrade, just straight up booting the version 290 app), same result with both.

No issues with 289.

>>7707

>>7709

just like in 289 hydrus is starting, but not launching to the splash screen even. 287 worked. How would I go about downgrading my version to that?

client_debug.exe output:

PyInstaller Bootloader 3.x
LOADER: executable is C:\Hydrus Network\client_debug.exe
LOADER: homepath is C:\Hydrus Network
LOADER: _MEIPASS2 is NULL
LOADER: archivename is C:\Hydrus Network\client_debug.exe
LOADER: No need to extract files to run; setting extractionpath to homepath
LOADER: SetDllDirectory(C:\Hydrus Network)
LOADER: Already in the child - running user's code.
LOADER: Python library: C:\Hydrus Network\python27.dll
LOADER: Loaded functions from Python library.
LOADER: Manipulating environment (sys.path, sys.prefix)
LOADER: sys.prefix is C:\HYDRUS~1
LOADER: Setting runtime options
LOADER: Initializing python
LOADER: Overriding Python's sys.path
LOADER: Post-init sys.path is C:\Hydrus Network
LOADER: Setting sys.argv
LOADER: setting sys._MEIPASS
LOADER: importing modules from CArchive
LOADER: extracted struct
LOADER: callfunction returned...
LOADER: extracted pyimod01_os_path
LOADER: callfunction returned...
LOADER: extracted pyimod02_archive
LOADER: callfunction returned...
LOADER: extracted pyimod03_importers
LOADER: callfunction returned...
LOADER: Installing PYZ archive with Python modules.
LOADER: PYZ archive: out00-PYZ.pyz
LOADER: Running pyiboot01_bootstrap.py
LOADER: Running pyi_rth_twisted.py
LOADER: Running pyi_rth_pkgres.py
LOADER: Running pyi_rth_win32comgenpy.py
LOADER: Running pyi_rth__tkinter.py
LOADER: Running pyi_rth_mplconfig.py
LOADER: Running pyi_rth_mpldata.py
LOADER: Running client.py
2LOADER: OK.
LOADER: Cleaning up Python interpreter.

Is there a way to migrate installations? I want my Windows database and all it's associated settings to be moved to a Linux installation on the same drive, but I could not find an easy way to do this.

>>7761

just copy the db folder

I think some of the options pages could use a "reset to default" button. Specifically the maintenance and processing, and speed and memory pages. Those are kind of critical and if you make a wrong change it would be nice to be able to easily get back to the "recommended" settings.

Hydrus has crashed twice now while using the dupe filter. Both times when rapidly switching from one image to the other.

>>7747

I have a small set of ratings right now

safe/suggestive/explicit is as it sounds, not porn, not porn but have fun proving it, and porn

real/drawn

art

funny

favorite

pony

tag later - this one implies there is something special about the image and I damn sure need to catch it.

now, every image will go through this and it will filter every image into 6 categories, with 2 of these optional ticks being endpoints that every image could fall into too.

Personally there is very little I want to 100% tag images for, or tag all my images, but everything I keep, I want to be able to say

"Yea, the image was safe, drawn and funny"

There will be a few more 'high speed' ratings, like comic, one page, webcomic, part of something bigger that I could feed these through when i'm doing more in depth tagging, but up front I just want a few things to think of to rapidly go through them.

If you have had trouble with booting the client in Windows due to a 'locale' problem, please try this build:

http://www.mediafire.com/file/8p8z1g1yn4j3778/Hydrus%20Network%20290%20no%20locale%20-%20Windows%20-%20Extract%20only.zip

I will make the changes in this version permanent starting next week.

>>7744

Not yet. I'm starting on the new parsing system this week, which the new login manager needs. Shouldn't be all that long, although I can't give a firm estimate.

>>7746

Can you describe what you tried to do to set ratings to your thumbnails? I would expect:

Select some thumbnails.

Hit a shortcut key in the 'media' 'reserved' shortcut set that sets a rating.

Hit F4 to open manage ratings to check they got set.

If you are trying the shortcut, does it work in the media viewer? Is the shortcut in a 'custom' set (these aren't working on thumbnails (yet))?

I like the idea of summarising ratings on thumbnails in some way, although I don't know where I would fit it in. I sort of like the idea of having something small pop up when you hit the shortcut. I also think putting the top-right ratings display in the preview window would be both helpful and easy to add.

>>7747

The v287->v288 switch is the big wx update, I think. I can't talk confidently about how the old wx handles shortcuts in your Linux, but it might also be my code. I only map a certain subset of keys for a couple of internal reasons, but backspace, enter, tab should be ok with ctrl and shift and alt for any mapping (except where some OS-level global hook jumps in beforehand, like alt-tab). Esc won't map in most cases because wx likes to use it as a dialog-cancel. Some stuff like ?-\/ isn't there yet, but I've made a job to add it.

Just so I can verify–are these shortcuts working ok when you enter them (and so you can save them to the shortcut set ok) but not when you actually enter them in the media viewer? I've had trouble getting Linux media viewer focus working properly–if you click on the 'white' background of the viewer (moving focus to it), do they work then? What if you specifically click on the media itself, or one of the hover windows?

I don't know anything about making packages, so I can't help there. If you do, feel free to package and distribute whatever you like.

>>7749

Thank you. I'm changing my policy to be, "fuck locale", then. If someone doesn't get correct commas, I'm going to assume it is an OS-level problem with setting language preferences and not try to fix it my end. I'm reverting back to the safe way of instantiating this starting next week, which is the build I link above. I apologise for the inconvenience.

>>7752

>>7750

Thank you for this comprehensive report. In your first crash, during the import, did you import the files by going through the file import dialog (where it lists all the possible files to import and their size and guessed mime)? I believe this dialog is not cleaning itself up nicely, leading to delayed crashes.

Unfortunately there is not a good debug way to catch crashes–they are at a lower level than I deal with (i.e. the crash literally means it can't deal with the errors itself and passes it up to the OS).

I will keep working on this and hope to have it working better for v291. I apologise for the inconvenience.

>>7751

Great, thank you! This is very similar to other crash causes, so I think this was it! I'll have it fixed for v291.

>>7753

I am sorry to hear this. Please get the tar.gz release and extract it somewhere new and then open the client_debug executable in a terminal. (Right-click on folder->open in terminal, then type './client_debug' and hit enter on the new terminal window that opens up). This should print a lot of LOADER statements to the console. Take a screenshot of that and post it here or mail it to me.

>>7754

Please try this new build:

http://www.mediafire.com/file/8p8z1g1yn4j3778/Hydrus%20Network%20290%20no%20locale%20-%20Windows%20-%20Extract%20only.zip

If you would like to roll back to v287, just download and install/extract v287 as you would any other release:

https://github.com/hydrusnetwork/hydrus/releases/tag/v287

>>7761

>>7764

Yeah, just move the install. Everything user-created is portable and stored in the 'db' folder, so do something like this:

Make/update a backup, so you are safe if anything goes wrong.

Move Windows install's 'db' directory to a space Linux can access it, if it isn't already.

Boot Linux

Extract Linux build somewhere.

Merge the 'db' folder into the Linux install's one. You really care about the .db files and the 'client_files' folder.

Boot the linux client.

I've also written this document as background reading:

http://hydrusnetwork.github.io/hydrus/help/database_migration.html

Let me know if you run into any trouble.

>>7763

Everything is fine. These are repository update files, which are kept even after processing is done. They are stored in the same file system as everything else, but they are hidden from user presentation in almost all contexts.

They are gzipped json, iirc.

>>7766

Thank you for this suggestion. The options pages are a big mess right now. I want to rework the whole thing and split it up into three or so lighter-weight dialogs. Adding 'reset' buttons to each page in a uniform way is a good idea.

>>7768

Thank you for this report. I will check this this week.

I have this habit of importing\exporting files a bunch of times to reorder them a bit before importing them for good, as such sometimes i don't want "previously deleted pictures" to be hidden when i reimport them

But in 290, even tho there's the option to uncheck "exclude previously deleted files", it doesn't stay saved, i apply it, retry to import, and the files are still hidden because the option rechecked itself

Where do i have to changed it so it stays saved for all future imports until i change it back?

>>7752

>>7774

>did you import the files by going through the file import dialog

Correct. Further, I ran a long session of similar searches and it only crashed right after attempting to clear all closed pages. I then did a few more sessions (3-4) of similar searches but without clearing closed pages and it only crashed once after about an hour of activity.

>Everything is fine. These are repository update files, which are kept even after processing is done. They are stored in the same file system as everything else, but they are hidden from user presentation in almost all contexts.

Interesting, I considered that possibility but discarded it. Sorry for the panic.

>>7772

>>7754

First tried it with a copy of my current DB, and that worked splendidly!

another kinda stupid quation, does hydrus save the filename even when you don't set it up to do so? the program imported a big import folder with a 130 files series before I set it up to set the filename tag and now I'm afriad the order is lost. Downloading the set again to overwrite the images is near impossible.

>>7774

I removed all the files from my Windows install except the db folder and put the Linux files in and everything seems to work. I got this error, but I don't think it's related to the db migration.

wxAssertionError
C++ assertion "win" failed at /home/ubuntu/wxPython-4.0.0b2/ext/wxWidgets/src/gtk/menu.cpp(84) in DoCommonMenuCallbackCode(): event for a menu without associated window?
  File "include/ClientGUICommon.py", line 3146, in EventDirty
    self._func( *self._args, **self._kwargs )
  File "include/ClientGUI.py", line 3726, in RefreshMenu
    self._menubar.Replace( old_menu_index, menu, label )

>>7753

>>7774

Thank you, here's the output from client_debug: https://pastebin.com/4jJXnfxi

I'm using Python 2.7.10 when I run python –version (comes as default with macOS). Although you probably bundle in your own Python executable I'd assume?

>>7753

>>7774

>>7786

Although I've just noticed that I get the same error when running client_debug in 289. So that's probably not the issue here (289 works fine as I've said earlier).

>>7753

>>7774

>>7786

>>7787

And finally an excerpt from the system log after I force quit version 290 when it hangs: https://pastebin.com/aR1WtxpC

Not sure if there's anything useful in there for you. But it's weird that the Binary Image shows the path to the 289 version I have in my Applications folder, even though I'm running 290 from the Downloads folder.

Deleting from 289 and moving 290 there instead doesn't make a difference, still hangs on startup before the splash screen appears.

Is there anything else I can provide you with? I'm developer myself (not Python though, mainly JS and PHP) so feel free to get technical :D

>>7753

>>7774

>>7786

>>7787

>>7788

**deleting 289 from the Applications folder

>>7721

>>7725

>>7742

Hey hdev, sorry I didn't get back to you -- I don't usually leave a lot of pages up. At most, it's a files page on system:everything (~5000 images), and a couple pages for importing from a webpage/local files/raw urls. That's the most I'll ever have going.

I'm on Windows 10, it only started happening with last week's version (289). I don't notice anything unusual when it happens. No strange messages, nothing. It crashes mostly when the client is idling in the background.

>>7791

found this in Hydrus Network/db/client - 2018-1.txt

2018/01/20 23:29:24: Traceback (most recent call last):
  File "include\ClientNetworking.py", line 1646, in Start
    raise HydrusExceptions.ConnectionException( 'Could not connect!' )
ConnectionException: Could not connect!

2018/01/21 17:39:58: Traceback (most recent call last):
  File "include\ClientNetworking.py", line 1646, in Start
    raise HydrusExceptions.ConnectionException( 'Could not connect!' )
ConnectionException: Could not connect!

2018/01/21 17:39:58: Failed to refresh account for public tag repository:
2018/01/21 17:39:58: Could not connect!

>>7792

(This was on version 289)

I am getting the random crashes with v290 as well, on Windows 7, though this is the only Hydrus version I've tried.

So far it's only happened during times of low activity (no DB processing going on). Once while filtering duplicates, once while filtering inbox, and once with just a search page open w/ 200 thumbnails when I wasn't even interacting with the program (though not idle long enough for maintenance/processing to start).

Also, this is likely only coincidence, but the random crashes only started after I finished the PTR sync and migrated my client files to a separate location (using the dialog within Hydrus). However, I also changed a bunch of options, such as increasing the idle timers, so I can't really narrow down what specifically might have caused it.

Let me know if there is anything I can do to help debug the crashes. There are no errors in the log when they occurred. There were some older errors about being unable to access "\appdata\\local\\temp\\hydruspjpmcs" because the file is in use by another process, but I assume those are unrelated.

>>7794

I want to add that in moving the database to a new hdd, I found 4300 extensionless files, I cant find out what the hell they even were, and if this was a 'my hdd is dying' issue 4300 files that only hit hydrus images is a bit… unlikely.

Ok, I have a question.

Is it possible to have more file definitions?

I mean currently there is inbox, trash and archive?

I have 1.4 million files, when my archive was still in good condition, something around 200k exact matches on the duplicate list, and I want to get around to dealing with that, my issue I have been thinking of the solution was a way to tag deleted files for their deletion reason, i'm not able to leave well enough alone, and every time I see 'this file was previously deleted' I have to check it every goddamn time and everything but around 100 files I can clearly see why I got rid of them, the 100 that were deleted I can't find a reason for, I can only assume it was because they were alternates or lower resolution versions.

I don't think I can deal with this through ratings, as the images would still clearly show up and it would also be very easy to accidentally an image.

if i'm able to put them in folders, and lets say I can have a folder treated like trash, I could use the dup detector and dump every image from there into an 'I have higher quality version' folder, have it act like trash, but not lumping everything in with trash

When im going through images I import, I could easily send images to shit quality / bait / image macro folders, just things I don't want to keep and be done with them, and either keep them as a look up 'its already in archive' when importing, or if a way to take why something is deleted, I would easily be able to en mass tell the program why and be done with them so when I see "Previously deleted" In the import bar, and I look it up I can see "Deleted | day | day | Source | Reason I got rid of it" and be done with it.

I don't know how big of a problem this is or who's problems this may solve, but its one that I have.

>>7775

Thank you for this report. I am sorry you have had trouble. Unfortunately, I was not able to reproduce it. It sticks for me whatever I set it and no matter what the default is. Which OS (and OS version) are you on?

Could there be any mechanical workflow issues, like are you navigating through the 'file import options' panel with tab and enter, or are you clicking the checkbox and apply button? I am just wondering if maybe there is a focus issue that I am not encountering, where somehow the checkbox is getting hit again in the same moment as the apply event.

What are your default settings under options->importing? If you set it there, does it stick? This is the place to set a default for future imports, so it may as you say be a good stopgap until we can figure the larger issue out.

>>7776

Thanks. I did some more crash cleanup work yesterday. I hope the file import and session stuff is better if not completely fixed for you tomorrow.

>>7779

Great, I am glad we fixed it.

I am sorry, but the filenames are not saved. Hydrus deletes files to the recycle bin by default, so the original files may still be in there with the proper names. If so, undelete them to a new folder and then run the import again with the filename tagging options you want.

>>7783

Great. Thank you for this report. You are right–it isn't a problem with your migration, but another bit of fun from the wx update. I will look at it today or next week.

>>7786

>>7787

>>7788

>>7789

Ah, fugg, yeah, the OS X debug build needs to be built as a non-window application in order to write its debug stuff to the log, and then when it tries to actually run the app, OS X complains because it doesn't have window stuff built in with it.

Yeah, a python runtime is all bundled with the exe, so everything is supposed to just go on its own.

The log is odd, because it seems to be going fine. It is only dumping out on the final 'ok, let's get going with all this stuff' line once everything is loaded.

So it looks like the new version is having a problem actually doing that window stuff.

I had a pain in the ass time trying to get a newer version of OpenCV working for the OS X release this week, so I wonder if that is doing this. I am on 10.9.5 and it wasn't happy with the newer versions. This is super cheeky, but if you try to run a fresh extract of v290 but swap in the cv2.cv2.so file from v289, does it work?

I will work on this my end a bit, either updating OS X or making a test build tomorrow with the old OpenCV.

>>7791

>>7792

Thanks. I've done some work on this this week and will continue going if needed, so please let me know how v291 goes for you.

>>7794

The best way to immediately update the client where its files are is to boot it with the old location missing. It will throw up a pre-launch dialog that will list the missing paths and ask you for updated ones.

>>7802

Thank you for this report. This information is useful. I apologise for the inconvenience, particularly given that you have just started. I have fixed several possible crashes for tomorrow's release, so please let me know how it works for you.

The temp location errors aren't anything to worry about.

>>7803

Those are likely the update files for any repositories you have synced to. These are kept even after processing and are stored in the same way as your media files, but they are kept hidden in most user-facing contexts.

If you check your services->review services frame, you should see 'repository updates' local file service wrapped up in the 'all local files' domain. Hopefully, it has about 4,300 files!

>>7805

Thank you for this suggestion. I would like to add the ability to create multiple custom 'local file' domains, which would act a bit like folders, and I think would do most of what you would like here. Essentially, in the autocomplete dropdown where you can select 'local files'/'all local files'/'trash', you would be able to create new domains like 'sfw'/'nsfw' or 'to be deleted' or 'lower quality' or whatever. There would be a bunch of new ui to handle copying and moving files between these domains.

This will be a big job, so it will be on the next 'big thing to work on next' poll, which will be up when I am done with the current downloader overhaul.

>>7810

>>7789

I've tried using the cv2.cv2.so from 289, still getting the same error.

Thank you for looking into it. :)

>>7811

yep, comes out to exactly that amount i found.

with the more domains, just keep in mind an ability to have folders act like something else, like 'low quality' I would have act as trash bt a different name, that would get most of the issue. Ideally, having the ability to tag the reason something was deleted would serve this even better because the file would be gone.

With this you would also have to have something like dup filter where you are able to go in and instead of trashing exact matches, send them to a 'lower quality' folder.

>>7810

>>7779 (You)

> Great, I am glad we fixed it.

> I am sorry, but the filenames are not saved. Hydrus deletes files to the recycle bin by default, so the original files may still be in there with the proper names. If so, undelete them to a new folder and then run the import again with the filename tagging options you want.

my hero, I could just copy them from trash. This was incredibly helpful