/hydrus/ - Hydrus Network

>>7675

>>7682

This worked. It still shows up in the saved session from my previous similar search but does not in any new similar search.

>>7697

Got a minor error after deleting that saved session and then clicking on "pages" again

RuntimeError
wrapped C/C++ object of type Menu has been deleted
  File "include\ClientGUITopLevelWindows.py", line 727, in EventMenuHighlight
    menu_item = menu.FindItemById( event.GetMenuId() )

Seems it spams the message until it gets done deleting the session. I also continue to receive 1 for every time I click on "pages" afterwards.

I can't import my database from a backup on Linux.

>>7700

I solved it by importing my database from an older version, but it still gave me a good scare.

Ok hdev could use a bit of help here

Got a 3tb drive over christmas for images

I want to move the archive over to it, but retain the archive in its current location for 3 months, just in case of crib death of the new hdd.

How should I do this?

>>7703

I don't see a problem with just copying the Hydrus folder (and especially the "db" subfolder that should contain everything important on default settings).

Or you can use the backup feature under "database" if you prefer.

Don't know if you're aware, but the current dev for 4chan X put out a thread in 4chan's /qa/ recently asking if anyone had feature requests. Of course the mods deleted it because they're all terrible people who should be taken out back and shot, but the thread continued in its archive for anyone who uses 4chan X anyways.

Point being, the dev himself mentioned some concerns about 4chan removing legacy captcha and adding features for 8ch and other websites, which hopefully could fix some of the crashing and other bullshit. This got me thinking, is there any way you and 4chan X dev would ever collaborate to make your applications share a workflow?

For instance, in 4chan X I watch, post, and comment in threads until they die, or just pin them in the 4chan X thread watcher and wait for them to archive (or die in which case 4chan X redirects to an external archive), then "harvest" them by setting up Hydrus download tabs and doing a lot of copying and pasting. It would be really useful if instead of that I could hit a button to either export all dead threads, or open up a menu from which to pick threads to export, pick a name for the batch of threads, then hit export and get something easy to import to hydrus, maybe an image like the ones used to share hydrus scripts. Only instead of download scripts, it would just cause Hydrus to create a parent page filled with child pages of 4chan download pages (or archive download pages/page of images downloads, depending on if the thread is archived on-site or off), with the links pre-filled and maybe some tags pre-filled as specified from entry boxes or presets in 4chan X. Maybe they'd be paused by default and the user could start the downloads at leisure.

Meanwhile, back in 4chan X, you could hit the button for "yes I successfully imported to Hydrus" and it would remove the links from the specified threads from its thread watcher.

Most of this would probably have to be coded on the 4chan X client side, but I though it would be an interesting thing to bring up at least.

Client doesn't boot for me after this update. windows 10 creators update. intel CPU, last update did work. The client.exe grabs 264.8 MB of ram and then stops doing anything. windows defender is running really fast, but nothing is happening

Same here. client_debug output:

2018/01/12 16:41:22: Traceback (most recent call last):

File "client.py", line 82, in <module>

controller.Run()

File "include\ClientController.py", line 1015, in Run

self.frame_icon = wx.Icon( os.path.join( HC.STATIC_DIR, 'hydrus_32_non-transparent.png' ), wx.BITMAP_TYPE_PNG )

wxAssertionError: C++ assertion "strcmp(setlocale(LC_ALL, NULL), "C") == 0" failed at ..\..\src\common\intl.cpp(1579) in wxLocale::GetInfo(): You probably called setlocale() directly instead of using wxLocale and now there is a mismatch between C/C++ and Windows locale.

Things are going to break, please only change locale by creating wxLocale objects to avoid this!

2018/01/12 16:41:22: hydrus client shut down

>>7707

client_debux output is:

PyInstaller Bootloader 3.x
LOADER: executable is C:\Hydrus Network\client_debug.exe
LOADER: homepath is C:\Hydrus Network
LOADER: _MEIPASS2 is NULL
LOADER: archivename is C:\Hydrus Network\client_debug.exe
LOADER: No need to extract files to run; setting extractionpath to homepath
LOADER: SetDllDirectory(C:\Hydrus Network)
LOADER: Already in the child - running user's code.
LOADER: Python library: C:\Hydrus Network\python27.dll
LOADER: Loaded functions from Python library.
LOADER: Manipulating environment (sys.path, sys.prefix)
LOADER: sys.prefix is C:\HYDRUS~1
LOADER: Setting runtime options
LOADER: Initializing python
LOADER: Overriding Python's sys.path
LOADER: Post-init sys.path is C:\Hydrus Network
LOADER: Setting sys.argv
LOADER: setting sys._MEIPASS
LOADER: importing modules from CArchive
LOADER: extracted struct
LOADER: callfunction returned...
LOADER: extracted pyimod01_os_path
LOADER: callfunction returned...
LOADER: extracted pyimod02_archive
LOADER: callfunction returned...
LOADER: extracted pyimod03_importers
LOADER: callfunction returned...
LOADER: Installing PYZ archive with Python modules.
LOADER: PYZ archive: out00-PYZ.pyz
LOADER: Running pyiboot01_bootstrap.py
LOADER: Running pyi_rth_twisted.py
LOADER: Running pyi_rth_pkgres.py
LOADER: Running pyi_rth_win32comgenpy.py
LOADER: Running pyi_rth__tkinter.py
LOADER: Running pyi_rth_mplconfig.py
LOADER: Running pyi_rth_mpldata.py
LOADER: Running client.py
2018/01/12 16:54:53: hydrus client started
2018/01/12 16:54:53: booting controller...
2018/01/12 16:54:53: hydrus client failed
2018/01/12 16:54:53: Traceback (most recent call last):
  File "client.py", line 82, in <module>
    controller.Run()
  File "include\ClientController.py", line 1015, in Run
    self.frame_icon = wx.Icon( os.path.join( HC.STATIC_DIR, 'hydrus_32_non-transparent.png' ), wx.BITMAP_TYPE_PNG )
wxAssertionError: C++ assertion "strcmp(setlocale(LC_ALL, NULL), "C") == 0" failed at ..\..\src\common\intl.cpp(1579) in wxLocale::GetInfo(): You probably called setlocale() directly instead of using wxLocale and now there is a mismatch between C/C++ and Windows locale.
Things are going to break, please only change locale by creating wxLocale objects to avoid this!

2018/01/12 16:54:53: hydrus client shut down
LOADER: OK.
LOADER: Cleaning up Python interpreter.

>>7704

The only way I know how to do it in program I think would move every file to the hdd, erasing them from the initial location.

If I move the db without going through the program I have a feeling its going to toss errors forcing me to revert and go through the program anyway.

I don't really want to do a full migration to the new drive where I don't have a backup till I know there will be no crib death, which is about 3 months from now.

The way I have files set up

I have 1 import folder on the g: drive, this one doesn't remove images as any image imported from here is one I hand selected to save

a second import on g: that is a bulk save, as in I found an archive I want to import and don't want the files past import

and one on the s: drive that is a temp import location because before I got the new g: drive I was pressed for g: space

I want to keep the files on the s: drive, but have all new interactions go through the i: drive for now, and in 3 months, I can delete the s: drive backup. and free up around 1tb of space.

>fixed some wx menuhighlight issues

>improved some shutdown code

>misc wx fixes

I've seen changelog entries like this for some time now, but there's no point in writing them. They don't tell the user anything at all. They either need to be more detailed so that people know what changes were made (the purpose of a changelog, right?), or they should be skipped entirely so as not to waste space in a changelog which is already rather large every week.

>>7699

>>7698

>>7702

Thank you for these reports. I have had a couple of crashes myself. They seemed to come after one of these unhandled RuntimeErrors, so I think the new wx has trouble cleaning up when one happens, which can either happen in regular usage when the garbage memory is cleaned or on shutdown when it is forced.

It is odd you got that menu error as I specifically wrote a handler to catch that issue and deal with it. I am going to have another look at it.

EDIT: I just tried the same thing, deleting a session, and then I got it. It looks like wx menus don't report their dead status like other wx objects, and menu close events that would normally clear them away isn't always firing. This seems to have been worse on submenus several layers deep, like the save/load session stuff. I have rewritten how these menus die and now cannot reproduce the error. Please let me know how you get on in v290!

>>7700

>>7701

I am sorry to hear you had trouble. Can you describe in more detail what you tried and what errors, if any, you received?

>>7703

>>7710

Unfortunately, there is no way to split storage spaces in hydrus by import date, so you cannot 'freeze' your existing archive on the old drive and only put new files on the new drive.

I recommend using a third party program like FreeFileSync to create or modify your backup regime so that anything on the new drive is mirrored to a place you know is safe. Use the database->migrate db dialog to put a little bit, say 1/10th, of your collection on the new drive and slowly increase the 'weight' of this new location over time.

If you haven't seen it, here is a bunch more info on moving the client around:

http://hydrusnetwork.github.io/hydrus/help/database_migration.html

But let me know if that doesn't answer anything you need.

>>7705

Great, thanks for letting me know. I guess the slight mismatch here meant your commas weren't getting initialised in the old code.

EDIT: Seems like your fix breaks people on Win10 creator's update. I've tweaked the code for next week, hoping to keep your fix but skip it for them–please let me know what you see then.

>>7706

I am not good at working directly with others, but I am very open to making hydrus more open for others to work with. I have a long-term plan to create an API for the client so external programs (or scripts you write) will be able to tell it 'open a new thread watcher for [URL]' and so on, but this will be a big job. When the time for it comes around, I'll be interested in a conversation with any devs who want to talk to hydrus on the sorts of things they are looking for.

If there are simple things you can think of to improve your workflow speed in the meantime, please let me know. I think I'd like to add some better paste/drop method for thread watchers. Some better way for the client to catch or be informed or URLs. I was thinking of maybe popping up like a 'dropoff' area that will treat drag-and-drop drops differently, but I am not really sure. Something like 'drop multiple threads' support that could do the create many child pages thing you are thinking of, or an easy way to import from encoded png data. I still expect to add some kind of multiple-thread-per-watcher-page iupdate in this downloader overhaul, so maybe that new system will give us some new ideas for this stuff. Maybe something as simple as 'copy all watched thread urls' so you could then inform the external program of what to clear.

>>7707

>>7708

>>7709

Fugg, thank you for this report. Please revert to v288 for now. Seems like the better comma support for old windows breaks on new Windows. Do either of you have an unusual language/keyboard combination?

I have had a look at this and believe I have it fixed, but I don't get the problem so I can't test it myself. Please let me know what v290 does for you.

>>7708

>>7717

I've got en_US International and en_DE though Windows likes to add en_GB and de_DE randomly.

I'll report back once 290 is out.

>>7717

Me:

>>7709

>>7707

I have a microsoft ergonomic keyboard, and my settings are british english. My real location is in the netherlands, does that matter?

>>7719

>>7718

Thanks. The problem here is that there are two things that handle localisation stuff here, but one gets pissy in your Windows if I use the other in any way. However, I only got the other guy's commas to show up if I initialised both. I'm wondering if maybe the pissiness is increased if there is a mismatch between system language and keyboard language, but since neither of you are Pashtun keyboards on Russian systems, I'm now guessing it is more down to different Windows versions. I'm on a couple different versions of Win10 and am fine, so I think maybe the creator's update stuff is a bit stricter about some things?

In any case, the attempted fix next week is to only initialise the second locale thing if it looks like the first didn't do the job properly. This should let you boot and will hopefully keep the comma fix in place, but we'll have to see.

Hmm, after the past couple of updates it seems to crash rather frequently. How do I get to the logs? It usually happens when hydrus is just running normally in the background, although once it did crash in the middle of a ptr sync as I was actively using the program.

>>7721

Thank you for this report. I recently made a big update to my ui library, and the new version is more strict about certain things. I have fixed several crash bugs in the past two weeks and am working to iron out the last ones.

Can you describe any behaviour that seems to increase the likelihood of a crash? Does it happen for you after a lot of video playing, or a lot of popup messages, or any other patterns? For example, one last week was that the program would crash on client close if a page was still in the 'undo' menu.

I am currently working on a crash that seems to happen about five or ten minutes after receiving a certain 'Runtime: dead object' error popup which seems not to be being cleaned up neatly. Have you seen any of these error popups before your crashes? The crash itself will not be written to the log, but you can check what was happening beforehand by checking the 'client - 2018-01.log' file in your install_dir/db directory.

Also, which OS are you on, and which version, and how are you running the client–from one of my releases or a different way? Do you tend to have a client with a lot of pages with many thousands of thumbnails, or do you run leaner?

>>7716

I wont have an actual db backup drive for at least several months, the small backup im going to have is incase of crib death (the first 3 months of a hdds life where it fails or it lasts for years) and after this time frame its going to be deleted.

right here

"please do not move their folders around yourself–the database has an internal 'knowledge' of where it thinks its file and thumbnail folders are, and if you move them while it is closed, it will throw 'missing path' errors as soon as it boots. "

Is my main concern, what I want to do is this

take the files from G: and move them to I: Then have the database only work with I: without removing the files from G:

I have no idea how to do this in program at all, as the only in program method I believe will delete files from G: and move them to I:

The second part where I inform the database that shit has moved with the –db_dir= seems like it would work, at least for the time being, but if there is a way to do this in program that would be much better.

hey hydrus_dev, what would you think about adding lepton support?

https://github.com/dropbox/lepton

>>7727

custom gallery parsers when

>>7720

Hello.

I have client crashes after some period of time (approximately 15min). It started to happening when I updated Win10 to the last version. If you need more info, just tell me what to do.

>>7726

Thanks, I see what your problem is now.

Since you have an unusual situation, I think you'll want to skip my advice in the help here and move the files manually, while the client is shut down. I think this sort of thing will work:

Shut down client

Copy from G:\my_media to I:\new_my_media in explorer

Rename G:\my_media to G:\my_media2 (or move it somewhere else: just change its location so the original path to your files is gone)

Boot the client–it will complain about not being able to find G:\my_media and give you a dialog that will let you reassign the path. Tell it to look at the new location on I:

Make sure everything looks right in the migrate database dialog

Rename G:\my_media2 back to the original name

You know your situation better than me, but your 'my_media' folder is probably something like G:\hydrus\db\client_files. It should have 256, 512, or 768 subdirectories named like 'fd1'.

And then if you would like to move the client*.db files as well, you can do that in the program or manually. If any paths mess up doing it manually (because they were 'portable' in the migrate dialog and did not come along with the .db files when you moved them), you'll get the pre-boot 'fix the paths' dialog again. If you move the install and the db subdirectory to the new location as one unit, then you won't have to do the –db_dir since 'db' is still right beneath the client.exe, but if you split the installation with the executable from the 'db' folder, then you'll have to tell the exe where to find it with the shortcut switch.

Anyway, let me know how it goes. I recommend only trying one step at once.

>>7727

I'm afraid I hadn't ever heard of it before you linked it here, so I can't talk about it confidently. Is it a legit thing or a bit of a meme? Have you tried it before and had success? Does it load and run fast? I can't see a python library, so we'd probably be instantiating it from the command line every time.

If we wanted to implement support, we'd probably want to write another layer in the file storage system that could support arbitrary additional conversion calls for particular types of file (in this case jpegs). This might be neat to support in general for any sort of command line program, since you could add quick-and-dirty per-file encryption support or any other zipping program if you wanted.

I don't think I can fit this sort of additional support into a regular week's small work, but if enough people would like it or come up with different uses for the broader additional file conversion layer.

>>7728

First stuff should roll out in the next few weeks!

>>7732

Thank you for this report. I am sorry you are having so many crashes. I have fixed some more crash stuff in the release I am putting out today, so I would appreciate feedback on whether the situation is better in v290. I would also like to know what sort of client and usage you have:

Do you typically have a few page tabs open or many? Do they have a few hundred thumbs or many tens of thousands?

Do you see many popup messages in normal usage before a crash? Are they regular stuff like subscriptions or errors for other things?

Do you watch many videos in the media viewer? Do you do many complicated searches, or do lots of tagging or rating? What sorts of things do you tend to do with the client in normal usage? Which windows and dialogs do you tend to open, if any?

The remaining crashes seem to be related to the new ui library not being happy with the order I delete things. If I can figure out what you might be doing that other users are not, then I may be able to replicate your situation or at least infer what is going on and make the respective delete process safer. This new Win 10 caused other problems this week–are you on the Creator's Update, or just regular? Home or Pro?

>>7739

Thanks for your help.

I am using Win10 Pro 1709 build 16299.192

My usual workflow:

1) Open hydrus

2) Import new images from HDD. Usually a few hundred.

3) Put proper tags on them

4) Archive images

5) Repeat step 2-4

At some point program crashes.

I have 1 or 2 open tabs. There are no popup messages before crash. I dont watch any videos neither I add them.

If I launch debug version of hydrus will it make a more detailed log, so I could provide it for you?

>>7739

legit, dropbox developed and uses it for their servers. they announced it about a year ago. I don't know of any software that supports it yet.

i ran a test on 302 jpegs. 1 threw an unsupported_jpeg error.

time to compress, real:40s, user:1m59s

time to decompress, real:15s, user:55s

Size compressed: 170.5 MiB

Size uncompressed: 203.8 MiB

Average compression of 16.3%

Adding support for arbitrary conversion calls sounds neat, but I don't think it'd be worth the effort. Probably best just to wait for a python library.

>>7740

Thank you for this information. I now have several leads on the last crash(es) and will work on it more this week.

Unfortunately, the nature of a full python crash means you can't really catch it in debug, especially at the user-level. By the time the crash starts, the whole error is passed up to the OS, which shuts things down in an emergency. I never get told anything, so the log (and anything else that was going on) just cuts off.

At least, this is as far as I know. If you like, you can try and launch the client with the client_debug.exe that comes with the release–I usually use this to determine boot problems, as it lists LOADER stuff and errors on that end, but maaaaybe it will produce a 'segfault' kind of error on crash. It will write everything to console, so you probably want to run it from the command line so you can see any error info.

Anyway, I will try to repeat the crash this week and see if I can figure out what causes or avoids it and improve my object deletion order. This seems to be most of the thing wx is unhappy with–some window getting deleted before its parent was willing to let it go, that sort of thing.

>>7741

Thank you for this. I don't have time to fit this in quickly, but I will absolutely keep it in mind in future. I'm a little hesitant to add additional complexity and overhead to the file system, but that space saving isn't anything to scoff at either.

Perhaps if similar compression comes out for small webms/mp4s, it might be time to jump on this and make a proper layer for it all. Might be worth writing a little analysis tool so users can see how much of their local_files space is in each mime. Afaik, average file size on most clients has been rising, up to something like 1MB now, driven by newer 5-20MB 720-1080p webms driving the average up.

>>7771

Time period between crashes seems more large

in version 290.

I tried debug version, the result was the same and no useful info in console.

Is it possible to catch the problem with dev tool (debugger or else), or there is no option only to make a guess about location? I can launch hydrus with debugger if it helps.

>>7778

I don't have a much experience with that sort of debugging, so I can't say confidently how doable and/or reasonable it is. If you know what you are doing, please do give it a go and let me know if you figure anything out. But my suspicion is that while you might be able to trace the problem to a particular location in the C/C++, linking that to the python end and actually figuring out what sequence of events went wrong may be too difficult to be worth spending time on. Maybe there is a debugger clever enough to figure it out, but I don't know about it.

I'm doing python-end debugging in WingIDE to figure some of this stuff out, but I know what I am looking for now with these crashes, so it is mostly a case of taking reports and looking in the right area for the order of object deletion.