/hydrus/ - Hydrus Network

Great work as always.

Would it be possible to make Flatpak build of Hydrus?

Thanks again my man, very exited about the new network engine getting there, I couldn't open my settings menu anymore, and I'm glad to report it's working now.

welp, my import folder that I have set to once a day, as imported 5 times so far...

also

>>7959

I am from the old 4chan days, I can't just let a threadwatcher go unattended/un checked, whatever method you choose for multithread they need to be able to bring up the images they got so they can be quick checked.

>>7982

as I submitted this, it imported again.

>>7983

I have turned this to paused, as I now know what the weirdness my computer was experiencing was likely due to this.

>>7985

I got a popup similar in size, it got pushed away by two import folders and the update message, is there something I should be afraid of?

>>7992

There's a bug thread here >>5115

Got a crash after tagging some stuff. Different from the other crashes, I actually got some windows error reporting pop-up. It didn't have any real info on its own, but here's the event log:

Faulting application name: client.exe, version: 0.0.0.0, time stamp: 0x00000000
Faulting module name: unknown, version: 0.0.0.0, time stamp: 0x00000000
Exception code: 0xc0000005
Fault offset: 0x000000d08d8d48ea
Faulting process id: 0xa8
Faulting application start time: 0x01d3a20814ee5ca6
Faulting application path: D:\Hydrus Network\client.exe
Faulting module path: unknown
Report Id: b5cd75a0-0dfc-11e8-b113-00224d80bd5e

And then within the same second:

Possible heap corruption detected (exception code 3221225477). Initiating further diagnostics.

I'm also still getting crashes after tagging stuff. Also, how do I produce logs?

>>7995

I just open up event viewer, the default administrative view option will show the crash itself.

>>7978

>>7962

>>7979

>>7980

Thanks lads. I pushed it a bit hard this week and got blind-sided by this IRL shit, but it all went ok so I am back to it if a bit frazzled. I will take it easy for this week, mostly doing cleanup and fixes and help, and then I'll reevaluate where we are. I might want to take another week or two writing testing code on all the new stuff we already have before I start pushing more new stuff with the gallery downloader.

>>7978

I don't know much about package managers, so I'm afraid I can't say anything clever about Flatpak. But if you know it and know how to use it, please feel to create whatever package you like and distribute it wherever.

>>7982

>>7983

>>7984

Did you figure this out? What was the weirdness–something you think the import folders were causing or something that was causing the weird import folders? I changed import folders to not run more than ten minutes at a time this week, which is my first guess at what you were seeing. Do you have a very large import folder? What sort of numbers were the popup messages showing, like 300+? Or was it maybe giving some unusual errors that would cause the import folder to take a step back and wait a bit but not fully pause itself and error out?

>>7985

>>7986

>>7987

Thank you. There is nothing to worry about for these popups. You might have had some unusual workflow with your duplicate files recently, but nothing is seriously wrong.

You have some files that didn't get deleted correctly, so as I write some orphan maintenance code in the next few weeks, I know what to target. The exact numbers are not that important–I have now seen users with orphaned files in both domains.

>>7987

I would really like to give the autocomplete a full pass to make its searching work more asynchronously. Some unusual combinations of letters cause huge lag for pretty stupid reasons, and the current system is not great at dealing with it or keeping ui control in your hands.

If you are entering this a lot, you might like to try copying 'category:2d' to your clipboard and just pasting it and hitting enter real quick, as this will skip the count lookup entirely. You can also customise some of the letter delay timings under I think options->speed and memory.

>>7989

>>7992

Thank you for this report. The bug thread or here is fine for bug reports. You can also email me at hydrus.admin@gmail.com. I mean to keep working on the crashes, so if you can bear with me, I hope I can improve the situation.

If this problem remains completely untenable, I will see about writing a 'downgrade' script that will make your db ready for going back to v287, or whichever release was the latest pre-wx-update one while we figure it out.

When you next run hydrus, please jump over to the 'details' tab of task manager and add the 'threads' column. How many threads is hydrus running–is it more than 200? I believe in some circumstances, an explosion of threads may be being created, which may be part of your CPU usage here.

Also, please try running client.exe with the switch "–no_daemons", so open a cmd window in the install dir (shift+right-click on it to get menu entry for the command window) and enter "client –no_daemons". A bunch of stuff like subscriptions will not run in this mode. Do you still get the high CPU in this case?

How many tabs do you have open in your client?

Do you have many subscriptions or import folders?

If you open a fresh extract of this release to your desktop and run it, does it do high CPU while idling then?

>>7994

>>7995

>>7996

Thank you for these reports. I have some new leads on the manage tags dialog. I am going to keep hammering away at it. Please try abstaining from manage tags for a few days and let me know if you still get any crashes. There may be one still in the media viewer.

>>7997

Thanks m8

>>7998

Well, a bit of an addition to the import folder now,

so, each import was small, the largest being 76 images, subsequent one swere 1 or 2 images each till I paused the import folder. This folder is a small hand selected set of images, that imports 1 time a day, or should.

so, since I set it to import on my command only, its been nearly 3 days, I go ti import manually, as that should work, effectively making an import folder import on my command opposed to a schedule, it refused to import at all.

Now I know global pause import syncrenation will kill all imports no matter what, but setting the folder in the options to pause, should, or at least did allow it to be a manual import only folder, this seems to be broken at the moment and acting as a global pause

for the time being, I unpaused it and will let it do its thing.

and the weirdness I was experiencing was my mouse telling me shit is loading when i'm doing nothing. I linked it to hydrus because it was doing weirdness too with imports.

>>8005

>setting the folder in the options to pause, should, or at least did allow it to be a manual import only folder, this seems to be broken at the moment and acting as a global pause

I was about to report this because it's happening to me too

>>7961

Do you think I'll have any problems jumping from v270 to current?

What's the URL classes stuff about? I updated from v277 to this one so I missed a bunch of changelogs and I can't find anything in the manual.

I'm getting a 404 error with every image in 8chan threads, does this have to do with the new parsing system being phased in?

>>8017

This seems to only be affecting the board wx, anything I can do?

>>7998

>I don't know much about package managers, so I'm afraid I can't say anything clever about Flatpak. But if you know it and know how to use it, please feel to create whatever package you like and distribute it wherever.

I don't know anything about packaging, but I want to try messing around with it. Is there a build command for Hydrus?

>>8001

Thank you, this is useful. Having more responsive and cancellable autocomplete is another thing I want to work on. There's a ton of shit threading in several places, and I have now decided to prioritise it. There will be some improvements in tomorrow's release and then in the coming weeks. Please continue to let me know how you get on.

>>8003

Thanks, this is a great idea. There is a ton of stuff I want to do with the dupe workflow, but most of it is huge jobs–but I can add a couple quick buttons without too much work. I will add this to my immediate todo.

>>8005

>>8006

>>8008

>>8012

Sorry lads, this is a mess because I dashed this stuff out without proper testing. Tomorrow's release will have a bunch of import folder improvements:

A new checkbox: 'check regularly' that controls whether regular checks occur. (pause now means "don't work ever")

Import folders get an optional 'working popup' like subscriptions, so we'll get better feedback as they do their stuff. A big problem here is the feedback on what is happening has been shit until the folder is done working.

Better checking logic that splits the folder check from the ten-minute separated file import.

Popup file buttons with the same name now merge/update.

Import folders can now publish to a page, skipping the popup button middleman entirely. If the page does not exist, it will be created with the import folder's name. If it does exist, it will be updated on future checks.

>>8013

Go to v287, which is the last build before some crashing was introduced. Then wait to see how tomorrow's v294 goes for other users, and if it looks like I have finally fixed the manage tags dialog crash (I am very confident I have, this time ;_;), then you can jump up to that.

>>8016

As it happens, I started the help on this yesterday. Please check it out in tomorrow's release.

>>8017

>>8018

Doesn't /wx/ have some many-years-old threads? I think some of the older threads use an old file storage system that my parser may not be able to deal with. For instance, this early post:

>>>/wx/83

Uses this old format:

https://media.8ch.net/wx/src/1411322547266.webm

Whereas the new system is:

https://media.8ch.net/file_store/c640ac33ec20255ef603d80de7b87551adf81c317fad9a4a4be7558f6aaee853.png

If you check your file import status on that thread watcher, I expect it is giving the file_store URL format but with the old timestamp filename.

The JSON API I use in the new parser doesn't provide this information, and my new parsing system no longer uses my hardcoded hacky way of fixing this 8chan-specific problem.

Please try putting the thread URL into the 'page of images' downloader, as this will fetch URLs directly from the page HTML and should be good. Let me know if it doesn't work.

>>8020

It is all custom duct-taped batch files and bash scripts on my end, but the basic thing you want to do is get pyinstaller and then run something like:

pyinstaller --windowed --icon="C:\code\Hydrus\static\hydrus.ico" --name="client" "C:\code\Hydrus\client.pyw"

And then copy some stuff like the static and help dirs into the resulting 'dist' directory. It is a bit of a hassle, so you might just like to start instead by getting the Extract Only zip release I put out and then make a script that repackages the contents of that into whatever Flatpak wants. It might be a good interim step, at least, so you are learning about Flatpak and whether you enjoy that as the first step rather than learning about my build process.

If you eventually want to do the whole shebang from source, let me know and I'll pastebin my current build.bat or something.

>>8025

Sound good for me on the import folders

Am I still able to set a time, as in I want it to check 1 time a day, then wait till tomorrow? I honestly don't want the program checking as often as it did >>8006, I mean in a 25 minute window it made 9 different archive things, if it did it every 10 minutes that would still be an issue, but something was clearly just checking at all times.

As for making a new window… I like this idea, but is there a way to break this up?

What I mean is I like having the download folder separated by days, this way going through them isn't cumbersome. I could honestly see leaving an import folder alone for a week or so, coming back to it being several thousand images big opposed to 7 or so windows that are 50-200 images big.

>>8027

Yeah, it will return to its old checking pattern in tomorrow's release. Your folder will only scan the folder once a day, and it will try to catch up to its queue in one go. You'll only get one files popup.

If you find your import folder can be tricky to deal with, I recommend doing something like adding an explicit tag for it like "from import folder x" to your local tags and then creating a search page for:

'from import folder x'

system:limit=256

system:inbox

This way you can sample whatever has come through with just one page and deal with it in smaller chunks when you have time without leaving a whole bunch of big pages lying around.

But we'll see, depending on how the new presentation options go. An option to say 'don't merge import folder presentation–always create a new page/popup' would not be difficult. Subscription file buttons will also merge tomorrow, and I want to try having the 'send files to page' for subs as well, so I am interested in how it works out for you and everyone else in the coming weeks to see what else can be done in future and if any of it can be replicated elsewhere.

>>8025

>Please try putting the thread URL into the 'page of images' downloader, as this will fetch URLs directly from the page HTML and should be good. Let me know if it doesn't work.

Indeed putting it in a page download does the trick but I'm not too fond of this method as it messes up the order in which stuff was upoaded and I also can't actively monitor threads.

Is there anything you could do to get the thread watcher working properly with wx?

>>8035

This problem should only be on old uploads from years ago. New uploads should all be good in the new parser, even if they are posted in old threads. Please use the page of images downloader to grab the few early files still hanging around in ancient threads and then continue to use the thread watcher to keep up with anything new.

I expect an html imageboard parser will be coming out in the coming weeks, though, either by me or another user, which should cover everything in this case (although it may need to be adapted to 8chan).