/hydrus/ - Hydrus Network

>>11545

hot!!!

>>11545

Wait… If that's supposed to be a bee giving honey shouldn't it be coming out of her butt?

That's how real bees do it

>>11555

Thank you for this report. I got one of these on Thursday, but it cleared up after a refresh account call. I am not sure if it is related to me rebooting my server on Wed to update it, or if it is related to some recent power cuts I had that maybe knocked my network about for a few 60s-segments that maybe my client just happened to try something on, or something else.

This sounds stupid, but if you try refresh account again, does it work now? I've had this stuff fix itself over time before.

Otherwise, if you check your client log under install_dir/db, is there any traceback related to the login error? You can prob search the log for "public tag repository" to help find anything relevant.

And if not anything good there, what happens if you try a 'refresh accounts' with help->debug->report modes->network report mode on? It should dump some popup messages and eventually write it all to your log again. What requests is it making to my server, and what responses is it logging, at all?

Having trouble closing any page of pages; I don't use this feature often so I don't know if it's new on latest release or what

AttributeError

'PagesNotebook' object has no attribute 'TestAbleToClose'

File "include\ClientGUIMenus.py", line 156, in event_callable

callable( *args, **kwargs )

File "include\ClientGUIPages.py", line 907, in _ClosePage

page.TestAbleToClose()

>>11526

>Do you have, say, more than 100 pages open?

I've never had anything close to even 50 pages open because I'd find myself overwhelmed by it. I do however have a good number of downloads from galleries, I download artists I like entire gallery and then sort through it so that could easily be the cause of the huge number of pending tags.

I have not tried page of pages, but since it's been a week on the new version (still on 338) I haven't run into the bug since then and I think I would have run into it again by now. I believe it's been fixed from moving to wx4.0.

Clearing the deleted mappings record doesn't seem to be doing anything. The deleted tags still show up in the tags manager.

I restarted my client because the usable area turned blank (though I could still see my tabs and they highlight blue when I hover over them; didn't try clicking this time since this happens every so often)

.. and on restart it said "OSError" ; "File not found!"

OSError
File not found!
Traceback (most recent call last):
  File "include\HydrusThreading.py", line 342, in run
    callable( *args, **kwargs )
  File "include\ClientGUIDialogs.py", line 1021, in THREADParseImportablePaths
    mime = HydrusFileHandling.GetMime( path )
  File "include\HydrusFileHandling.py", line 315, in GetMime
    if HydrusVideoHandling.HasVideoStream( path ):
  File "include\HydrusVideoHandling.py", line 309, in HasVideoStream
    lines = GetFFMPEGInfoLines( path )
  File "include\HydrusVideoHandling.py", line 132, in GetFFMPEGInfoLines
    CheckFFMPEGError( lines )
  File "include\HydrusVideoHandling.py", line 37, in CheckFFMPEGError
    raise IOError( "File not found!" )
OSError: File not found!

If it matters, what i was doing immediately before the client area (except tabs) turning blank- which was what prompted me restarting the client- was importing an old windows folder I was too lazy to sift through manually. I never learned how to really tackle this stuff, so I was running irfanview's (image viewer) batch rename command to rename all the images to "date (ascending)" to import under a unique namespace. I let that do its thing into a new folder it created, while at the same time I was importing the original folder with original filenames to Hydrus.

Hydrus said it couldn't access two images, probably because they were in use, but I couldn't be arsed to import the folder again over two images, so I just let it go. I assumed that was what this error was about but I don't know. I'm on version 339 (latest version as of the time of submitting this).

>>11592

Actually the two images Hydrus didn't have access to on importing this folder, that appears to be independent of my running an irfanview batch rename command alongside importing to Hydrus.

The folder irfanview's batch rename thing produced has two fewer images than the original folder. So two of them must just be broken.

In short all my text after pasting the error was probably irrelevant and independent of the error.

Since 339 i am getting an error when uploading data to repositories on the linux client, now with 340 it still happens.

first i got this

C++ assertion "win" failed at /home/vagrant/wxPython-4.0.4/ext/wxWidgets/src/gtk/menu.cpp(84) in DoCommonMenuCallbackCode(): event for a menu without associated window?

File "site-packages/wx/core.py", line 3259, in <lambda>

File "include/ClientGUICommon.py", line 3351, in WXDoIt

self._func( *self._args, **self._kwargs )

File "include/ClientGUI.py", line 4940, in RefreshMenu

old_menu = self._menubar.Remove( old_menu_index )

then the pending menu get gray and vanish and i get this

Menu not found!

Traceback (most recent call last):

File "include/HydrusPubSub.py", line 141, in Process

callable( *args, **kwargs )

File "include/ClientGUI.py", line 4756, in NotifyNewPending

self._DirtyMenu( 'pending' )

File "include/ClientGUI.py", line 1164, in _DirtyMenu

menu_index = self._FindMenuBarIndex( label, name )

File "include/ClientGUI.py", line 1236, in _FindMenuBarIndex

raise HydrusExceptions.DataMissing( 'Menu not found!' )

include.HydrusExceptions.DataMissing: Menu not found!

until the upload is complete

>>11620

>whereas page 2 was then semi-corretly named "page 2.jpg".

*whereas page 2 was then semi-corretly named "page 1.jpg".

>>11620

Use "[page]" not "(page)"

I have a problem on linux where the popup window with all the system searches like in the pic in >>11631 doesn't show up. Same with the popups in the media viewer.

V.338

>>11589

>>11555

>>11629

Thank you. I am sorry for the inconvenience here. When I wrote the new network engine, I had to fold in the old hydrus account checking stuff into the new login system, and it was a little messy. I suspect there are some dead-end error states in there somewhere that I am not catching properly. In your case, perhaps after several days some timer clicked over and it realised that the login status really was 'unknown' instead of 'failed' or something, so it was able to try one more time in the background, failed cleanly, and then set the service paused in a 'true' unknown state. You were then able to honestly retry and it all sorted itself out.

Please let me know if this happens again or you find out anything new.

>>11581

Hey, I am sorry for this bug. It was a mistake I made in 339. I have fixed it for the most recent 340 and written a new UI test to stop this stupid stuff happening again.

>>11582

>>11607

Ok, thank you. I will keep working on this. I rejiggered some menubar stuff this past week for 340, so maybe if >>11607 was on 339, it will now be improved, but I am not sure. I don't think this is menubar but instead something else overflowing. If you are also getting crashes, I recommend you scale back your client in any other easy ways. If you have a lot of thumbnails sitting there waiting to be processed, give them a local tag for like 'process later' and close the tab. Make your client leaner is probably the best bet here for now.

>>11585

Was this through the advanced content update dialog, where you say like '(copy) (all mappings) from (x) to (y)'? That had a counting bug that meant it would not perform enough update rows (often it would do 0) in many cases. This is fixed in 340, so if this is so, please try again in that and let me know if it works.

>>11592

>>11597

Thank you for this report. A couple of users have got boshed by this. Please check the chain at >>11594 . I am interested in answers to all the questions in that.

>>11615

Thank you. I will revisit this code this week. I am not sure what is happening here, but I think it is OS- or window manager-specific–is this on Linux, as running from source?

>>11620

>>11624

Hey, I think >>11633 has it. Try with brackets, [ and ], and not parentheses, ( and ). With parentheses, it is looking for 'page' rather than the x in 'page:x', and since the files do not have 'page', the resultant filename is the empty string. Each file gets the same empty string, so my 'duplicate-filename de-deduper' is putting ' (n+1)' on the end of each one.

>>11631

Thank you for this report. I do not recommend you try to load that many files at once! I recommend you set a limit under options->speed and memory. 10k is the default.

That said, this bad number may be due to some file miscounting in the past. Please try running help->debug->data actions->clear db service info cache and then restart the client (you might like to note down your exact numbers here beforehand). Afterwards, have the inbox/archive/everything numbers changed? What about the 'all local files' and 'my files' file domains under services->review services panel? 'all local files' should have the same number of files as the sum of 'my files', 'repository updates', and 'trash' domains.

>>11634

I have had a lot of trouble getting those floating windows to work right in non-Windows, so by default I have them 'embed', like in the manage tags dialog. Please check the 'always embed autocomplete …' option under options->gui and restart the client–does that work better?

There is an ugly BUGFIX option on the same page to make the hover windows on the media viewer always show. It isn't nice, but it works better on the Linux window managers that do not like my code.

Did you by any chance migrate from Windows?

>>11652

not running from source, i am running the the executable. To be more specific, The problems happens if i rover the mouse over the pending menu after i put one to sync. I am runing ubuntu 18.04 with KDE plasma, if i go over GNOME the problem is worst, most of the menus get greyed out

pressing "restore from a database backup" on client ver 340, newly install debian 9 causes the client to crash, output in terminal is "Aborted."

>>11667

I've got 341 to do some more intelligent swapping during fast update events here. It may fix your issue, or it might move it somewhere else. Please keep me updated once you have tried 341, and I'll give it another go if needed.

>>11679

Thank you for this report. Is this when exporting via the export files dialog (i.e. from the thumbnail right-click menu), or drag and drop exporting? If you haven't tried one way, can you try it and let me know if that works?

Does it freeze at the beginning of the action, like as soon as you click, or at the 'end', when the copy is completed and control passes back to the program? Do the files ultimately get exported or not?

>>11686

Thank you for this report. I will revisit this code. It was always a jank way of doing it and something I did not maintain, so the solution here may be to remove it and write a better guide on how to do it manually.

>>11706

ok m8, you are doing gods work, i will report wfter i try 341

Since I updated from 338 to 340 I've been having a problem with file discovery crashing my client. I don't get an exception message, no logs, just a message in my terminal:

[1] 19988 segmentation fault (core dumped) hydrus-client -d ~/mnt/data/hydrus/hydrus/

So if I click the discover duplicates button, my client crashes. Anything else on the page works fine (except the button to launch the filter, I don't have any potential duplicates and trying to find them crashes my client, so maybe it does, maybe it doesn't.) I figure I somehow fucked up my duplicate processing, unfortunately for me I overwrote my previous backup with a 340 one because I'm retarded.

Even more unfortunately for me, on a fresh db this doesn't happen which probably means I fucked up.

Also I'm running 341 on Linux.

Any suggestions for unfucking duplicate discovery?

Version 340, downloading from SVTFOEbooru

Source: http://svtfoe.booru.org/index.php?id=23786&page=post&s=view

<html>… (Copy note to see full error)

Traceback (most recent call last):

File "include\ClientImportFileSeeds.py", line 1071, in WorkOnURL

network_job.WaitUntilDone()

File "include\ClientNetworkingJobs.py", line 1133, in WaitUntilDone

raise self._error_exception

include.HydrusExceptions.ServerException: <html>

<head><title>503 Service Temporarily Unavailable</title></head>

<body bgcolor="white">

<center><h1>503 Service Temporarily Unavailable</h1></center>

<hr><center>nginx</center>

</body>

</html>

I was trying to make collections out of my newly downloaded files, it was fucking up and turns out it had downloaded an mp3 and that's what was fucking up.

Looks like the program is set to accept mp3 files but not programmed to collect mp3 files properly or something.

Here's the file: yiff.party/patreon_data/20184639/Whatevs_01.mp3

TypeError
'>' not supported between instances of 'NoneType' and 'int'
Traceback (most recent call last):
  File "include\HydrusPubSub.py", line 141, in Process
    callable( *args, **kwargs )
  File "include\ClientGUICanvas.py", line 2585, in MediaFocusWentToExternalProgram
    self._MediaFocusWentToExternalProgram()
  File "include\ClientGUICanvas.py", line 1674, in _MediaFocusWentToExternalProgram
    elif self._current_media.HasDuration():
  File "include\ClientMedia.py", line 1702, in HasDuration
    def HasDuration( self ): return self._media_result.GetDuration() is not None and self._media_result.GetNumFrames() > 1
TypeError: '>' not supported between instances of 'NoneType' and 'int'

>>11736

Are you using binary distribution from tarball or built version from github? Pre-built hydrus is often broken on Linux because of the plethora of libraries shipped with it that might be incompatible with your system libraries.

>>11758

I use a Arch based distro so I get Hydrus from the AUR, the pkgbuild appears to compile hydrus from source. Regardless I've been running it out of pycharm for debugging purposes, so unless I have a library that's so new it's breaking stuff, I don't think that's the problem.

Now as I just mentioned I've been running hydrus out of pycharm so I could debug it. Tl;dr I debugged the shit out of hydrus, eventually we get to the Write method in /hydrus/include/HydrusDB.py:

I figure either

line 967: job = HydrusData.JobDatabase( job_type, synchronous, action, *args, **kwargs )

is producing an incorrect job which in turn causes this:

line 974: self._jobs.put( job )

to flip out.

Or if we go with what gdb says (even though I can't even find a single piece of sqlite related code in the entire set of methods I followed), then sqlite is flipping out which probably means I need to manually edit a table in my database which I am quite capable of…if I knew which table to look at.

So if anyone has any knowledge on where exactly the aforementioned write method is calling to sqlite, I'd love the info.

Found an image that was incorrectly detected as a decompression bomb by the importer, here is the original link

https://yiff.party/patreon_data/24364306/3223347/01-30.png

and here is a mirror

https://anonfile.com/71ufV4v0b8/01-30_png

Fair warning: pic is 13 MB of furry porn.

>>11788

Well I figured it out. sqlite 3.27.1 can't execute the following statement:

SELECT phash_id, phash, radius, inner_id, outer_id FROM shape_perceptual_hashes NATURAL JOIN shape_vptree WHERE phash_id IN (123778, 113226, 135502, 94192);

Every other version can. Guess which version of sqlite my distro upgraded to when I performed the system update that upgraded hydrus to 340?

>>11706

ok, i tried to replicate the error.

now the menu stay greyed out and the error that appear is this

2019/03/06 00:45:02: wxAssertionError

C++ assertion "win" failed at /home/vagrant/wxPython-4.0.4/ext/wxWidgets/src/gtk/menu.cpp(84) in DoCommonMenuCallbackCode(): event for a menu without associated window?

File "site-packages/wx/core.py", line 3259, in <lambda>

File "include/ClientGUICommon.py", line 3351, in WXDoIt

self._func( *self._args, **self._kwargs )

File "include/ClientGUI.py", line 4959, in RefreshMenu

self._menubar.Replace( old_menu_index, menu, label )

sometimes the menu comes backup, sometimes not. but i can state that it is more stable, specially on the gallery download page and on my gallery(+- 18k images)

I am sorry lads, I had a mixed couple of weeks so I am late getting to all this. I have a new tablet for hydrus that I hope will help me keep on top of this better when I am away from my dev machine, even if I put out shittier shorter posts at times.

>>11714

Thank you for this follow-up. There is some really shit ui code in the client that can lead to hangs during big operations. I have a long-term plan to fix it all, but for now I'll see if I can make large export files jobs less susceptible to the problem.

The 'solution' for now is just to wait it out. It will get there in the end.

>>11736

>>11758

This should be finally fixed in the current release. It was an odd low-level issue related to the order in which I was doing some CPU stuff and database iteration for distance>0 searches, and specific to some flavours of Linux. I am sorry for the inconvenience. Please let me know if you still have it.

>>11741

Hey, 503 is a server-side error. I presume booru.org was overloaded at that time. It works for me now. Please try the download again and let me know if you still have problems.

>>11747

Thank you for this report. I have been doing some work here recently, so it might be fixed already, but I will check it out again.

>>11752

Thank you for this report. This is odd. It happened a bit a while ago, but I thought I had it cleared up. If you are on the search page and click on the thumbnail area, does it dissappear as expected then, or does it always stay up no matter where the focus is? Which version of Windows are you on?

This is not an ideal solution, but you can make the dropdown always 'embed' into the panel under options->gui->always embed autocomplete….

>>11788

>>11797

Continuing from >>11899 . Yeah, this way the problem query, which should now be fixed. If you are interested in the details, the problem seems to have been the subsequent HydrusData.Get64BitHammingDistance call being interleaved with the row fetching. This Hamming function does some mickey-mouse CPU stuff to count 1s from an XOR, and when I did it during the SQLite query iterating its results, it crashed in some flavours of Linux. When I instead complete the query and then do the phash distance searching, all in smaller batches, it is ok.

I am not expert enough in this stuff to know what the problem was, but I presume SQLite was still hanging on to a buffer pointer or something of the phash bytes, to be used in the next row fetch iteration, and when python then did either the struct or n & 0xF0F0F0F0F0F0F0F0 bullshit on it and SQLite returned to it, it caused a memory access violation.

Again, seems to be fixed now. I am sorry for the mess–this was a pain to figure out because I could not replicate it on my test machine. Let me know if you still have trouble.

>>11795

Thank you for this report. Yeah, this seems on the lower end of bombs. I don't really see any memory explosion as it loads, even if it is a big file in itself.

I recommend you turn on 'allow decomp bombs' under file import options and reattempt the import.

The decompbomb test isn't actually mine–it is something internal to one of my image loading libraries. I think I can bump up the limit–especially since python 3 is better at handling memory all around. We'll see how that works out.

>>11800

Thank you. I will keep working on this.

>>11899

>>11903

I'm also having the same issue with it crashing on >0 duplicate searching on Arch Linux. I'm using the latest v343. Was it supposed to already be fixed or do you have it queued up for the v344 release?

>>11925

F U G G

U

G

G

Yeah, this should be fixed in 343, which means there is still more to go here.

Ok, please keep the search at 0 distance for now. I will see if I can do something about this this week. Maybe make a super-safe debug mode or similar for you to test out.

>>11934

For some reason it's working now that I restarted my computer. It's weird because it's not like I was still running an older version. I turned on my computer, updated all my packages, then opened hydrus and encountered the error. I even made sure to double check the version I was running before I posted. Today I restarted both of my computers for a kernel update and it's working fine on both of them.

Latest version on Linux is constantly crashing. And doesn't show the tags on the search menu. I'm on Manjaro.

And one question: when I asked to export some files to a folder, only a few are actually are. Like there is 138 files under "X" tag and only half of them are exported to the folder. Am I missing something?

>>11941

Thanks. Please keep me updated–there is just the smallest off-chance this was a crash due to a different problem that either happened to occur at the time you clicked, or maybe the click fired off some garbage collection that triggered it otherwise.

Or perhaps the kernel update actually fixed some deeper memory-management issue! Anyway, let me know how you get on from now on.

>>11943

>>11944

Thank you for this report. Are you running from source or the built release I put out? If you ran 342, the previous week's release, was that ok for you?

I see you have the hidden button borders as well. Is that new as well, or have you had this before?

Unfortunately, the frozen build I release is made on Ubuntu 16.04 and is not very compatible with certain flavours of Linux and window manager. If you are running from that, you may like to look into running from source:

https://hydrusnetwork.github.io/hydrus/help/running_from_source.html

A bunch of UI weirdness is generally fixed, and stability improves as well.

As for file exporting, I am not sure. Is this an 'export folder'? Is there any chance the filename phrase you set may result in duplicates? Is it possible it is writing two different files to the same 'blue eyes.jpg' or whatever filename?

>>11957

>No, this is the first time I'm using hydrus on Linux.

On options check the "Always autoembed dropdown results".

>>11957

>>11960

>>11961

Yeah, that's more of the hidden ui elements. The checkbox is there, afaik, but just not rendering right. The embed autocomplete is now forced always-on for non-windows, you don't have to worry about it. My guess it is squashed because your WM isn't reading/storing my SetInitialSize etc… calls right.

Thanks for that info about Firefox sometimes doing similar with themes. I do not really set any colours for my controls, and I am not sure I even can set colours for different parts of checkboxes–I think wxPython just uses the system defaults. This borderless stuff seems to be related to my Ubuntu .so files clashing with whatever your system wants. I believe the best path is to try running from source (or maybe try switching theme/WM for hydrus, if that is possible?).

Let me know if you figure out a particular behaviour seems to cause a crash, or if otherwise avoiding something seems to eliminate it completely.

>>11907

>I recommend you turn on 'allow decomp bombs' under file import options and reattempt the import.

That was how I worked around the issue, yes, I reported mainly because it was pretty weird behaviour.

>>11974

>>11976

>>11977

>>11978

Thank you, these are great. I will try to check these out this week.

>>11651

Dude, I'm sorry. I am dummy depressed all the time and I feel stupid even making this post.

I didn't reply to you for now over a month cause I was kinda panicking over your reply. I don't use Linux, and I didn't get all the stuff you mentioned, and I wasn't doing so well anyway. I figured maybe later it'd be easier.

But I still can't do it. It's hard. That doesn't justify anything but I don't know.

I'm on version 341 now, and I was about to install version 344, which is the latest right now. I am on Windows 7 Ultimate.

I don't really fetishize ignorance so I don't really need you to say "it's ok to not provide context". I'm not really saying anything for any purpose

I'm having an issue with the API. When I GET /get_files/file_metadata with the parameters file_id and only_return_identifiers, I get this error:

Traceback (most recent call last):
  File "threading.py", line 916, in _bootstrap_inner
    
  File "threading.py", line 864, in run
    
  File "site-packages\twisted\_threads\_threadworker.py", line 46, in work
    
  File "site-packages\twisted\_threads\_team.py", line 190, in doWork
    
--- <exception caught here> ---
  File "site-packages\twisted\python\threadpool.py", line 250, in inContext
    
  File "site-packages\twisted\python\threadpool.py", line 266, in <lambda>
    
  File "site-packages\twisted\python\context.py", line 122, in callWithContext
    
  File "site-packages\twisted\python\context.py", line 85, in callWithContext
    
  File "include\ClientLocalServerResources.py", line 1255, in _threadDoGETJob
    file_ids_to_hashes = HG.client_controller.Read( 'hash_ids_to_hashes', file_ids = file_ids )
  File "include\HydrusController.py", line 588, in Read
    return self._Read( action, *args, **kwargs )
  File "include\HydrusController.py", line 183, in _Read
    result = self.db.Read( action, *args, **kwargs )
  File "include\HydrusDB.py", line 931, in Read
    return job.GetResult()
  File "include\HydrusData.py", line 1577, in GetResult
    raise e
include.HydrusExceptions.DBException: TypeError: _GetHashIdsToHashes() got an unexpected keyword argument 'file_ids'

Database Traceback (most recent call last):
  File "include\HydrusDB.py", line 557, in _ProcessJob
    result = self._Read( action, *args, **kwargs )
  File "include\ClientDB.py", line 9265, in _Read
    elif action == 'hash_ids_to_hashes': result = self._GetHashIdsToHashes( *args, **kwargs )
TypeError: _GetHashIdsToHashes() got an unexpected keyword argument 'file_ids'

The request returns OK if I get rid of the only_return_identifiers parameter or if I use hashes instead.

>>11983

No worries m8. I think I read your error report wrong in the first place and assumed you were having 'finding ffmpeg executable' problems, whereas it looks actually like your ffmpeg is ok but could not find the file hydrus was pointing it to. If you had a batch rename thing going on at the time, this may well have been related–maybe hydrus queued the file up to be imported, and then irfanview renamed it before hydrus got around to trying the import. Or maybe hydrus had a hiccup when trying to do some temporary storage stuff and lost track of the file mid-import for other reasons.

If you are still seeing this error over and over, then I am happy to chase it up. But if this was a one-time thing that may have been because of two programs accessing the same space at the same time, no need to worry about it.

You are good to update to 344. Let me know if you run into any more trouble.

Sounds like you are having a shit time. I understand this completely. If you can get a few bucks together, please consider getting this book: https://www.amazon.com/Feeling-Good-Handbook-David-Burns/dp/0452281326/ . It saved my life about ten years ago. You can find it in any big corporate bookstore as well, just say you are buying it for a friend.

>>11984

Thank you for this report. I fucked up the variable name here, and this is now fixed for 345. I have a background job already to improve my testing to catch this shit, which has been a blind spot previously.

I'm not sure if this is a bug with Hydrus or what is going on. I downloaded the standalone Flash player so I could view swf through Hydrus' open externally menu. However, when I do that the program opens like it's hidden. I can only hear sound and find the process in the task manager, but I can't find a window. If I run the launch path (X:\Hydrus\flashplayer_32_sa.exe "%path%") manually it works fine.

>>12087

Fuck, that's crazy. Thank you for this report. If you can repeat this now, can you try clicking on just one of those blank spaces? What info does it say in the status bar then, like 0 selected out of 149? What happens if you right-click on that ghost thumb, does it give any info or an error? What happens if you Ctrl+A and then try to drag the selection up to the greyspace tab area up top (i.e. moving it to a brand new page)? What's the 'file import options' 's 'presentation' options for that downloader? It looks like it is trying to present all 149 successful files, but some subset were maybe deleted or otherwise file-domain-invalid and so can't be sourced/counted somehow.

Hitting F5 on a page without tag search (typically any import page) will force a resort according to what the page has set (rather than the default there, which is downloader-queue order), so looks like 'oldest first time imported' in that screen.

>>12096

Thank you for this report. That's odd–I assume if you double-click a swf in explorer, the standalone projector launches ok?

The default windows 'open externally' call is very simple on my end–I literally just say "Hey Windows, open this path m8: ( path )", so I presume what you are seeing is an artifact of an odd Windows-side 'open with' setup.

Perhaps your environment is unusual in some way, so if you would like to pursue this further, please try help->debug->report modes->subprocess report mode and then launch with your "%path%" command. It will popup a bit about the launch environment. If there is anything interesting here, please post it (and feel free to scrub out any personal info), and it might help here. There's an outside chance that maybe the hydrus exe is running as one user but the flash projector is owned otherwise, or there are some funny permissions going on here.

EDIT: I didn't read your post right. Did you try setting up that %path% in hydrus's options->files and trash, or is that something you were doing from the command line? If you haven't tried it for hydrus, that may 'open externally' fine.

>>12101

Thank you for this report. I am afraid I am a little limited by my ui library here. I use wxPython for its ease of use (and I use it in a basic way), but that also offers a little less customisation as a result–it generally pulls from your system defaults for things like how to draw buttons and font sizes and so on, and I do not generally set those sizes myself.

That said, I can improve the layout sizers in the left-side management panel for smaller screens, and I will see if I can make the system preds either fit better or scroll to view when you interact with it.

If you want the left panel a bit wider, the program should remember the current page's current width when you close the program, so you might like to make it wider and go down to five thumbnails wide on your main thumbgrid. Also, if you want the extra verticality and are willing to sacrifice it, you can hide the bottom-left preview window under options->gui.

>>12110

I had it set up in options->files and trash, yeah. But I removed it from there and set it up as an "open with" association in windows and now it works fine. There appears to be an issue with setting a manual launch path. The path was "X:\Hydrus\flashplayer_32_sa.exe "%path%"", if that helps. The program opens and plays the swf file, but the window is hidden, and has to be closed in task manager.

Had disk io errors, ran a chdsk and it fixed some things, now I have

>sqlite3.OperationalError: table local_hashes already exists

im completely fine with reimporting all my images, but is there a way to do that without losing my processed tags?

>>11908

hi, it's me again. got on 345 and for a long time i couldnt replicate the error. and got harder on 346, but i got to(dont know how tho). The erro message was this:

2019/04/08 13:03:48: Uncaught exception:

2019/04/08 13:03:48: wxAssertionError

C++ assertion "win" failed at /home/vagrant/wxPython-4.0.4/ext/wxWidgets/src/gtk/menu.cpp(84) in DoCommonMenuCallbackCode(): event for a menu without associated window?

File "site-packages/wx/core.py", line 3259, in <lambda>

File "include/ClientGUICommon.py", line 3351, in WXDoIt

self._func( *self._args, **self._kwargs )

File "include/ClientGUI.py", line 4973, in RefreshMenu

self._menubar.Replace( old_menu_index, menu, label )

>>12117

Thank you, I will have a closer look at this next week.

>>12118

Thank you, this is excellent. I am not totally sure what is going on here, but there are issues in 346 with drawing certain thumbnails and making files appear even when they are not in the appropriate file domain.

Neither of these issues is precisely what you are seeing here, although that 'thumb ghosts on click' makes me think the animation is a bit busted, perhaps as a result of it not being able to locate files due to dupes or whatever.

Whatever the case, 347 should be better in several ways here. Please give it a try when convenient and let me know if the situation is any better or exactly the same.

That bad 'known url' stuff is annoying. Some boorus prepend extra metadata in the known url area (in this case "リタを触手で.avi | ranken [pixiv] "), and it is parsing wrong. I'll see if I can auto-detect this stuff better and cut off everything up to any http.

>>12120

I cannot say with confidence without more information, but could one of your .db files been deleted or moved as you fixed things? There should be:

client.db

client.caches.db

client.mappings.db

client.master.db

In install_dir/db.

Are your three auxiliary dbs fairly large and old, but the client.db say only 200KB and brand new? Could the original client.db have been moved somewhere, and then the client was booted with it missing? That error looks like the client booted without a client.db, made a fresh empty one, and then ran into a roadblock when it discovered client.master.db already existed.

If this is exactly what happened (it may help to compare the creation date for your four files to see if one was only made recently), your solution will be to move the stub db file (likely client.db) to a new safe location and then move the original client.db back to the database folder so all four files are together again.

If you are absolutely certain you did not move your client.db, I believe it was deleted in the drive damage or the chkdsk. Please double-check your recycle bin, just to make sure. if you cannot recover a good client.db, I am afraid your client is lost and you will have to start again. The short steps for this are:

Move current db folder contents and client_files folder somewhere safe.

Boot a fresh client.

Import contents of client_files's 256 'fxx' folders in batches.

I am afraid I cannot help you recover subscriptions or other core metadata, as this is all stored on client.db. But with some work we should be able to migrate your tags to a new db.

If your client.db is large and old, please let me know, as a more complicated error has occurred here.

Do not delete anything, and please check if you have any backups anywhere, even very old ones.

>>12123

Thank you. I will keep pushing on this. I am glad it is happening less. I still have some ideas for code cleanup and reduced replace calls, particularly during pending tags commit.

>>12141

yeah, client.db is only 4KB, though the creation time is similar to all the others

unfortunately no way to get the old client.db back, i didn't take the advice to backup :(

images are back, but no tags yet

The yiff.party parser is technically broken, but not really. Nothing about the creator json on the site changed, but any and all attachments downloaded through hydrus result in either a 403 or a 404 for me, except the links are correct, viewing them in a browser works.

My uneducated guess is yiff.party is blocking whatever Hydrus uses as its user agent or something similarly simple. Not that surprising since that site is in desperate need of funding and has limited bandwidth.

>>12147

im good now, just got tags from

https://cuddlebear92.github.io/Quicksync/

>>12147

>>12231

I am sorry you had a hard time with this, but I am glad you are putting things back together.

I know it sucks to hear right now, but this sort of problem can be a great driver to get a good backup routine going. It happened to me about ten years ago, and has happened to more than a few hydrus users since I started doing this–you lose a load of shit and previous effort and decide to spend a bit of cash to avoid that pain again. I highly recommend a cheap external usb drive, either from a store or second-hand from a friend. Even just having a copy of the .db files–rather than including all your client_files as well–can make a real difference.

If you are interested and not sure how to do it:

https://hydrusnetwork.github.io/hydrus/help/getting_started_installing.html#backing_up

>>12149

Thank you for this report. Have you had any crashes or power outages recently? The file identifier error may have occured due to a recent-ish hard shutdown of the client. As a precaution, I recommend you shut down the client and check the 'help my db is broke.txt' document under install_dir/db/ as background reading. You might like to just check the integrity of your db and maybe run a chkdsk to be sure your drive is ok.

Can you also check install_dir/db/client - 2019-4.log? If you scroll way down to the timestamp where this was happening (ctrl+f -> 'file identifier' will probably get you to it too), are there any error tracebacks related to the pause/unpause or missing/broken file imports? I would be interested in seeing that information, as it may better explain why subs were acting wrong here.

Subs may take a few seconds to register a 'pause' event, and will typically not stop working until the current gallery page or file being worked on is done, but they shouldn't flip behaviour entirely.

If this was just a blip due to a power cut or whatever, and you are confident your drive is healthy and database is not corrupt, you I think have two paths: if you keep using your client and it seems totally fine, especially if those subscription files are reimported and are fine, I think it may have fixed itself, but if you keep getting it, particularly in relation to the same set of files that are tagged on the PTR, your solution is to go services->review services->remote->tags->PTR and hit reset processing cache.

Let me know how you get on!

>>12189

Yeah, it looks like they started putting secondary post attachments to a new subfolder in the URL structure, and it was being recognised and incorrectly truncated hydrus-side. I am rolling out a new url class for 348 that seems to fix it. Please let me know if you still have trouble here.

It wasn't a block, but the accidental hydrus truncation was doing something like this:

https://yiff.party/patreon_data/9078764/24083586/3164951/coffee-girl--SFW.jpg

->

https://yiff.party/patreon_data/9078764/24083586/3164951

I've noticed that the url import isn't attaching tags to gelbooru and pixiv links anymore. Is there a setting I'm missing? I am on version 347 now, I was on 343 before.

>>12271

It seems ok to me in a drag-and-drop test on a gelbooru URL I just did now, and I do not think I have changed anything in the past four weeks–can you say any more about your situation? How are you importing this URL: through a search or a drag and drop? Have you imported any new parsers for those domains? And what are your 'default tag import options' set up as under network->downloaders->manage default tag import options? Do you have one TIO for all domains to rely on, or specific ones for gelb and pixiv? Do they all look correct?

when I try to manually process tags, the entire client locks up

when I wait for it to do it automatically, it immediately finishes

it's about 200 tags behind

>>12352

i can only process tags using shutdown processing

>>12353

>>12352

Thank you for this report. Are you manually triggering repo processing through the 'process now' button under review services? This button has always been a bit debug tier and I have heard it increasingly working badly for some users with the current PTR.

If you have it run in idle time, does it run ok? When you say it 'immediately finishes', does that mean it does no work at all? Is it 200 updates behind on the review services page, and it doesn't seem to be catching up, even in idle time?

Was doing some quick and dirty processing and when I attempted to set 2 images as alternates I got a DBException "IndexError: Cannot choose from an empty sequence"

DBException

IndexError: Cannot choose from an empty sequence

Full Traceback:

File "include\ClientGUICommon.py", line 708, in EventButton

self._func( *self._args, **self._kwargs )

File "include\ClientGUIManagement.py", line 1370, in _SetCurrentMediaAs

self._ShowSomeDupes()

File "include\ClientGUIManagement.py", line 1385, in _ShowSomeDupes

hashes = self._controller.Read( 'random_unknown_duplicate_hashes', file_search_context, both_files_match )

File "include\HydrusController.py", line 593, in Read

return self._Read( action, *args, **kwargs )

File "include\HydrusController.py", line 183, in _Read

result = self.db.Read( action, *args, **kwargs )

File "include\HydrusDB.py", line 931, in Read

return job.GetResult()

File "include\HydrusData.py", line 1582, in GetResult

raise e

Database Traceback (most recent call last):

File "include\HydrusDB.py", line 557, in _ProcessJob

result = self._Read( action, *args, **kwargs )

File "include\ClientDB.py", line 9520, in _Read

elif action == 'random_unknown_duplicate_hashes': result = self._CacheSimilarFilesGetRandomUnknownDuplicateHashes( *args, **kwargs )

File "include\ClientDB.py", line 1285, in _CacheSimilarFilesGetRandomUnknownDuplicateHashes

hash_id = random.choice( list( potential_hash_ids ) )

File "random.py", line 260, in choice

IndexError: Cannot choose from an empty sequence

Just found out that the media was actually set as alternates, it just would not remove the media from the duplicate page.

Closing the tab entirely and opening a new duplicate tab showed no potential pairs once again.

>>12417

>>12418

Thank you for this report. This is a very helpful error, and I think related to the 'ghost pair' issue I'll be tackling this week.

>>12422

Thank you for this report. I think I am forgetting how some zoom stuff works, or just misunderstanding. Can you talk about it a bit more? When you say the 'zoom toggle', do you mean the zoom button/action that switches between 100% and the full media viewer size, or something else? Can you say what you would like to happen when it is hit and the image is the same height as the monitor?

>>12412

>Are you manually triggering repo processing through the 'process now' button under review services

yes

>Is it 200 updates behind on the review services page, and it doesn't seem to be catching up, even in idle time?

i mean 200 processed tags behind the number downloaded

when idle time starts, the popup in the bottom right says it's starting to process tags, then immediately says it's finished

Another thing:

ctrl+shift+T / "unclose_page"

When several pages have been closed, I expect it to re-open the most recently closed page, which is at the TOP of the "closed pages" list.

Actual result: It re-opens the page from the BOTTOM of the "closed pages" list

It feels very counter-intuitive to me and is not how browsers do it, so I assume it's a bug.

Ctrl + Backspace in Windows inserts a control character instead of deleting the previous word in a text box.

Could that be fixed?

When entering incorrect login credentials for Pixiv, Hydrus falsely reports that the login is valid and the user is logged in even though they actually haven't logged in.

While in the Hydrus Client Media Viewer, if you open the manage tags window, the media viewer's hover-over popups (tags, known urls, and the top toolbar) still pop up and obstruct the manage tags window.

This occurs even when the manage tags window is the current active window.

I am sorry, I let this thread get away from me last week.

>>12431

What are your idle settings under options->maintenance and processing? If you turn the system CPU setting off, does that stop this behaviour?

>>12432

Thank you, I will check this this week.

>>12442

Thank you, I will check this as well.

>>12494

Thank you. I have been doing some thumbnail drawing work recently, and I am not done. It may be better now. I sometimes get something like this on my IRL client, but on only 4-6 thumbs, and it fixes itself with a ctrl+a. I will investigate this more.

>>12501

Thank you for this report. I had no idea this happened. It looks like some Rich Text something, as here: https://trac.wxwidgets.org/ticket/10344

I'll see if changing the style removes this.

>>12512

Thank you for this report. I think I wrote the Pixiv login script. If I remember right, Pixiv still gives you a cookie whether you are a guest or correctly logged in. I can't promise anything too fast, but I will make a job to see if I can add a login step to pixiv to check for login/logout buttons or whatever on the homepage once the login attempt happens.

>>12514

Thank you for this report. I'll check this code. The animated part might have something to do with it.

>>12516

Thank you, I am sorry. I had this from a couple of other people as well. In letting the hovers display even when the new duplicate filter always-on-top has focus, I accidentally added these dialogs to the permissable 'you can show' logic. I will fix it for 352.

When I manually stopped a dupe search at distance 8

RuntimeError
wrapped C/C++ object of type BetterStaticText has been deleted
  File "site-packages\wx\core.py", line 3259, in <lambda>
  File "include\ClientGUIManagement.py", line 1272, in wx_code
    self._refresh_maintenance_status.SetLabelText( '' )
  File "include\ClientGUICommon.py", line 944, in SetLabelText
    wx.StaticText.SetLabelText( self, text )

>>12535

Wow, thank you for this report. I am 97% confident I can knock that on the head this week.

>>12537

I have just had a look at the idle calculation code and nothing stands out. I am not sure what is going on here.

This is a long-shot, but what happens if you hit help->debug->debug modes->force idle mode? Do it at least 30s after booting the client. It should wake the process daemon and do some work, and the 'should I stop early' calculation should be functionally the same as shutdown processing.

Otherwise, if you have not written down the number of updates currently downloaded and processed, can you do that, and then check the numbers again in three days? I'd like to see if the idle processing is doing any work at all when you aren't looking, or if it is always stopping early.

>>12538

Thank you for this report. This was a compromise I added to support touch-drag for touchscreens, which sperg out at the hidden anchor mouse warping. Can you just confirm that under options->media, you have the 'WINDOWS ONLY: anchor blah' option checked atm?

I will add an option here to override the touchscreen hack so it always anchors (probably, I'll change it so that is the new default, and you have to turn on touchscreen support, so hopefully you'll just be fixed here soon). I am 68% confident I can get this done this week for 352, and 95% confident I can get it done for 354.

>>12142

Me again. got to 351, error occuring less and overall better stability. the erro i was getting changed, but doing the same action, and with same outcome, greyed out pages and pending menus. here the error:

2019/05/13 14:51:40: Uncaught exception:

2019/05/13 14:51:40: wxAssertionError

C++ assertion "win" failed at /home/vagrant/wxPython-4.0.4/ext/wxWidgets/src/gtk/menu.cpp(84) in DoCommonMenuCallbackCode(): event for a menu without associated window?

File "site-packages/wx/core.py", line 3259, in <lambda>

File "include/ClientGUICommon.py", line 3365, in WXDoIt

self._func( *self._args, **self._kwargs )

File "include/ClientGUI.py", line 5003, in RefreshMenu

self._menubar.Replace( old_menu_index, menu, label )

>>12558

>>12549

>I'd like to see if the idle processing is doing any work at all when you aren't looking, or if it is always stopping early

my computer isn't on most of the day, so this isn't really applicable sorry

>>12494

What GTK theme and what distro?>>12494

>>12556

Thank you for the update. I will not have time today to look at this, but I am 95% certain I can put some time into it by 354.

>>12558

>>12559

When you say crashing, does it actually quit the program unexpectedly, or does it only hang, 'Not Responding', indefinitely? Depending on the client, big maintenance jobs like this can sometimes block the UI thread until they are done (due to my old shit code not handling big asynchronous tasks properly). If your window is only hanging, greyed out, can you please check if the client.exe is doing CPU/HDD work under Task Manager? If so, it is probably working ok, just not updating the UI. It will eventually catch up and finish cleanly, even if it takes some time.

If it is actually crashing and quitting the program, then something worse is happening. It might be related to running out of memory or another unexpected OS/hardware bottleneck. Processing is typically very stable, especially for Windows users atm.

Nonetheless, if your client is having trouble doing processing work in idle time under any context, I think your most practical solution right now is to limit it to only do this work in shutdown time. I have plans to improve big maintenance task ui politeness, but it will not come quickly.

>>12572

yeah, seems like it was just blocking ui, it is still accessing disk

annoying, but whatever

When shortcuts are used to open the manage tags window (f3 or a custom one), if you press spacebar while typing the tags, the window closes.

>>12578

Thank you for this report. I am afraid I cannot reproduce it unless I bind 'spacebar' to 'manage_file_tags' shortcut action under the 'media' shortcut set. Do you have something like this? Do you have spacebar set to any other shortcuts?

And if you open manage tags from the right-click menu (i.e. not using a shortcut), do you not get this behaviour? What happens if you press spacebar on the similar manage ratings or manage known urls dialogs?

>>12580

Somehow I had spacebar binded to manage the tags. Removed and now it works fine. Sorry.

>>12581

Great, no worries. Thanks for letting me know.

Seems that whenever I have my file page collected(by series, chapter, title, etc.) and I refresh said page, it chooses a random image to show rather than showing the first image from my 'sort by.' So when I have a comic, and I sort by series-creator-title-chapter-page it should show page 1 (the title page) when I collect, but when I refresh the files, it shows a random page. If, when I'm seeing a random page from refreshing, though, I change what I'm collecting by(uncheck and recheck the box), it shows the first page again, correctly.

>>12514

>>12533

Just to follow up on this, I think it may have been my fault. It seems like the trashed images only still appear on pages which are/were opened as new page -> files -> all local files and do not appear on pages opened with new page -> files -> my files. I'm still not really sure what the distinction is between these two page types though.

That being said, there may still be a bug here. On the all local files pages, the images will be removed from view right after being deleted, but then show up again after a refresh (with the little trash can icon).

>>12606

Thank you for this report. Yeah, it looks like I messed up some of the sort/collect publishing code when I added collect save this past week. I am 99.7% confident I can fix it for 353.

>>12619

all local files is a catch-all I added that combines both my files and trash (and also the secret repository updates domain that you don't normally see). If a file is in all local files, it should be on your hard drive, and when a file leaves this domain at the db level, it is scheduled for physical deletion. It is a bit of a stupid technical difference, and I generally recommend you just search my files unless you are doing something special. When I eventually add support for multiple local files domains, it will gain importance. I think I will hide this file domain for users who have not turned on help->advanced mode as it is not important and has made for some confusion before as well.

Thank you again for the report, it was not your fault. I looked through the code, and the remove on all local files pages was unintentional. The remove option is supposed to reflect a quicker 'sync' with what is true, so when you trash from a 'my files' domain, removing instantly is an accurate option since a search refresh will not reproduce the trashed file. But all local files pages include trash, so sending to trash should not remove them.

advanced operations > copy tags from ptr to local db is not working for me in 353. No error, just no effect. Happens for single and multiple images.

>>12697

Actually, does the mass copy not lock Hydrus anymore? It might have happened slowly in the background after all. A visual indicator would be a great addition if so.

Separate bug:

File URLs are not working in Firefox for me although it does for my chromium browser. Maybe it's because my Firefox path has a space in it?

Was scrolling through my library and got this error message:

AttributeError

'ThumbnailMediaCollection' object has no attribute 'GetMediaResult'

Traceback (most recent call last):

File "include\HydrusPubSub.py", line 141, in Process

callable( *args, **kwargs )

File "include\ClientGUIMedia.py", line 2414, in ProcessContentUpdates

self._RedrawMedia( affected_media )

File "include\ClientGUIMedia.py", line 3000, in _RedrawMedia

self._FadeThumbnails( thumbnails_to_render_now )

File "include\ClientGUIMedia.py", line 2717, in _FadeThumbnails

self._DirtyAllPages()

File "include\ClientGUIMedia.py", line 2628, in _DirtyAllPages

self._DirtyPage( clean_index )

File "include\ClientGUIMedia.py", line 2642, in _DirtyPage

HG.client_controller.GetCache( 'thumbnail' ).CancelWaterfall( self._page_key, thumbnails )

File "include\ClientCaches.py", line 2418, in CancelWaterfall

cancelled_media_results = { media.GetMediaResult() for media in medias }

File "include\ClientCaches.py", line 2418, in <setcomp>

cancelled_media_results = { media.GetMediaResult() for media in medias }

AttributeError: 'ThumbnailMediaCollection' object has no attribute 'GetMediaResult'

Simple downloader (all files linked by images in page) gives the below error with a fireden.net thread (for example: https://boards.fireden.net/v/thread/464181401/#464181401). Images will only import as normal if they are opened in a browser first. Client 353.

The server's error text was too long to display. The first part follows, while a larger chunk has been written to the log.… (Copy note to see full error)

Traceback (most recent call last):

File "include\ClientImportFileSeeds.py", line 1274, in WorkOnURL

self.DownloadAndImportRawFile( file_url, file_import_options, network_job_factory, network_job_presentation_context_factory, status_hook )

File "include\ClientImportFileSeeds.py", line 594, in DownloadAndImportRawFile

network_job.WaitUntilDone()

File "include\ClientNetworkingJobs.py", line 1137, in WaitUntilDone

raise self._error_exception

include.HydrusExceptions.ServerException: The server's error text was too long to display. The first part follows, while a larger chunk has been written to the log.

<!DOCTYPE html>

<!–[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]→

<!–[if IE 7]> <html class="no-js ie7 oldie" lang="en-US"> <![endif]→

<!–[if IE 8]> <html class="no-js ie8 oldie" lang="en-US"> <![endif]→

<!–[if gt IE 8

>>12697

>>12699

Thank you for these reports. I will check the UI for the advanced operations. It is debug-tier at the moment, so if it is completing asynchronously, I'll change it to wait and throw up a modal progress dialog or something.

For the file urls, is this clicking on them in the top-right hover or similar? And they fail to open if you have firefox's path set in options->external programs? If firefox is in "Program Files" or something with a space, I think it is supposed to be ok–especially if Chromium is similar and works–, but you can try wrapping the path in quotes, maybe?

"C:\Program Files\firefox\firefox.exe" "%path%"

Or could it be a firefox profile issue? If you have multiple profiles set up and need to use -profile profile_path to launch your FF, maybe that is it?

You can explore this further by turning on help->debug->report modes->subprocess report mode. That should spam a big set of popups if you have a custom url launch path set up. If it isn't obvious to you, feel free to take a screenshot of the popups and post them here, but make sure to exclude or redact any personal info.

>>12766

Shit, I am sorry. I have had this from a couple of other users as well. I fucked up the new clever thumbnail queueing with collections. This will spam any time collections that are about to fade get an update event like fast scrolling. I am 99.7% sure I can have this fixed for next Wed. I apologise for the inconvenience.

>>12768

Thank you for this report. When I put that URL in the simple downloader with 'all files linked by images in page' simple downloader formula it works ok. When I do '4chan thread (all linked images)' it downloads the thread apparently ok but don't parse any file URLs.

That error says ServerException (typically a 5XX error), so is it possible the domain was having trouble when you tried it? If you go to install_dir/db/client - 2019-05.log and do a ctrl+f to find that section, what does the extended stuff say? That should be most if not all of the html returned by the server–does it have something like 'currently undergoing maintenance' or anything? Or maybe a cloudflare page?

>>12792

Thanks for your help. Seems to be working for me now too.

The extended log message is 112 lines long, but the gist of it is that there was an error 522 (connection timed out), and, yeah, some cloudflare page stuff (short version below, tags removed). Weird because I had no such trouble connecting via Firefox at that time.

…

What happened?

The initial connection between Cloudflare's network and the origin web server timed out. As a result, the web page can not be displayed.

…

What can I do?

If you're a visitor of this website:

Please try again in a few minutes.

If you're the owner of this website:

Contact your hosting provider letting them know your web server is not completing requests. An Error 522 means that the request was able to connect to your web server, but that the request didn't finish. The most likely cause is that something on your server is hogging resources. Additional troubleshooting information here. https://support.cloudflare.com/hc/en-us/articles/200171906-Error-522

>>12793

The Eternal CloudFlare strikes again.

I'll update that ServerException result to put the actual status code, 522 or whatever, at the front of the text so it is more obvious that the solution here is to wait a bit and try again.

>>12789

Experimented some more, and I found out that it was purely due to a peculiarity in how I launch Firefox (from a batch file). The same problem happens no matter how I open a URL, so it is nothing to do with Hydrus :)

>>12821

Great, thanks for letting me know.

I keep getting these errors when trying to use collections after the last update. It only works if I set the collection then f5 the search.

AttributeError
'ThumbnailMediaCollection' object has no attribute 'GetMediaResult'
  File "site-packages\wx\core.py", line 3259, in <lambda>
  File "include\ClientGUIPages.py", line 2234, in MediaDragAndDropDropped
    source_page.GetMediaPanel().RemoveMedia( source_page.GetPageKey(), hashes )
  File "include\ClientGUIMedia.py", line 2454, in RemoveMedia
    self._RemoveMediaByHashes( hashes )
  File "include\ClientMedia.py", line 854, in _RemoveMediaByHashes
    self._RemoveMediaDirectly( affected_singleton_media, affected_collected_media )
  File "include\ClientGUIMedia.py", line 3063, in _RemoveMediaDirectly
    self._DirtyAllPages()
  File "include\ClientGUIMedia.py", line 2628, in _DirtyAllPages
    self._DirtyPage( clean_index )
  File "include\ClientGUIMedia.py", line 2642, in _DirtyPage
    HG.client_controller.GetCache( 'thumbnail' ).CancelWaterfall( self._page_key, thumbnails )
  File "include\ClientCaches.py", line 2418, in CancelWaterfall
    cancelled_media_results = { media.GetMediaResult() for media in medias }
  File "include\ClientCaches.py", line 2418, in <setcomp>
    cancelled_media_results = { media.GetMediaResult() for media in medias }

>>12828

Hey, thank you for this report. I apologise. I had this from several other people as well. I fucked up how collections are dequeued from the thumbnail 'waterfall' fade system in 354, so many update events that occur while collection thumbs are fading in (like fast scrolling) throw this error. It is fixed for 355.

I don't use collections myself much, and I keep messing up their rendering and sorting, so I will add some basic tests to my weekly routine to stop this.

Cannot import webms on Ubuntu 18.04, client 355. Details:

> fresh xubuntu 18.04 install, still a babby with it

> extract hydrus, run client

> import a few JPG, PNG, GIF; complains about ffmpeg missing

> install ffmpeg, version 3.4.6-0ubuntu0.18.04.1, reboot

> now can import those (and most) files, but a popup says ffmpeg version not detected; lines from "db/client - 2019-6.log" below. Cannot import webm (can as normal on Win7 install), but CAN import MP4.

2019/06/06 02:09:33: FFMPEG was recently contacted to fetch version information. While FFMPEG could be found, the response could not be understood. Significant debug information has been printed to the log, which hydrus_dev would be interested in.
2019/06/06 02:09:33: FFMPEG was recently contacted to fetch version information. While FFMPEG could be found, the response could not be understood. Significant debug information has been printed to the log, which hydrus_dev would be interested in.

{'universal_newlines': True, 'startupinfo': None, 'env': None}

environ({'GDM_LANG': 'en_GB', 'XDG_SEAT_PATH': '/org/freedesktop/DisplayManager/Seat0', 'MPLCONFIGDIR': '/tmp/tmpb0q9iesu', 'HOME': '/home/localuser', 'SSH_AGENT_PID': '1292', 'XDG_DATA_DIRS': '/usr/share/xubuntu:/usr/share/xfce4:/home/localuser/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share:/var/lib/snapd/desktop:/usr/share', 'QT_ACCESSIBILITY': '1', 'GPG_AGENT_INFO': '/run/user/1000/gnupg/S.gpg-agent:0:1', 'SSH_AUTH_SOCK': '/run/user/1000/keyring/ssh', 'XAUTHORITY': '/home/localuser/.Xauthority', 'DESKTOP_SESSION': 'xubuntu', 'MATPLOTLIBDATA': '/home/localuser/toplevel/Hydrus/mpl-data', 'LANG': 'en_GB.UTF-8', 'SESSION_MANAGER': 'local/xubuntu-laptop:@/tmp/.ICE-unix/1310,unix/xubuntu-laptop:/tmp/.ICE-unix/1310', 'LD_LIBRARY_PATH': '/home/localuser/toplevel/Hydrus', 'GDMSESSION': 'xubuntu', 'PWD': '/home/localuser', 'XDG_VTNR': '7', 'SHELL': '/bin/bash', 'XDG_CONFIG_DIRS': '/etc/xdg/xdg-xubuntu:/etc/xdg:/etc/xdg', 'QT_QPA_PLATFORMTHEME': 'gtk2', 'XDG_SESSION_TYPE': 'x11', 'XDG_SESSION_ID': 'c2', 'XDG_SEAT': 'seat0', 'USER': 'localuser', 'DISPLAY': ':0.0', 'XDG_SESSION_PATH': '/org/freedesktop/DisplayManager/Session0', 'GLADE_PIXMAP_PATH': ':', 'CLUTTER_BACKEND': 'x11', 'GTK_OVERLAY_SCROLLING': '0', 'XDG_GREETER_DATA_DIR': '/var/lib/lightdm-data/localuser', 'GLADE_MODULE_PATH': ':', 'XDG_MENU_PREFIX': 'xfce-', 'XDG_SESSION_DESKTOP': 'xubuntu', 'GLADE_CATALOG_PATH': ':', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus', 'LANGUAGE': 'en_GB:en', 'LOGNAME': 'localuser', 'XDG_CURRENT_DESKTOP': 'XFCE', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'XDG_RUNTIME_DIR': '/run/user/1000', 'SHLVL': '0'})

Response: 

>>12864

Thank you for this report. I have been chasing up this same issue with another user. The PATH in your environment is all correct there, and FFMPEG is being found, but it looks like hydrus is getting no response back. Just no text at all.

Ok, thank you for the debug data. I'll have another look at what I am doing here and why this is happening. I don't have my devstuff in front on me right now, but I'll check this and update the debug info to include more about what if anything was read back.

If you are feeling cheeky, you can try downloading a static ffmpeg exe here: http://ffmpeg.org/download.html and then place it in install_dir/bin. The client may find that is ok after a restart–that's how Windows users get it, and it should work the same way for Linux. If you try this, let me know if it works.

>>12868

Fifth option down, 'Hide the preview window'.

>>12889

Oh hey, that does work (with 4.1.3-static). Appreciate that.

Incidentally, I did try running hydrus via wine, which, again, I am using for the first time. I think I've set it up fine because Notepad++ runs, but hydrus crashes it immediately (see below). No big deal to me.

Thanks for the great support and enjoy your E3.

wine: Call from 0x7b44c1e7 to unimplemented function api-ms-win-core-path-l1-1-0.dll.PathCchCanonicalizeEx, aborting
wine: Unimplemented function api-ms-win-core-path-l1-1-0.dll.PathCchCanonicalizeEx called at address 0x7b44c1e7 (thread 002b), starting debugger...

Getting strange behavior where it hangs at committing for hours, then if you alt-f4 out, it closes the window and begins eating RAM in the background until manually killed.

2019/06/08 02:38:25: shutting down gui…
2019/06/08 02:38:25: waiting for daemons to exit
2019/06/08 02:38:25: public tag repository sync: processing updates
2019/06/08 02:38:25: fattening service info
2019/06/08 14:39:32: processed 5,072,024 content rows at 117 rows/s
2019/06/08 14:39:32: committing
2019/06/09 21:17:57: shutting down controller…
2019/06/09 21:17:57: hydrus client shut down
2019/06/09 21:19:10: hydrus client started
2019/06/09 21:19:10: booting controller…
2019/06/09 21:19:10: booting db…
2019/06/09 21:19:10: preparing disk cache
2019/06/09 21:19:14: preparing db caches
2019/06/09 21:19:14: booting db…
2019/06/09 21:19:14: initialising managers
2019/06/09 21:20:16: booting gui…

I've got two images, one of which i've accidentally tagged as a worse quality dupe to the other. Trying to set that image as an alternate image by right clicking on both, Duplicates > Set relationship > Set as Alternates results in both files still being tagged as one worse and one better. Can anyone else confirm?

>>12910

Yeah, although I'm like 95% sure hydrusdev knows, I recall reading something about it like a week ago.

If you're on 355 (possibly 354 too), the duplicate system is in the middle of an overhaul and hydrusdev needs to fix the context menu duplicate commands. I've been having a fun time with cg sets due to this.

Using v355 on linux. I filter deduplicates bit it doesn't apply anything. After increasing the search distance from 0 to 2 it was still listing all the previous images. I moved to 4 and it still lists the previous images from distances 0 and 2 since the filter didn't apply the commits I guess. It worked previously so v355 broke it.

>>12940

Same here. I made the mistake of importing multiple CG sets, then scratching my head wondering if I've already sorted their duplicates halfway through.

>>12895

Hey, I have extended that FFMPEG debug info dump to say more in today's release. If you like, you can try doing help->about again in 356 with your system FFMPEG, but it is not a huge priority–I am working with another user on this same bug. If you work and are happy now, I am fine to leave it alone for the time being.

I am not totally sure what that dll issue is, but I think it may be related to VCRedist. Ever since I moved to Python 3, I think you need VCRedist 2015 on a pre-Win10 Win environment:

https://www.microsoft.com/en-us/download/details.aspx?id=48145

I think those api-ms-win-… dlls are related to that.

>>12903

Thank you for this report. It seems like the client did a gigantic repository processing job there, with the 5M rows in one go over about twelve hours. Are you by any chance on a slightly older version of the client? New clients should only ever do 1 hour of repo processing at once and make checkpoint transactions to keep cancel lag low, and they should keep big transactions out of memory.

If you are on an older version, can you try updating to, say, at least 354? As it happens, today's 356 will have improved cancel tech here, including a cancel button on the exit splash screen. If it is convenient, I would recommend today's release.

>>12906

That's interesting, well done! If you have a github account, there is a big repository of parsers here:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/Download%20System

I have previously made commitments not to personally work significantly on new Sankaku parsers (one of the owners contacted me to let me know they are always short on bandwidth, for which I sympathised), but the owner of that repo would be interested in this neat discovery, I am certain.

>>12910

>>12915

>>12940

>>12941

Hey, I am sorry for the current difficulties here. The duplicates system is in slight limbo as I move each component over. Some features such as undoing/dissolving groups are not yet available, and certain logic behind figuring out which files are alternate inside an alternate group are still inefficient. Today's release will move better/worse/same quality dupes over to the new system, which will improve the situation but not complete it.

Now I am in the weeds of this code, I am only discovering how bad the old system was. Even though some 'reset and requeue' actions appeared to work, they often left invalid relationships behind. The new system I am moving to is comprehensive and reduces potential pairs much faster due to correct transitivity, but it will be a few more weeks until everything is working on the same data.

You shouldn't be able to set anything 'invalid' on the new system, but in situations where it doesn't know what to do, like manually overruling a better/worse with an alternate, it makes no changes. Please bear with me for a bit.

>>12940

When you apply an action to some duplicates, is this through the duplicate filter or the 'show some random dupes' button? Are the 'potential' pair counts decrementing at all? I believe the duplicate filter will always decrement the potential pairs count, but some actions may not decrement the count completely yet (I think in some cases, like setting alternates, it may not decrement it at all), but I am not certain. The situation will improve in today's 356, particularly for setting groups of alternates, and moreso in future weeks.

>>12949

Yes, I am afraid they set CloudFlare javascript protection on that API request URL. The hydrus downloader cannot pass through that yet, so the downloader is currently broken.

Encountered a strange bug with client version 355 where after the client had been open for over 24 hours, Ctrl+V was not working for pasting links. A restart of the client fixed it, but I thought it should be made aware of.

>>12956

Sure - 356 reports the same thing in-UI, but the log now gives the below:

2019/06/20 01:36:53: FFMPEG was recently contacted to fetch version information. While FFMPEG could be found, the response could not be understood. Significant debug information has been printed to the log, which hydrus_dev would be interested in.
2019/06/20 01:36:53: FFMPEG was recently contacted to fetch version information. While FFMPEG could be found, the response could not be understood. Significant debug information has been printed to the log, which hydrus_dev would be interested in.

{'env': None, 'universal_newlines': True, 'startupinfo': None}

environ({'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'DESKTOP_SESSION': 'xubuntu', 'SSH_AUTH_SOCK': '/run/user/1000/keyring/ssh', 'SHELL': '/bin/bash', 'LOGNAME': 'localuser', 'SESSION_MANAGER': 'local/xubuntu-laptop:@/tmp/.ICE-unix/3665,unix/xubuntu-laptop:/tmp/.ICE-unix/3665', 'XDG_GREETER_DATA_DIR': '/var/lib/lightdm-data/localuser', 'XDG_SESSION_PATH': '/org/freedesktop/DisplayManager/Session0', 'LD_LIBRARY_PATH': '/home/localuser/toplevel/Hydrus', 'QT_QPA_PLATFORMTHEME': 'gtk2', 'QT_ACCESSIBILITY': '1', 'XDG_SEAT': 'seat0', 'GDM_LANG': 'en_GB', 'DEFAULTS_PATH': '/usr/share/gconf/xubuntu.default.path', 'DISPLAY': ':0.0', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus', 'MPLCONFIGDIR': '/tmp/tmpzqtgebog', 'XDG_CONFIG_DIRS': '/etc/xdg/xdg-xubuntu:/etc/xdg:/etc/xdg', 'GTK_OVERLAY_SCROLLING': '0', 'XDG_SESSION_DESKTOP': 'xubuntu', 'GLADE_CATALOG_PATH': ':', 'XDG_CURRENT_DESKTOP': 'XFCE', 'XDG_RUNTIME_DIR': '/run/user/1000', 'GLADE_MODULE_PATH': ':', 'GPG_AGENT_INFO': '/run/user/1000/gnupg/S.gpg-agent:0:1', 'GLADE_PIXMAP_PATH': ':', 'XDG_SESSION_ID': 'c2', 'XDG_DATA_DIRS': '/usr/share/xubuntu:/usr/share/xfce4:/home/localuser/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share:/var/lib/snapd/desktop:/usr/share', 'GDMSESSION': 'xubuntu', 'MATPLOTLIBDATA': '/home/localuser/toplevel/Hydrus/mpl-data', 'USER': 'localuser', 'XAUTHORITY': '/home/localuser/.Xauthority', 'SSH_AGENT_PID': '3647', 'LANG': 'en_GB.UTF-8', 'MANDATORY_PATH': '/usr/share/gconf/xubuntu.mandatory.path', 'CLUTTER_BACKEND': 'x11', 'LANGUAGE': 'en_GB:en', 'SHLVL': '0', 'XDG_SESSION_TYPE': 'x11', 'HOME': '/home/localuser', 'XDG_SEAT_PATH': '/org/freedesktop/DisplayManager/Seat0', 'XDG_VTNR': '7', 'PWD': '/home/localuser', 'XDG_MENU_PREFIX': 'xfce-'})

STDOUT Response: 

STDERR Response: ffmpeg: /home/localuser/toplevel/Hydrus/libz.so.1: version `ZLIB_1.2.9' not found (required by /usr/lib/x86_64-linux-gnu/libpng16.so.16)

This is LTS, but it is up-to-date. Looking at my packages tells me roughly that zlib is at 1.2.11.

I'll get back to you about the dll issues.

Encountered this bug in 355, updated to 356 and it persists.

Tried to separate a Hentai Foundry artist lookup subscription in the 'manage subscriptions' window.

TypeError

'<' not supported between instances of 'SubscriptionQuery' and 'SubscriptionQuery'

File "include\ClientGUICommon.py", line 781, in EventButton

self._func( *self._args, **self._kwargs )

File "include\ClientGUIScrolledPanelsEdit.py", line 4952, in Separate

panel = EditChooseMultiple( dlg, choice_tuples )

File "include\ClientGUIScrolledPanelsEdit.py", line 197, in init

choice_tuples.sort()

Same thing happens with the rule34xxx tag lookup when trying to separate the subscription.

Bug didn't happen when separating a danbooru+gelbooru tag lookup subscription.

>>12959

>is this through the duplicate filter or the 'show some random dupes' button?

Through the duplicate filter.

>Are the 'potential' pair counts decrementing at all?

Yes when I'm finished the potential pairs count goes back to 0 but it really hasn't applied anything.

I updated to v356 today. I imported a few new images and rescanned for duplicates. I tried doing the same thing and it's still broken. Hopefully in the next release or two you can figure it out.

>>12991

parser is working again, didn't do anything just left it idle. not really sure what caused it or why it happened. something on e6's side?

>>12993

They turned on pretty aggressive ddos protection for a few hours. Everything that was not a legitimate browser received access forbidden errors.

>>12979

Thank you for this report. Do you have the new 'clipboard watcher' on? Is it possible the clipboard crashed from, say, trying to pull an in-clipboard screenshot as text or anything?

>>12981

Thank you, this was exactly what I was looking for. It looks like the env being passed still has some hydrus-specific path somewhere so ffmpeg is trying to load .so files from the hydrus dir. LD_LIBRARY_PATH, looks like, which I thought I was already redirecting–looks like the LD_LIBRARY_PATH_ORIG, which I am supposed to have, isn't there for your situation. What a headache. I will give this another go.

>>12984

Shit, thank you for this report. I am 99.7% certain I can have this fixed for 357.

>>12987

Are you clearing your duplicate search progress ('reset potential duplicates' from the cog icon under the duplicates page) and researching everything? If so, you do not need to do this, and I would not recommend this action for any normal operation. You do not need to 'rescan' for dupes in this way to keep up with new files. It would requeue many intra-alternate comparisons with the current system.

If you are just 'filling up' the 'finding potential duplicates' search bar and getting the same alternate pairs come up over and over, this would point to a different problem.

Either way, this whole situation with comparing alternates is on my mind, as here also >12999 . As I switch potentials over, I will write an alternates decisions cache to make sure you don't ever see the same thing twice, even if you were to flush and re-search everything.

>>12993

>>12991

>>12996

Thank you for this report. I'm going to try to write a 'This looks like a Cloudflare error' detector to try to give better summary info in these error cases, since this keeps happening.

>>13007

>If you are just 'filling up' the 'finding potential duplicates' search bar and getting the same alternate pairs come up over and over, this would point to a different problem

Yes, that's what I've been doing. I can give you logs or something if it'll help debug the issue. If you're re-writing it I guess you won't need them though.

For whatever reason, the pixiv single file page parser stopped working for me - every url I plug in tells me it found nothing in the document. In looking at the raw data from the example urls I noticed that any nsfw pages tell me I'm not logged in but I've checked the login credentials outside of hydrus and they're correct, and regardless sfw submissions don't download properly either. It had been working as of yesterday and the tag gallery parser still picks up urls, albeit only non-nsfw. Did something change on pixiv's end or what?

It chucks a few

include.HydrusExceptions.ParseException: Unable to parse that JSON: Expecting value: line 1 column 1 (char 0)

>>13005

Looks like v357 is detecting my system ffmpeg properly, such that it can import webms without the static one now. Great stuff. Thanks for your hard work.

So I found an interesting yiff.party issue. So I've noticed one subscription keeps failing and gives me the message 'connection broke mid-request' and retries over and over to no success.

So I decided to look into it, the particular json it is trying to download is here: https://yiff.party/7242260.json

If you actually look at it in a browser you'll see that it literally cut off part of the way through, or at least it does for me off my VPN, on several different VPN servers, and on TOR. I suppose the error message it gave me was correct. Any idea on what's causing that? I'd guess a server error

>>13038

Also similiar issue here, for example the parser finds nothing for https://www.pixiv.net/member_illust.php?mode=medium&illust_id=64809850

>>13011

Thanks. Yes, I will wait a bit for this. The new potentials system will work better all-around and plug into the new alternates and duplicates storage logically correctly, so a bunk of jank will clear out. There will be a couple of weekly follow-ups later as well that will ensure certain alternate decisions are remembered better. Please keep me updated on how this goes for you as I roll more of this out.

>>13055

>>13038

Thank you for these reports. Yeah, it looks like Pixiv changed up their internal API again. A user has lined up a good parser that I will fold into 358.

As for yiff.party, I am no expert with it, but in some testing I have been doing with another user's new parser, I have found the server very unreliable. It is always under heavy load and often just cuts off connections, including for media content. For some reason my network engine is not picking some of these errors up and is delivering broken json and jpegs–I assume Yiff is not delivering Content-Length header or something, so I don't know final expected content length. I will keep looking at this a bit and likely fold in this newer better parser in the coming weeks. My engine deals with connection errors better in recent weeks, but there aren't excellent ways to detect half-done content. I will keep thinking about it.

>>13047

Fantastic, thanks for letting me know.

If I have both the media viewer and file tag editing windows open, I can no longer scroll down on the media viewer to advance to the next image until I click on the media viewer to give it focus.

>>13074

Thank you for this report. The new unified shortcut processing test, which fixes various focus issues, accidentally does not have an exception for scroll events. I am 99.7% confident I can have this fixed for 358.

In the meantime, one way of browsing from manage tags is to hit page up/down on the tag autocomplete input box–it should pass a previous/next event up to the media viewer to which it is attached.

Small bug:

If you open the manage tags window in the dupe filter, when you close it, focus is set to the main hydrus window instead of the dupe filter. The main window pops up over. Windows 10.

Hey there,

my client (v357) seems to reliably stall when I try to import https://e621.net/post/show/1915926

It downloads the image but never gets done with the processing. Not sure on what step exactly it gets stuck. But the client.exe will show around 30% CPU load. (Windows 10, 4 cores/threads system).

It makes no difference if the url is hit by a subscription or a gallery downloader.

I tried the same with various debug>reporters enabled and the log looked like this:

2019/07/02 14:05:08: Query "id:1915911..1915940" pre-work file login test. Login ok: True.
2019/07/02 14:05:08: Running read url_statuses
2019/07/02 14:05:08: CallToThread doing a job.
2019/07/02 14:05:09: Running read url_statuses
2019/07/02 14:05:09: Running read hash_status
2019/07/02 14:05:09: Running read in_inbox
2019/07/02 14:05:09: Running write content_updates
2019/07/02 14:05:09: Running read nums_pending
2019/07/02 14:05:09: Query "id:1915911..1915940" pre-work bandwidth test. Bandwidth ok: True.
2019/07/02 14:05:09: Query "id:1915911..1915940" pre-work file login test. Login ok: True.
2019/07/02 14:05:09: Running read url_statuses
2019/07/02 14:05:09: CallToThread doing a job.
2019/07/02 14:05:10: Running read url_statuses
2019/07/02 14:05:10: Running read url_statuses
2019/07/02 14:05:10: Running read url_statuses
2019/07/02 14:05:10: Running read hash_status
2019/07/02 14:05:10: CallToThread doing a job.
2019/07/02 14:05:12: Running read hash_status
2019/07/02 14:05:31: CallToThread doing a job.
2019/07/02 14:05:31: Running write serialisable
2019/07/02 14:05:31: Running write serialisable
2019/07/02 14:06:00: CallToThread doing a job.
2019/07/02 14:06:00: CallToThread doing a job.
2019/07/02 14:06:01: CallToThread doing a job.
2019/07/02 14:06:22: CallToThread doing a job.
2019/07/02 14:06:31: CallToThread doing a job.
2019/07/02 14:06:59: CallToThread doing a job.
2019/07/02 14:07:11: CallToThread doing a job.
2019/07/02 14:07:11: CallToThread doing a job.
2019/07/02 14:07:11: CallToThread doing a job.
2019/07/02 14:07:11: CallToThread doing a job.
2019/07/02 14:07:11: CallToThread doing a job.
2019/07/02 14:07:12: CallToThread doing a job.
2019/07/02 14:07:12: Running read file_system_predicates
2019/07/02 14:07:13: CallToThread doing a job.
2019/07/02 14:07:13: CallToThread doing a job.
2019/07/02 14:07:13: Running read serialisable_names
2019/07/02 14:07:13: Running read serialisable_names
2019/07/02 14:07:41: CallToThread doing a job.
2019/07/02 14:08:11: CallToThread doing a job.
2019/07/02 14:08:11: CallToThread doing a job.
2019/07/02 14:08:11: CallToThread doing a job.
2019/07/02 14:08:22: CallToThread doing a job.
2019/07/02 14:08:41: CallToThread doing a job.
2019/07/02 14:09:11: CallToThread doing a job.
2019/07/02 14:09:11: CallToThread doing a job.
2019/07/02 14:09:11: CallToThread doing a job.
2019/07/02 14:09:41: CallToThread doing a job.
2019/07/02 14:10:11: CallToThread doing a job.
2019/07/02 14:10:11: CallToThread doing a job.
2019/07/02 14:10:11: CallToThread doing a job.
2019/07/02 14:10:13: CallToThread doing a job.
2019/07/02 14:10:13: Running read serialisable_names
2019/07/02 14:10:13: CallToThread doing a job.
2019/07/02 14:10:13: Running read serialisable_names
2019/07/02 14:10:22: CallToThread doing a job.
2019/07/02 14:10:41: CallToThread doing a job.
2019/07/02 14:11:11: CallToThread doing a job.
2019/07/02 14:11:11: CallToThread doing a job.
2019/07/02 14:11:11: CallToThread doing a job.
2019/07/02 14:11:41: CallToThread doing a job.

Cheers.

>>13089

Thank you for this report. There's some funky focus rules with this stuff–I think when the always-on-top-of-parent manage tags window exits, it reverts focus on like the initial run to the most top-level window, rather than its parent. I think I have some catches for this in the main media viewer, but I presume the duplicate filter isn't handling it right. I will have a look at this today.

>>13091

Thank you for this report. I am afraid I do not get this problem either on my dev machine or my irl client.

Do you sync with my PTR? Have you personally uploaded a number of new tag siblings or parents to it before?

Can you try downloading the file manually, should be this link:

https://static1.e621.net/data/01/9e/019e3fd020b4ea13e578576b128e5986.jpg

And importing it just from your hard drive? If it imports, does it come in as a new file or 'already in db'? Do you see it having tags?

If it imports ok as just a bare file, what happens if you queue up the URL again, now the file is imported? If the URL freezes up, what happens if you open a new url downloader and set its tag import options to parse no tags and try again–does it still freeze?

If that still fails, what happens if you open network->downloader definitions->manage parsers and edit the e621 file page parser and put in the URL in the top-right of that edit panel to pull the data and then click 'test parse'? I get this:

*** 1 RESULTS BEGIN ***

tag: character:mario
tag: character:tanooki mario
tag: creator:ultimatesol
downloadable/pursuable url (priority 75): https://static1.e621.net/data/01/9e/019e3fd020b4ea13e578576b128e5986.jpg
tag: 1up mushroom
tag: black eyes
tag: blue eyes
tag: brown skin
tag: cloud
tag: coin
tag: fungus
tag: hi res
tag: mushroom
tag: red skin
tag: star
tag: super leaf
tag: super mushroom
tag: super star
tag: video games
tag: white skin
md5 hash: 019e3fd020b4ea13e578576b128e5986
source time: 2019/06/26 19:00:00
tag: series:mario bros
tag: series:nintendo
associable/source url (priority 0): https://www.deviantart.com/ultimatesol/art/Super-Mario-3D-Land-272390217
tag: species:canid
tag: species:canine
tag: species:goomba
tag: species:human
tag: species:lakitu's cloud
tag: species:mammal
tag: species:raccoon dog
tag: species:tail goomba
tag: species:tanuki
tag: species:toad (mario)

*** RESULTS END ***

>>13096

> Do you sync with my PTR? Have you personally uploaded a number of new tag siblings or parents to it before?

Yes, I do sync and I have uploaded new tags before.

> Can you try downloading the file manually, should be this link:

https://static1.e621.net/data/01/9e/019e3fd020b4ea13e578576b128e5986.jpg

And importing it just from your hard drive?

I used

- file->import files->add files

- selected the file

- 1/1 files parsed successfully

- import now

and the page "import (0, 0/1) does not appear to do anything. Closing the page asks if I really want to close while it is still importing.

> If that still fails, what happens if you open network->downloader definitions->manage parsers and edit the e621 file page parser and put in the URL in the top-right of that edit panel to pull the data and then click 'test parse'? I get this:

I tried this just in case and the result at first glance looks like your's.

Deviantart login appears to have broken for me. I have been gone for about a month, so it is likely cookies have expired.

However, attempting to refresh the login does not work, and returns: "invalid - The temporary variable 'validate_key' was not found!"

I also tried turning the login off, and downloading some content that did not require login, then turning it on again, no luck.

>>13097

Thank you. Looks like it is definitely the file itself, then. If you are still up to chase this error down, can you try extracting a new fresh copy of hydrus to your desktop or something and then trying to import the file from your hard drive to that new, empty client? This will help us figure out if there is a hardware problem with your computer, like some weird bug in your graphics driver that means hydrus can't do some processing on file import on that file, or if there is an issue with your specific IRL client install db (which I strongly suspect).

My best guess here is that on import the file has some similar-files metadata or something that is not inserting correctly into your db. Maybe something got janked up and your similar files search tree has a loop. I am gearing up now to put today's release out, so I don't have time to add it for 358, but I will try to fit in a 'file import report mode' or similar for 359 that will expose what is going on here so we can see better where your client is running into trouble.

>>13099

Thank you for this report. Sounds like they have changed their process. I will try to have a look at this next week.

>>13104

I'm totally up to chase the error down.

I tried a fresh copy of 357 and the file imported fine.

Using my existing db with the fresh copy showed the broken behaviour again.

Updating to 358 (because the change log said something about db loops) made no difference.

So as you suspected, it has to do with my db.

If there's anything I can query on the sqlite db directly that'll help, let me know. I'm not afraid to get my hands dirty with the command line client.

>>13119

Great. Shut down the client and let's open up the sqlite3 in install_dir/db and then do this: (you should be able to copy/paste each line)

.open client.db
ATTACH "client.master.db" AS cm;
ATTACH "client.caches.db" AS cc;
SELECT hash_id FROM hashes WHERE hash = x'8fdab765d788fb987fceee739bcfe3abdc0af747e45ef0e204b6e930739ff1aa';

(note this hash_id number. if it does not give anything, then .exit now and let me know)

SELECT * FROM shape_search_cache WHERE hash_id = (your_hash_id);
SELECT * FROM shape_perceptual_hashes NATURAL JOIN shape_perceptual_hash_map WHERE hash_id = (your_hash_id);
SELECT * FROM shape_perceptual_hashes WHERE phash = x'fb33c17a84e0966c';
SELECT phash_id FROM shape_perceptual_hash_map WHERE hash_id = (your_hash_id);

(note any phash_ids here, should be either one or none)

SELECT * FROM shape_vptree WHERE phash_id = (your_phash_id);
.exit

And let me know what you get. It is possible the shape search stuff will just give you no results. But this will let us know if the file has been seen by your db–likely through PTR sync–and if it somehow already has shape data, or if the shape data is colliding for some odd reason with an existing file.

Note if the error is located here, maybe due to a malformed search tree, there is an outside chance that database->regenerate->similar file search tree will fix this. It should take a couple of minutes or less. If the operation hangs indefinitely, then we have definitely found our problem.

>>13131

SQLite version 3.27.2 2019-02-25 16:06:06
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> .open client.db
sqlite> ATTACH "client.master.db" AS cm;
sqlite> ATTACH "client.caches.db" AS cc;
sqlite> SELECT hash_id FROM hashes WHERE hash = x'8fdab765d788fb987fceee739bcfe3abdc0af747e45ef0e204b6e930739ff1aa';
30364954
sqlite> SELECT * FROM shape_search_cache WHERE hash_id = 30364954;
sqlite> SELECT * FROM shape_perceptual_hashes NATURAL JOIN shape_perceptual_hash_map WHERE hash_id = 30364954;
sqlite> SELECT * FROM shape_perceptual_hashes WHERE phash = x'fb33c17a84e0966c';
sqlite> SELECT phash_id FROM shape_perceptual_hash_map WHERE hash_id = 30364954;
sqlite>

I guess this means no results and there's no need to regenerate the similar file search tree?

What's next?

>>13060

Just letting you know the deduplicate filtering works normally again for me in v358.

I've just updated from 354 to 358, and I can't use file lookup scripts anymore. I get this error every time:

UnboundLocalError

local variable 'f' referenced before assignment

Traceback (most recent call last):

File "include\HydrusThreading.py", line 342, in run

callable( *args, **kwargs )

File "include\ClientGUITagSuggestions.py", line 483, in THREADFetchTags

parse_results = script.DoQuery( job_key, file_identifier )

File "include\ClientParsing.py", line 2722, in DoQuery

parsing_text = self.FetchParsingText( job_key, file_identifier )

File "include\ClientParsing.py", line 2688, in FetchParsingText

if f is not None:

UnboundLocalError: local variable 'f' referenced before assignment

>>13105

Unfortunately still not working for me.

I tried just clicking "do login now", no dice. "The temporary variable 'csrf_token' was not found!"

I also reset login (clear cookies), then "do login now", still no dice.

Version 359.

I'm playing with the login script now, I'll let you know if I figure anything out….

>>13105

>>13161

OK…..

So after waiting a while, trying some more with no success, and toying with the login script to no success, it suddenly started working in the login script testing area sometimes, but wouldn't work for the real logins…..

I then Reset the login completely, and tried login several times, no dice.

After testing it probably 15 more times, I can get it to work now…. about 20% of the time, and I have absolutely no idea why.

So it's working for me now…… I'm just left confused.

Thanks!

>>13143

So I enabled the new file import report and got this

2019/07/13 01:03:12: File import job starting work.
2019/07/13 01:03:12: File import job hash: 8fdab765d788fb987fceee739bcfe3abdc0af747e45ef0e204b6e930739ff1aa
2019/07/13 01:03:12: File import job pre-import status: , 
2019/07/13 01:03:12: File import job mime: image/jpg
2019/07/13 01:03:12: File import job testing for decompression bomb
2019/07/13 01:03:12: File import job file info: (655563, 1, 918, 1279, None, None, None)
2019/07/13 01:03:12: File import job generating thumbnail
2019/07/13 01:03:12: File import job generating phashes

And then it sits there.

>>13147

The deduplicate filtering isn't working properly anymore in v359. Distance 0 is giving me very dissimilar matches.

>>13172

>>13143

Hey, I am sorry I am getting back to this late. By that 'file import job generating phashes' being the last statement, it is definitely having trouble here simply using CPU/GPU time to generating similar search data for that file. The db appears fine.

This is double-odd, as none of this code stands out as something that could hang. It basically loads the image and does some transforms and math on it–normally, if there were a probeem, it would error out with some decent text. It can seemingly load your image ok, since it figures out the resolution and generates the thumbnail ok.

The image looks like a normal RGB jpeg, not like a 14,000x22,000 png or something that might blow up ram doing this. So, my guess here is it is something like your graphics driver is unhappy with some particular part of the shape of that file so that when it does DCT to it, or tries to greyscale it, it is hanging on that call. The image libraries I use send difficult image math to the GPU using OpenCL, and this sort of this has come up once or twice before.

I can think of two things to try:

1) This sounds stupid, but update your graphics driver. If this happens to be a bug in your current version, it may be cleared up in the latest.

2) See if you can disable OpenCL acceleration for client.exe under your graphics driver settings, at least for this one file. You do this just like you can override FXAA and Vsync etc… for a game exe. For Nvidia, it is under the normal Nvidia Settings panel and then 'Manage 3D Settings', and then you make a new profile for hydrus's client.exe.

Otherwise, I'll make a new 'similar files data generation report mode' so we can drill down to exactly what is hanging here for you.

>>13147

>>13186

Thank you for this report. Can you post an example pair of dissimilar images that are being matched for you, so I can check on my end? I don't mind nsfw.

Also, if you load up one of these images in a search page and right-click->file relationships->find similar->exact match/speculative, does it give the same bad results in those cases, or does it give just the original file? Do either of the files in the example bad pair have duplicates or alternates already set? Could they be accidental false-positives?

>>13150

Hey, I am sorry for this. It should be fixed in the latest 359. Thank you for the report.

>>13161

>>13163

Thank you for the feedback. As far as I can tell, DA is rolling out the new layout/login system/whatever on a pseudo-random A/B testing basis or something. It seems like once you are logged in, it will 'fix' you to the new layout for the duration of that login, but I have also seen some 500 Server Errors. If they do it like Pixiv did, I assume they'll slowly roll it all out to all users 100% of the time and ultimately retire the old layout. Phoneshit compatibility and dynamic website drawing seems to be the way of the future.

>>13167

I am not sure. I have seen this seemingly false 'db is locked' error on boot a couple of times before but was unable to pin it down. Is there any chance there is/was an old 'client.exe' process hanging around in Task Manager, unable to close itself (and tidy up the db locks) from a previous session? Or was this on a freshly booted computer?

My best recommendation is to see if there is an old ui-hidden client.exe in Task Manager, and kill the process if so, and if not, try rebooting the computer.

If you still have trouble, try going into install_dir/db and running the sqlite3 executable there. It will open up a new console that you can paste commands into. Do this process four times for the four 'client' .db files:

.open client.db
SELECT * FROM sqlite_master LIMIT 1;
.exit

That will try to open and read some simple info from each file (i.e. swap in 'client.db' for 'client.caches.db' and so on). That may end up silently 'clearing out' and bad lingering locks so you can boot again, or it may report a better error here. Let me know how you get on.

>>11542

I cant seem to run python scripts from open externally anymore. This is the command I'm using [python "C:\Users\*\My_Documents\_tiff_shortcuts\reader.py" "%path%"]. It opens a blank cmd window with python as its title for a split second. I did some testing and if I just have the python exe open with no other arguments it still doesnt work. I also tried a very simple script with no %path% argument which asked for an input, same thing. I talso tried a program that just copied over sys.argv into a txt file to see whats wrong but the file is unchanged.

>>13202

Thank you for this report. Can you try again with help->debug->report modes->subprocess report mode on? That should dump a heap of debug info into some popups and your log and help us see how the command is failing. Some of that will be private environment info, so don't post it all unredacted, but is there anything in there that pops out?

>>13198

No worries. Everyone's busy and there's no rush.

What makes me doubt the graphics driver is the import working fine with a fresh database.

Could there be anything else with my database holding back the import?

Thousands of other files (yes, I am a hoarder) have imported fine since that particular one. So generally everything works.

>>13204

Thank you. The files do not get a potential pair for me, so this is not some legit false positive accident. There is definitely something going wrong in your db. I recommend you pause automatic idle time potential pair searching by clicking on the cog icon on the duplicates page. You'll probably want to completely reset your potential duplicates as well through the same menu, once we have figured this out.

Can you please close your client and go to install_dir/db and run the sqlite3 executable to bring up the sqlite terminal, and then copy/paste these lines in one by one:

.open client.db
ATTACH "client.master.db" as cm;
ATTACH "client.caches.db" as cc;

SELECT phash_id, hex( phash ) FROM hashes NATURAL JOIN shape_perceptual_hash_map NATURAL JOIN shape_perceptual_hashes WHERE hash IN ( x'232d97e6126a25b8ec4ae36f65b5e8ae08314e74d06f4fcbc72350a9fd3f5995', x'6ed82640fdd14643660cb4f50f022b5b0dd7a4e1c4ff3f6d430a5ed8e7c82fac' );

SELECT media_id, hash_id FROM duplicate_files, hashes ON (king_hash_id = hash_id ) WHERE hash IN ( x'232d97e6126a25b8ec4ae36f65b5e8ae08314e74d06f4fcbc72350a9fd3f5995', x'6ed82640fdd14643660cb4f50f022b5b0dd7a4e1c4ff3f6d430a5ed8e7c82fac' );

SELECT media_id FROM duplicate_file_members NATURAL JOIN hashes WHERE hash IN ( x'232d97e6126a25b8ec4ae36f65b5e8ae08314e74d06f4fcbc72350a9fd3f5995', x'6ed82640fdd14643660cb4f50f022b5b0dd7a4e1c4ff3f6d430a5ed8e7c82fac' );

(note those media_ids down, and put them in here a couple times):

SELECT * FROM duplicate_file_members WHERE media_id = (media_id here);

.exit

Does that first SELECT statement give you:

(some number)|EAE648A595946D8B

(some other number)|E8EAD2E49595C583

Or does it give very different numbers, or a much bigger list?

Does the duplicate_files, hashes SELECT give two lines, or less?

Does the duplicate_file_members SELECT where you put the media_ids in give you one line per media_id, or more?

>>13205

Hmm, I am not sure what it doesn't like there. I don't know enough about subprocess details. I have not had luck with anything but subprocess.Popen. If you do something like this:

p = subprocess.Popen( ['exiftool.exe', argv[1]], stdin = subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.PIPE, universal_newlines = True, encoding = 'utf-8' )

( stdout, stderr ) = p.communicate()

That may work out better, I am not sure. Maybe doing the PIPE stuff is ok, since I am already doing that to call your process and there is some sys-related std-'chaining' issue going on? I dunno though, because it looks like the .run command just calls Popen anyway deeper into subprocess.

Was that traceback you got hydrus-side written to the client.log in install_dir/db? Does it show the full trace there, so we can see the actual meat of what it was having trouble with?

>>13208

Shit, I forgot that it was ok in a fresh db. Unless something very odd is going on that is prohibiting the next message getting out or being flushed to the log or something, after that 'generating phashes' line should be 'File import job generated xxx phashes'. Since that message is not coming up, it suggests the hang is inside the pure-CPU graphics processing code that generates similar files phashes. This all happens before the db is opened for the actual file import.

Ok, let's see how this new phash generating report mode goes in tomorrow's release.

I tried to import an epub but hydrus somehow turned it into a zip file.

This is the file

https://files.catbox.moe/d6005p.epub

>>13215

2019/07/18 19:42:36: File import job generating phashes
2019/07/18 19:42:36: phash generation: loading image

>>13214

The same thing that I posted was written to the log, nothing more. Alright so I used a python library to get the title and that works just fine. Even the stdout in hydrus will show anything I print from the program. But it seems I cant load my json file.

Nevermind, I think the correct solution to all my problems was to use full filepaths in my program. I.e. "C:\\Users\\*\\My_Documents\\_tiff_shortcuts\\data.json" instead of just data.json. Thanks for the help, the debug subprocess report was useful.

>>13227

Ah, it looks like epubs are zip files, a bit like cbr/cbz:

https://www.lifewire.com/what-is-an-epub-file-2621084

Most ebook formats are not supported yet in hydrus. Text is difficult for a bunch of reasons, and accurate mime (filetype) detection based on file content (which is how hydrus currently determines filetype) is a big one. You can leave the epub in there if you like, and when support eventually comes in I'll add a file maintenance job to recheck all current zips to see if they are really epubs and fix it automatically for you, or you can keep it out so your ebook reader can recognise it in the meantime.

>>13230

Ah, it looks like PIL (Pillow), a backup image library I use, completely spergs out when it tries to load that file! When I force it to load with PIL, I get the same CPU problem. I looked inside PIL, and I don't completely know what's going on, but it looks like the LOAD_TRUNCATED_IMAGES setting, which is putting a fake EOF in the load file call, is not being recognised, so the virtual file being loaded is just a truncated file plus an infinite number of EOFs.

Can you check your options->media->BUGFIX: Load images with PIL setting for me? My guess is this is on, perhaps because you are/were a Linux user at some point? Uncheck it, and you should be good going forward. There was a while that PIL was more stable and reliable than OpenCV, and now I presume the opposite is true.

>>13236

This is now hidden behind help->advanced mode, as a lot of new users were stumbling into it by accident.

>>13237

I am very sorry you are having trouble. Can you talk more about your situation? When you say all your files disappeared, does it still seem like the client thinks you should have them (and the thumbs are not loading), or is it seemingly empty? Do you have 'system:inbox (some files)' or is it like 'system:inbox (0)')? Did it boot with your old session of page tabs, or just a fresh single empty page? Did it give the 'this looks like the first time you are running the client' popup message on boot?

Did the 360 folder get run before you copied it over 359, or was it a completely fresh extract? If you check your install_dir_359/db folder, what are the size and created dates of the four client*.db files in there? If you go deeper into the client_files directory and its subdirectories, do you still have media files in there? Or have you migrated your database/files to another location?

Do you have a backup of your 359 client?

Clicking on the button to change which site to search in a gallery downloader pops an error

'<' not supported between instances of 'GalleryURLGenerator' and 'GalleryURLGenerator'

>>13254

Sweet, that helped! Thank you!

It was indeed on, though I have always been on Windows. But I've been using Hydrus for so long, I might have just turned that on myself who knows when and forgotten about it.

Speaking of which, is there an option I might have set at some point (that I am unable to find now) which disabled Hydrus from playing sound in videos? I feel like it did that at some point, but it hasn't for ages now. Is that a feature? Or am I just too daft to figure it out?

>>13256

It seems like it was a momentary issue. After killing the client and restarting it all my images were present. Before then it was like they weren't even there, but things are fine now.

>>13201

>>13201

No, it's alright.. The client opened up normally after I clicked OK on both those errors and they never appeared again since that time.

I was just curious about the meaning of those errors.

I don't seem to be able to administrate my remote files service at all. I can upload files with the automatically-created administrator account just fine, but attempting to create new accounts for the file service, modify the account types, or ask who uploaded a file causes the client to lock up indefinitely. My first guess was a version mismatch, locally I'm running 361 on windows where my remote server is running 360 on ubuntu 19.04. I downgraded my windows client to 360, then upgraded back to 361 after running into the same issue. I've had no issues administrating the remote tags service running on the same server.

As an aside, the server is on 360 because I was unable to get 361 working on the server, even after reimaging the entire box and dropping a fresh copy of hydrus from the github on it. On 361 the server will run, but doesn't seem to listen on any ports.

Any ideas for either of these?

>>13262

Thank you for this report. I am sorry for the inconvenience. I am 99.7% sure I can fix this for 362.

>>13264

There's no native audio yet. It proved very popular in the last big job poll, so I expect to give it a proper go this year. Within the next 3-5 weeks I hope to have 'has audio' metadata sorted, which will display in the media viewer as a speaker icon or something.

>>13277

Ok, thank you for the update. Please let me know if it happens again and/or you discover more about this.

>>13291

I fixed a related error here in the past week or two–hitting 'forget it' on the 'a client is already running, do you want to wait or forget it?' boot dialog was accidentally clearing out the 'is running' file, SHIT, so a subsequent attempt to run the client was ploughing ahead regardless of any existing instance and throwing this error. This may be something of what you were seeing here.

>>13309

Thank you for this report. I am sorry you are running into this trouble. The service admin stuff has been caught up in some of the new network error handling recently, where if the network engine wants to wait because of a recent error, the old admin service code is now waiting to retry connections silently in the background. And because admin stuff is shit old code, it often waits on the ui thread, so it is locking up the client for x minutes until the connections time out. I suspect this is what you are seeing.

The job of making this workflow nice is bigger than I can quickly do, but I will go through all the admin commands and ensure they cancel snappily on a failed connection.

I did change how services start in v361, I had hoped for the better. I assume you have access to the file repository's server.log? Does it print any errors right after starting about being unable to map ports? I have a job this week to improve how this works further, improving error reporting.

>>13344

I went back and checked the logs from the v361 instance. Now that I'm looking more closely, it actually doesn't print "Starting services…" at any point until after I keyboard interrupt it, no matter how long I wait after starting the server on that particular attempt. Every one of the logs looks basically the same as this one with different timestamps:

2019/07/27 22:22:23: Initialising controller…
2019/07/27 22:22:23: Initialising db…
2019/07/27 22:22:24: Initialising daemons…
2019/07/27 22:22:24: Server is running. Press Ctrl+C to quit.
2019/07/27 22:23:03: Received a keyboard interrupt…
2019/07/27 22:23:03: Shutting down controller…
2019/07/27 22:23:03: Shutting down daemons…
2019/07/27 22:23:03: Shutting down db…
2019/07/27 22:23:04: Starting services…
2019/07/27 22:23:04: Services started
2019/07/27 22:23:04: Stopping services…

>>13345

Thank you, this helps.

>>13343

Yeah, that's possibly what caused it but I don't remember for sure as it only happened once for quite some time now.

Anyway, I'll update to the latest version and I'll make sure to let you know if that ever happens again.

>>13503

>>13524

nevermind sorry i think this is unrelated, there are quite a bit of 0 byte files on my hard disk now. its going down, down to china town, REAL TIME

I'm not certain where exactly it was introduced (probably in the past 3 or so releases), but there's a bit of a regression:

The trash display, when using "system:inbox"-selection displays ALL inbox entries (that is, both those in "trash" AND the ones in "my files"). It used to display just the inbox entries that were trashed (which is the desireable behaviour).

>>13529

Derpi scraping is working here, Though I can't vouch for filters working or not.

HOWEVER, I can vouch that I'm not getting any tags at all for anything downloaded from derpi currently.

The duplicate filter doesn't zoom in/out to the center of the screen, more like lower right. At 2560x1440 resolution.

>>13526

Hey, I am sorry, I am just coming back to this thread after a period of not keeping up correctly with my messages. I have done significant work with other users in these sorts of situations, so please feel free to email me or hit me on discord or just post here on any further issues you are having. I'm happy to work one-on-one.

There is a help file in hydrus under install_dir/db/help my db is broke.txt that will walk you through various hard drive error recovery situations, particularly if you are not familiar with chkdsk and other tools.

>>13551

>>13529

Thank you. I am afraid I am not a user of derpibooru and did not write the new parser, so I am a bit ignorant. I think I saw some conversation somewhere that the new site doesn't actually support the new filterless search stuff? Or that the format for that has changed somehow?

I will hit up the guys on the discord who made this to see if there is an update.

>>13532

Thank you for this report. This should be fixed now.

>>13554

I have noticed something like this in the normal media viewer too. I've been pulling all that code to pieces to get mpv integrated, so I think I messed some mouse reinitialisation coordinates up or something. Thank you for the confirmation report, I will look into it.

>>13559

Unfortunately, I am told the 'no filter' search strategy no longer works, so I think your best bet now is to use Hydrus Companion to copy your web browser's logged-in derpibooru cookies across to Hydrus, and then it will (presumably?) inherit your account's filtering preferences:

https://gitgud.io/prkc/hydrus-companion

>>13551 I just did a test download, and I am afraid I get all tags ok–this sounds stupid, but can you double-check your tag import options? If it is still bad, can you do a derpibooru parser test under network->downloader definitions->manage parsers?

Apologies if this is a duplicate bug report.

Shift select is broken sometimes. Select the first file -> select the last, only last gets selected. Then select the first, only first selected. Select the last again and then finally all the files are selected.

Linux, v380

Not sure whether to put this as a bug or feature request.

Collecting files while under 'my tags' collects them based on all tags (my files + public tag repo).

Apologies if this has already been reported.

Trying to load a .gif/.webm gives the following error on linux:

Non-C locale detected. This is not supported.

Call 'setlocale(LC_NUMERIC, "C");' in your code.

zsh: segmentation fault (core dumped) hydrus-client

going into the settings and switching to the default hydrus viewer solves this issue for me, but I thought it best to mention it.

Linux v381 but I'm pretty sure this has been going on since at least v380 or maybe even earlier

Does nijie work for anyone?

I tried with a new install of the latest version and still get no successful connection.

Knowing if it's just me or not would help.

When I try to add a tag that's the old sibling in a sibling pair, instead of adding the tag I entered in and saying that it will be displayed as the new sibling, it just adds the new sibling, even when I manually select the old sibling in the autocomplete panel, and won't let me add the old sibling at all. This is a huge problem if the sibling pair was ever removed, and I don't know what's causing it.

It happens with both "my tags" and "public tag repository" sibling pairs, and I believe this is a recently introduced bug.

>>13575

Thank you for this report. I have noticed this myself. I can't figure out if it is my code or my mouse driver (I have shift mapped on a Logitech mouse) or Qt in general–I have another Qt program that also does the same thing, again only sometimes. I can't obviously think of any place where I say 'ok, stop accepting shift for a bit', although I may be mistaken.

When this happens again, if you have another program in Qt with a list, can you check that at the same time? Does this only happen for you on thumbnail select, or can it also happen on column-ed lists, like the list of queries in a download page? That is where I see it most, when I am trying to shift-select six completed watchers or similar so I can delete them.

>>13577

Thank you. Unfortunately it is all merged at the moment. I had hoped to be putting some more time into per-service tag presentation and handling options in the past couple of weeks, but Qt boshed my whole schedule. I am not sad at all and am really pleased with Qt, but a bunch of those tag plans are now on the back burner.

Ideally, I want a whole new set of UI for saying which services apply and take precedence in different places (like thumbnail banners and tag siblings, as well). I am not sure when I will get to this.

>>13581

I am sorry for the trouble. This should be fixed in v382 now. It is a non-Windows problem with loading MPV, triggered accidentally for you because your system happens to have MPV access when I did not expect it to.

If 382 still give you trouble, please let me know.

>>13583

I do not know much about the site, but I feel like it has 'locked up' a bit since I last visited it. Do you have to have a login now to see anything? Pixiv went through an extensive round of updates over the past six months, perhaps nijie have as well, and this has broken the downloader in hydrus.

I am afraid I do not see a new downloader either in the shared repository:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/Download%20System/All-in-Ones/Single-Sites

What do you get when you try to download something? No results at all, or a parsing error, or a connection error?

>>13584

Is this in the 'manage tags' dialog? Some time ago I moved some automatic sibling/parent action options to the 'cog' icon button there–is your dialog set up to auto-replace tag siblings on a pend/add?

If it is turned off and still auto-replacing, can you say exactly the series of button-presses you are using to add the tag? Is it an enter press on an arrow up/down selection, or a mouse double-click, or anything else?

>>13587

Were you able to set up the Hydrus Client API for the Companion to talk to? Was it ever able to see the Client API? Under the Companion page here:

https://gitgud.io/prkc/hydrus-companion

There is a section 'connecting the extension to Hydrus', which goes through the steps. You'll have to turn the Client API on and then grant the Companion permission to access your client.

>>13593

You're right. I wasn't aware that there was an option for that. I must've accidentally enabled it sometime and didn't notice. My bad, although I often forget that not all options are in the options menu, and that some are hidden in gears in different parts of Hydrus. However, I recently encountered another bug. This one's been bothering me since I started using Hydrus at around V370, but it happened again just yesterday, and I'm more confident that this bug is legitimate because of how strange it is, but I'm welcome to being proven wrong again. It helps me learn.

After adding a child-tag to "male" or "female" in the local tag service, the tag "gender:male" or "gender:female", respectively, will be added to all files that I have tagged with the child-tag at the time of the relationship's creation. That's what I've gathered after doing a bit of experimenting.

gender:male and gender:female aren't even tags that are in my local tag service at all. They're the new-siblings of "male" and "female" respectively in the PTR, but the PTR shouldn't be interfering with the local tag service like this. This doesn't really seem like something that would be an option either, but after looking around for a little bit just in case (and keeping my eyes out for gears this time) the closest options I've found to what's happening here are "suggest all parents for all services" and "apply all siblings to all services", both of which I've never had enabled.

>>13595

It is possible that setting was reset at some point when a system was rewritten, I am not sure. The whole siblings/parents system is a bit of a mess behind the scenes, so I am sorry for the inconvenience.

For your multi-service parents situation, please uncheck the 'suggest all parents for all services' checkbox under options->tags. At the moment, the only parents options you have are 'each service by itself' or 'all merged together', and this controls it. I notice that 'suggest' is the now wrong word here–it should be 'apply'. I suspect the option got written into a logic overhaul at once point, as it used to always be 'all merged together' all the time.

I very much want to improve the settings for which services provide siblings and parents to other services. I want to give you full control, letting you turn them completely on or off or override or whatever on a per-service-to-service basis. I had planned to do this in November/December, and then unfortunately Qt boshed my big tag plans. I am not sure when I will get to it, but the once-prototype logic here is creaking at the seams and I really would like to overhaul it.

>>13600

That option already is disabled and it always has been. But this isn't exactly working how that option works anyway, because it only applies the parents when I create a new relationship, and only with files that are tagged with the child-tag. Also, it's treating the new-siblings from the PTR as if it's a parent in the local tag service, which is why "male" is getting gender:male added to it, instead of being replaced by it. That's why I'm so confused. After doing a bit more experimenting, I discovered that this appears to happen with all tags, not just "male" and "female", so I'll try to clearly explain what's going on:

The public tag repository has old-sibling tags that are replaced by new-sibling tags. Some of those tags exist in my local tag service as tags that I use and are not part of a sibling pair. For example, I use the tag "japanese language" in my local tag service. In the PTR "japanese language" is an old sibling that is replaced by its new sibling "language:japanese" but in my local tag service, "language:japanese" doesn't exist at all, and "japanese language" is not an old-sibling of it. That's the tag I use. a few days ago, I decided to make this tag the parent tag of "japanese text" (a tag that I've been using already) in my local tag service. When I did that, the bug took effect.

Every file that I had tagged with "japanese text" (the new child-tag) gets the tag "language:japanese" (the new-sibling of its parent tag in the PTR but NOT in my local tag service) added to it.

In other words, when I create a new parent-child relationship in my local tag service, everything that was tagged with the child-tag when I created that relationship gets tagged with the parent-tag's new-sibling in the PTR if it has one. the "suggest all parents for all services" option is not enabled when this happens, and I have never had this option enabled.

The bug treats the new-sibling of a tag in the PTR as if it is the parent the tag in my local tag service, but it only applies them when I create a new relationship between this tag, and another, and it only adds this tag from the PTR to files that I have tagged with the child-tag (not just the parent tag) in my local tag service at the time of the relationship's creation.

But again. Both cross-service relationship options are disabled when this happens, so those options should not be the cause of what's happening. I don't think it would make sense for that to be the cause anyway, because the bug is treating new-sibling tags as if they are parent-tags, which I don't think there's any option for in Hydrus.

Sorry if this still isn't a clear enough explanation. I wish I could just diagnose the problem myself, but I don't know Python, and I've looked at every option I could find to make sure that there isn't some weird "make siblings in remote tag services temporarily pretend to be parents in local tag services when you make a new parent-child relationship in a local tag service" option somewhere that I missed, but I couldn't find any, so I'm pretty sure that it's actually a bug this time.

>>13601

I just remembered that I should include steps to reproduce, since they'll help you see what I'm talking about.

1. open a client that's connected the the Public tag Repository and also has a local tag service as well. Make sure that "suggest all parents for all services" is disabled, just in case.

2. In the local tag service (I think by default it's called "my tags") add the tag "japanese text" to a file

3. Go to "manage tag parents" and in the local tag service, make "japanese language" a new parent tag of "japanese text"

4. Go back to the file you added "japanese text" to and you'll see the results of the bug in that file's tags:

This file will have the tag "japanese language" added to it, as it should, because you just made this tag a parent of "japanese text". However it will also add the tag "language:japanese" to the file. This tag is not supposed to be added. You didn't enter it manually, and you didn't make "language:japanese" a parent of "japanese text". "language:japanese" is the new-sibling of "japanese language" in the PTR, but this isn't the public tag repository, and "suggest all parents for all services" is disabled, so there SHOULDN'T be any interference from there anyway, but there is. Even if that option was enabled, "language:japanese" is the new-sibling of "japanese language" in the PTR, NOT the parent of it, like it's acting like here.

>>13589

>I have another Qt program that also does the same thing, again only sometimes.

It's probably QT then, heh. I tried searching through their bug tracker but there's so many selection bugs that it would take an hour to find a report.

https://bugreports.qt.io/browse/QDS-526?jql=text%20~%20%22shift%20select%22

>>13592

I'm sorry for not answering earlier, this has been a very busy week.

I get a connection error when trying to download anything.

It used to work flawlessly up to 2 week ago at least. (I mostly check during week-end).

BUG:

Open a video in media viewer. Zoom out (make it smaller) to 20%, zoom in to 100% again, repeat this many times. Video will move lower on screen every time. It does not center properly to the middle of the screen. Doesn't happen with images.

BUG:

The option "Remove files from view when they are sent to the trash" does not work on downloader pages or import pages.

>>11542

two write holes so far resulting in partially corrupted DBs on linux. haven't figured out the issue yet, my intuition tells me btrfs. I'm in the process of migrating to ext4 (not sure why I was on btrfs, heh, maybe for reflink support). I got a checksum error the first time, then I used quicksync for a fresh db and got another checksum problem while updating the public tag repo. I know how to recover an sql db so, w/e. nvme drive that is only a 3-4 months old so likely not a bad drive, and a pci-e card so likely not a bad controller or cable.

>>13659

uh oh. don't want to speak too soon, but I switched over to ext4. ran into the db corruption bug again while syncing the repo. happened on v383 and v384. hasn't happened yet on v382

>>13659

>>13660

Ignore these, the quicksync full database has an b-tree error in client.master.db. going to rebuild a new db from scratch and see if my issue really was a btrfs issue (I had multiple power outages which I'm pretty sure caused the write hole)

>>13602

>>13601

Thank you, I am sorry for the late reply. I have copied all this information down and will see if I can fix this in the near future.

>>13604

Damn. Thank you, and let me know if you discover anything new here. If it is triggered by an earlier ctrl+click or something, maybe we can figure it out.

>>13613

Thank you for this report. I will look into this bug. Does this fix itself after a client restart?

>>13614

Thank you for this, I am sorry for the late reply. Could this be a firewall issue? Do you have a third-party firewall that is set to automatically block any non-whitelisted executable? It should be client.exe in the hydrus install dir trying to host here, if you need to add permissions.

A good way to test the visibility of the client api is to open, in your case, http://127.0.0.1:45869 in your web browser. This should present a simple welcome page letting you know it is running ok, or perhaps a more helpful error page.

Also, you may want to check your install_dir/db/client…log file and scroll to the timestamps where you did tests. There may be some logged info, or again perhaps some errors about trying to bind the port.

If you only mean to connect to the client api from the same computer, I recommend you uncheck 'allow non-local connections' so other computers on your network cannot see it.

>>13621

No worries, I am sorry to say I am in the same situation, ha ha.

If you check the gallery/file icon buttons (they launch a list of URLs attempted and the results) on your downloader that gets the error, can you find the URL row that errored? It is probably top of the list. Right-click->copy notes to copy all the error info, and can you then paste it here or email it to me, so I can have a look?

If this is a legit network connection error, then I hesitantly guess this may be something other than hydrus–could you have a firewall anywhere stopping access? I assume it works in your browser. Maybe the website has a rule to disconnect a hydrus client, but usually you do not get an error in that case, but something more like a successful 503.