/hydrus/ - Hydrus Network

When downloading a deviantart gallery, hydrus throws out an exception if it fails too many files. This is a pain in the ass if the files are legit, but cannot be scraped, such as

http://white-angel-ariah.deviantart.com/art/Princess-of-the-Silk-Road-144206154

http://white-angel-ariah.deviantart.com/art/Lullaby-for-Lost-Souls-145623846

http://white-angel-ariah.deviantart.com/art/Song-of-Temadon-145912023

These count as failed images for hydrus, but they're just text. It makes no sense for it to fail them and count them towards the max fails. It should just skip them.

Not sure if this is something that i should email the dev about, but.

DataMissing
Tag error in database
Traceback (most recent call last):
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.HydrusDB", line 467, in _ProcessJob
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientDB", line 10139, in _Write
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientDB", line 3437, in _ExportToTagArchive
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientDB", line 5794, in _GetTags
DataMissing: Tag error in database

I've been trying to save all my tags in .db form every so often, because the last time hydrus borked up for me all the files were still there, and having a 40MB backup is easier then having a 450GB one. Recently updated to the recentest hydrus, 244a, and this happened alongside the backup failing. Possibly might be related to an earlier issue where hydrus panicked over something called a Zero-length Tag which i left well alone and didn't try to go back to fix.

Hi. Not shure if it's bug or not, but if you select same directory for file storage in options then Hydrus starts deleting files.

For example, by default Hydrus has path like this "C:\HYDRUS~1\db\client_files". But if you add same directory like this "C:\Hydrus Network\db\client_files" and press "rebalance file storage" then Hydrus start deleting files.

pixiv not working on Linux, SSL error

http://pastebin.com/v5CLh4aL

workaround:

set PYTHONHTTPSVERIFY=0 env. var.

>>5161

Hey dev, there's a race condition in case you try to manually import files from a folder that you set as monitoring. If the program decides to import from the folder while you're already importing those files manually, you get

IOError
[Errno 2] No such file or directory: u'/home/ayylmao/hydrus/general/optimized_2017.02.18.36.071674895.png'
Traceback (most recent call last):
  File "/home/ayylmao/hydrus/include/HydrusPaths.py", line 447, in MirrorFile
    shutil.copy2( source, dest )
  File "/usr/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: u'/home/ayylmao/hydrus/import/optimized_2017.02.18.36.071674895.png'

It's probably wise to make the program suspend folder monitoring for as long as the manual import screen is active, kinda like how subscriptions don't work if the subscription folder is open.

>>5174

>/home/ayylmao/hydrus/general

>/home/ayylmao/hydrus/import

These are the same directory btw, I just derped while trying to hide my username.

This is a weird little bug. I'm on linux btw.

>pick a page full of files

>right click -> select -> pick a category that does not have the first file of the page included in it

>press enter

>the first file of the page is selected instead of the first file of the selection

I accidentally added an empty namespaced tag to a file in the tag manager window, it was "page:". Now I get this error whenever I try to open that file's tag manager window again, and the window doesn't come up.

DBException
SizeException: Received a zero-length tag!
  File "include\ClientGUIMenus.py", line 106, in <lambda>
    l_callable = lambda event: callable( *args, **kwargs )
  File "include\ClientGUIMedia.py", line 865, in _ManageTags
    panel = ClientGUIScrolledPanelsManagement.ManageTagsPanel( dlg, self._file_service_key, self._selected_media )
  File "include\ClientGUIScrolledPanelsManagement.py", line 2827, in __init__
    page = self._Panel( self._tag_repositories, self._file_service_key, service.GetServiceKey(), self._current_media, self._immediate_commit, canvas_key = self._canvas_key )
  File "include\ClientGUIScrolledPanelsManagement.py", line 3069, in __init__
    self._suggested_tags = ClientGUITagSuggestions.SuggestedTagsPanel( self, self._tag_service_key, self._media, self.AddTags, canvas_key = self._canvas_key )
  File "include\ClientGUITagSuggestions.py", line 434, in __init__
    related_tags = RelatedTagsPanel( panel_parent, service_key, media, activate_callable, canvas_key = self._canvas_key )
  File "include\ClientGUITagSuggestions.py", line 207, in __init__
    self._QuickSuggestedRelatedTags()
  File "include\ClientGUITagSuggestions.py", line 238, in _QuickSuggestedRelatedTags
    self._FetchRelatedTags( max_time_to_take )
  File "include\ClientGUITagSuggestions.py", line 227, in _FetchRelatedTags
    predicates = HydrusGlobals.client_controller.Read( 'related_tags', self._service_key, hash, search_tags, max_results, max_time_to_take )
  File "include\HydrusController.py", line 260, in Read
    def Read( self, action, *args, **kwargs ): return self._Read( action, *args, **kwargs )
  File "include\HydrusController.py", line 93, in _Read
    result = self._db.Read( action, HC.HIGH_PRIORITY, *args, **kwargs )
  File "include\HydrusDB.py", line 696, in Read
    return job.GetResult()
  File "include\HydrusData.py", line 1998, in GetResult
    raise e

Database Traceback (most recent call last):
  File "include\HydrusDB.py", line 466, in _ProcessJob
    if job_type in ( 'read', 'read_write' ): result = self._Read( action, *args, **kwargs )
  File "include\ClientDB.py", line 7453, in _Read
    elif action == 'related_tags': result = self._GetRelatedTags( *args, **kwargs )
  File "include\ClientDB.py", line 5376, in _GetRelatedTags
    tag_ids = [ self._GetTagId( tag ) for tag in search_tags ]
  File "include\ClientDB.py", line 5833, in _GetTagId
    HydrusTags.CheckTagNotEmpty( tag )
  File "include\HydrusTags.py", line 146, in CheckTagNotEmpty
    raise HydrusExceptions.SizeException( 'Received a zero-length tag!' )
SizeException: Received a zero-length tag!

2017/03/02 09:35:55: Traceback (most recent call last):

File "include\ClientController.py", line 1110, in THREADBootEverything

self.InitModel()

File "include\ClientController.py", line 525, in InitModel

HydrusController.HydrusController.InitModel( self )

File "include\HydrusController.py", line 203, in InitModel

self._db = self._InitDB()

File "include\ClientController.py", line 64, in _InitDB

return ClientDB.DB( self, self._db_dir, 'client', no_wal = self._no_wal )

File "include\ClientDB.py", line 161, in init

HydrusDB.HydrusDB.init( self, controller, db_dir, db_name, no_wal = no_wal )

File "include\HydrusDB.py", line 244, in init

raise e

Exception: Updating the client db to version 244 caused this error:

Traceback (most recent call last):

File "include\HydrusDB.py", line 223, in init

self._UpdateDB( version )

File "include\ClientDB.py", line 8965, in _UpdateDB

self._c.executemany( 'INSERT INTO tag_sibling_petitions ( service_id, bad_tag_id, good_tag_id, status, reason_id ) VALUES ( ?, ?, ?, ?, ? );', new_inserts )

IntegrityError: UNIQUE constraint failed: tag_sibling_petitions.service_id, tag_sibling_petitions.bad_tag_id, tag_sibling_petitions.status

Following this, the program crash.

I can restore to 243 and work.

For context, I have at least 143k uncommitted tag, due to some unvetted filename.

I'm not smart enough to understand what these line of code mean either, sorry.

>>5254

Read the changelog for 245.

>- leading and trailing spaces are now removed from both the namespace and subtag components of namespaced tags, meaning 'title: blah' will be collapsed to 'title:blah'. tag repositories will clean their existing tags on update

>>5254

Also >that title bar font

Jesus fucking christ dude.

>>5257

Hey, i like Caslon. It's the Monkey Island font and it brings back happy childhood memories.

Also, this update is breaking my workflow. I import a large amount of tags from the public repository wholesale, and have difficulty telling which tags are mine and which tags are copied from other sources. At least add an option to disable or enable leading and trailing spaces as needed somewhere discrete.

>>5258

Quick update. If you had a tag with a leading space in hydrus 244a (And presumably earlier, but i haven't checked), updated to hydrus 245, and tried to remove or delete your old tag from a file, Hydrus will remove it from your tag editing window but not from the database. A quick refresh of the page confirms that all of the tags that have leading spaces aren't removed at all.

Since over half of my tags have leading spaces that i can't touch, get rid of, or change in any way, Hydrus is unusable for me right now.

>>5269

>Did you add this tag in v245 or a previous version?

It was v244. I tried it again on v245 and same thing happens. Now I've got two images that won't let me edit tags :P

>>5269

>>5254

>>5256

>>5258

>>5263

Following up on all of these, just made the switch from 245 to 246, so this will be first impressions only.

First off, i prepared for the switch. I've mentioned before my reasons for adding leading spaces to certain tags was because i like to both save files from 8chan with their original filenames and sync with the public tag repo, and i like having a easy, at a glance way of telling weather or not any tag was created by me or by some anon on a webm thread being clever.

With that in mind, before i made the switch, i moved across every single file that has only tags with leading spaces into my archive folder, and anything that doesn't, i kept in my inbox. Took me a while to do, actually. With 245, it was hit or miss weather or not tags with a leading space would autocomplete or not, so i had do to a lot of clever tricks by using - searching to narrow down everything that works before moving it all across.

With that done, and with all 80,000 untainted files moved across, i made the switch. I can say without fear of exageration that literally every single file in my database now has an "invalid tag:" subtag on it, because two of the ones that didn't carry across was "medium: image" and "medium: video", which between the two comprised every file on my database. But Autocomplete now works like it used to, I can save my tags to external DBs again, even submit them all back to the public tag repo when i thought i couldn't since v244, and my dog came back to life.

Good job, Hydrus dev.

>>5289

>>5289

>was "medium: image" and "medium: video"

So it was you who created these…

Why, pray tell, when we can sort by mime already?

Those could work but only as encompassing tag type if we had tag group. (as in: 'Franchise:tag' silently searchable for every 'anime:tag', 'visual novel:tag', 'video game:tag' or 'series:tag' we add, because they are part of the franchise tag group.)

I'm may "read" judgemental but I am legitimately curious.

>>5292

It's part of my workflow. When i import files, i always import a filename, then manually go through the imports and manually edit out anything with a wierd name, or one of Codemonkey's hexadecimals that don't actually mean anything. Then, and only then, do i add medium tags. I don't really care about the tag itself, because like you said, we have mime searches. But it helps me keep track of imported files, gives me a second chance to look through all of them, remove stuff that i don't like, add some tags that could fit on an individual basis. The medium: image tag itself doesn't matter. It's just a sign that this image has value to me and shouldn't be deleted.

Oh, and i forgot another thing i do; Anything that's a gif has to go through a pretty stringent set of requirements to stay a gif. If it doesn't move, it would work better as a jpg. If it's huge, it works better as a webm without sound. Since i'm always adding gif tags to everything, it's easy to check offhandedly if they meet my requirements or not.

If you try to delete many files from the trash immediately, like hundreds, you get an error about too many somethings to unpack.

Reminder that if you have a weird tagging thing you do that's going to be a headache for people n the repo you can always just tag it in local tag db instead. In fact that's what you should probably always do, and then submit tags to repo only if you're sure they're really useful.

Hydrus refuses to start on linux. Running under a terminal spits out this.

Traceback (most recent call last):
  File "<string>", line 20, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientController", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientCaches", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientDefaults", line 2, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientData", line 4, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientFiles", line 7, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.HydrusFileHandling", line 15, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.HydrusVideoHandling", line 7, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/numpy", line 180, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/numpy.add_newdocs", line 13, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/numpy.lib", line 18, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/numpy.lib.polynomial", line 20, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/numpy.linalg", line 51, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/numpy.linalg.linalg", line 29, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 420, in load_module
ImportError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.23' not found (required by /home/patxi/Downloads/hydrus network/libquadmath.so.0)

>>5336

Do you have glibc installed?

>>5362

It turns out this guy got it right. Linux Mint 17.3 ships with GlibC 2.19, where Hydrus requires 2.23. And updating it is a massive pain, so i just made the switch to 18. Which is annoying, cause SystemD and muh /tech/ cred. Issue resolved, i guess.

>>5371

Serves you right for

1) using linux but not with a rolling release distro

2) not updating as soon as possible

ZeroDivisionError
float division by zero
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientGUICanvas", line 862, in EventPaint
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientGUICanvas", line 771, in _Redraw
  File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientGUICanvas", line 686, in _GetXFromFrameIndex

Hydrus is trying to divide by zero.

Please stop doing that, Hydrus, lest you succeed.

Got this

2017/03/28 15:56:02: C:\HYDRUS~1\PIL\Image.py:2244: DecompressionBombWarning: Image size (161475000 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.

from trying to import the full size png of https://yande.re/post/show/340865

Would be nice if Hydrus could just throw and error and give up instead of taking up all ram and locking up the system.

This is maybe not technically a bug as much as a ux concern, but "delete on successful import" apparently means import everything and THEN delete the successes. Turned into a bit of a nasty surprise when I imported a 250GB folder from the same drive and had to rush to move other things off that drive temporarily to give free space for it to finish.

Dragging an image from the client to Discord does not seem to work.

>>5416

That's a discord bug, not hydrus'.

>>5453

The importer is very slow, and behaves strange when asking it to "import tags from nearby .txt file", and that file is actually an image file.

Also related feature request: Either allow {hash}.txt as tag file for a {hash}.{extension} file, or add a reminder that "paths include the file extensions", to the explanation of the feature.

Either of the above would have saved me hours today.

I don't know if it's really important but when I start hydrus it always says in terminal:

(client:$PID): Gdk-CRITICAL **: IA__gdk_window_get_origin: assertion 'GDK_IS_WINDOW (window)' failed

where $PID is the PID.

>>5536

Thank you for your answer. I don't know if it is any help but a quick search gave this page: https://bugzilla.mozilla.org/show_bug.cgi?id=539138

I don't know anything about this stuff but it might help? Also the error doesn't show up in any log in /db.

The danbooru downloader doesn't work right for me, when I try to download an image that has a larger original image I want the larger original image but I get the smaller image. Is there any way to get around this?

>>5553

Yeah it's been like that for a while now. You'll have to wait for the downloading engine, which will come after the dev finishes the duplicate filter.

I was letting hydrus(253) update and it threw a traceback when an update failed to import:

DBException
DataMissing: Service file hash map error in database
Traceback (most recent call last):
  File "/home/username/git/hydrus/include/ClientServices.py", line 1282, in SyncProcessUpdates
    ( did_some_work, did_everything ) = HydrusGlobals.client_controller.WriteSynchronous( 'process_repository', self._service_key, only_when_idle, stop_time )
  File "/home/username/git/hydrus/include/HydrusController.py", line 386, in WriteSynchronous
    return self._Write( action, HC.LOW_PRIORITY, True, *args, **kwargs )
  File "/home/username/git/hydrus/include/HydrusController.py", line 117, in _Write
    result = self._db.Write( action, priority, synchronous, *args, **kwargs )
  File "/home/username/git/hydrus/include/HydrusDB.py", line 824, in Write
    if synchronous: return job.GetResult()
  File "/home/username/git/hydrus/include/HydrusData.py", line 1662, in GetResult
    raise e
DBException: DataMissing: Service file hash map error in database
Database Traceback (most recent call last):
  File "/home/username/git/hydrus/include/HydrusDB.py", line 517, in _ProcessJob
    elif job_type in ( 'write' ): result = self._Write( action, *args, **kwargs )
  File "/home/username/git/hydrus/include/ClientDB.py", line 9396, in _Write
    elif action == 'process_repository': result = self._ProcessRepositoryUpdates( *args, **kwargs )
  File "/home/username/git/hydrus/include/ClientDB.py", line 7211, in _ProcessRepositoryUpdates
    did_whole_update = self._ProcessRepositoryContentUpdate( job_key, service_id, content_update )
  File "/home/username/git/hydrus/include/ClientDB.py", line 6222, in _ProcessRepositoryContentUpdate
    hash_ids = self._CacheRepositoryNormaliseServiceHashIds( service_id, service_hash_ids )
  File "/home/username/git/hydrus/include/ClientDB.py", line 677, in _CacheRepositoryNormaliseServiceHashIds
    raise HydrusExceptions.DataMissing( 'Service file hash map error in database' )
DataMissing: Service file hash map error in database


Database Traceback (most recent call last):
  File "/home/username/git/hydrus/include/HydrusDB.py", line 517, in _ProcessJob
    elif job_type in ( 'write' ): result = self._Write( action, *args, **kwargs )
  File "/home/username/git/hydrus/include/ClientDB.py", line 9396, in _Write
    elif action == 'process_repository': result = self._ProcessRepositoryUpdates( *args, **kwargs )
  File "/home/username/git/hydrus/include/ClientDB.py", line 7211, in _ProcessRepositoryUpdates
    did_whole_update = self._ProcessRepositoryContentUpdate( job_key, service_id, content_update )
  File "/home/username/git/hydrus/include/ClientDB.py", line 6222, in _ProcessRepositoryContentUpdate
    hash_ids = self._CacheRepositoryNormaliseServiceHashIds( service_id, service_hash_ids )
  File "/home/username/git/hydrus/include/ClientDB.py", line 677, in _CacheRepositoryNormaliseServiceHashIds
    raise HydrusExceptions.DataMissing( 'Service file hash map error in database' )
DataMissing: Service file hash map error in database

So the ptr sync hasn't been showing when it's running. So then I try to do something and it freezes up. The problem here is that it freezes up EVERYTHING, I can't even ctrl+alt+del. Log doesn't show anything unusual, just that it was processing updates at roughly the same time I was trying to do a search. It's happened at least 3 times now where it freezing completely locked down my computer until whatever was going on was finished.

>>5635

Just normal idle maint. On now general browsing within 2 minutes, no mouse movement within 10, and no CPU usage above 50%. Don't remember changing those.

The system responds normally if I don't touch anything in the client, but if I try to do a search or something (while unaware that it's processing), everything freezes up save for my cursor. Ctrl+Alt+Del does nothing until the processing has finished for that moment. Process now button is grayed out right now.

Actually thinking about it, it's happened almost any time the DB is locked. I was going to test it with the duplicate search, but it's already got all the information. And if that does cause it to lock up (iirc only the client locked up when processing duplicates and trying to click on anything) I don't really feel like losing my computer for over an hour while it runs.

>>5637

So memory usage isn't always going back down properly, I'd been doing stuff all day and ended up with only 1 tab, searching the inbox for untagged files, searching my files and all tags. Only 200 files and memory usage was nearing 3GB. Nothing else was happening at the time, and trying to delete things froze the client up for just under a minute each time. Restarted the client and did the same search, memory usage is fine now. I had been doing huge searches earlier, fixing PTR stuff though so almost no thumbnails were being loaded at all.

Recently gelbooru imports / subscriptions stopped working for me.

Import error:

File not found.
… (Copy note to see full error)
Traceback (most recent call last):
  File "include\ClientImporting.py", line 209, in _WorkOnFiles
    downloaded_tags = gallery.GetFileAndTags( temp_path, url, report_hooks = [ self._file_download_hook ] )
  File "include\ClientDownloading.py", line 1027, in GetFileAndTags
    ( file_url, tags ) = self._GetFileURLAndTags( url, report_hooks = report_hooks )
  File "include\ClientDownloading.py", line 1013, in _GetFileURLAndTags
    html = self._FetchData( url, report_hooks = report_hooks )
  File "include\ClientDownloading.py", line 677, in _FetchData
    return HydrusGlobals.client_controller.DoHTTP( HC.GET, url, request_headers = request_headers, report_hooks = report_hooks, temp_path = temp_path )
  File "include\ClientController.py", line 386, in DoHTTP
    def DoHTTP( self, *args, **kwargs ): return self._http.Request( *args, **kwargs )
  File "include\ClientNetworking.py", line 357, in Request
    ( response, size_of_response, response_headers, cookies ) = self._DoRequest( method, location, path, query, request_headers, body, follow_redirects = follow_redirects, report_hooks = report_hooks, temp_path = temp_path, hydrus_network = hydrus_network )
  File "include\ClientNetworking.py", line 323, in _DoRequest
    return self._DoRequest( new_method, new_location, new_path, new_query, request_headers, body, follow_redirects = follow_redirects, report_hooks = report_hooks, temp_path = temp_path, num_redirects_permitted = num_redirects_permitted - 1 )
  File "include\ClientNetworking.py", line 323, in _DoRequest
    return self._DoRequest( new_method, new_location, new_path, new_query, request_headers, body, follow_redirects = follow_redirects, report_hooks = report_hooks, temp_path = temp_path, num_redirects_permitted = num_redirects_permitted - 1 )
  File "include\ClientNetworking.py", line 283, in _DoRequest
    ( parsed_response, redirect_info, size_of_response, response_headers, cookies ) = connection.Request( method, path_and_query, request_headers, body, report_hooks = report_hooks, temp_path = temp_path )
  File "include\ClientNetworking.py", line 925, in Request
    return self._DealWithResponse( method, response, parsed_response, size_of_response )
  File "include\ClientNetworking.py", line 494, in _DealWithResponse
    elif response.status == 404: raise HydrusExceptions.NotFoundException( parsed_response )
NotFoundException: File not found.

Maybe they changed page layout, or something like that. I hoped the update would fix that (having updated for a month) but nope.

>>5661

It probably has to do with how all images are blocked if you're running an ad blocker.

>>5664

>go to gelbooru

>asks me to disable adblocker

>do so

>still don't see ads because of pi-hole

Gelbooru will never, EVER get my money.

>>5661

Same, gelbooru's broken here too. Fucked up half my subscriptions.

gelbooru's also fucked for me

something changed about gelbooru that broke it for hydrus, unquestionably.

e621 works fine so it isn't boorus in general

>>5676

>Honestly I'd recommend just not scraping from gelbooru (pretty much all of their content is mirrored from danbooru anyway)

Danbooru has loli behind a paywall tho. And Gelbooru is by far the most common result on iqdb.

>>5676

Problem is, most of us are fucked now because gelbooru was the only one of the big three (Danbooru, Sankaku) that worked with hydrus.

Hey guys. The recent breaking of Gelbooru has given me reason to update Hydrus for the first time in ages (to version 254). Unfortunately, it now no longer starts. I updated by simply extracting the old files in the same directory as the new, using the "replace all" option at the prompt.

System:

Linux Mint Cinnamon 17.3 64-bit

Some sort of Skylake i5 I forgot

12GB RAM

When running it from the console, I got this feedback:

Traceback (most recent call last):

File "<string>", line 20, in <module>

File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module

File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientController", line 1, in <module>

File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module

File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientCaches", line 1, in <module>

File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module

File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientDefaults", line 1, in <module>

File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module

File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientConstants", line 3, in <module>

File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module

File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/wx", line 49, in <module>

File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module

File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/wx._core", line 4, in <module>

ImportError: No module named _core_

Apt policy tells me that the "python" package is installed (providing version 2.7.5) and the "python3" package is installed (providing version 3.4.0).

>>5705

It's pretty late my time, I'll check for replies tomorrow

>>5705

I just tried 255, same issue. console output:

Traceback (most recent call last):

File "<string>", line 20, in <module>

File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module

File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientController", line 1, in <module>

File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module

File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientCaches", line 1, in <module>

File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module

File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientDefaults", line 1, in <module>

File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module

File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientConstants", line 3, in <module>

File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module

File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/wx", line 49, in <module>

File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module

File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/wx._core", line 4, in <module>

File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 420, in load_module

ImportError: /home/cfalcon/.hydrus/hydrus network/libX11.so.6: undefined symbol: xcb_poll_for_reply64

>>5734

What version were you updating from?

>>5119

It won't be a bug if it could be a feature!

See >>5737 and >>5738

>>5129

is is safe to assume that 1920x1080 and 1366x768 are the top screen resolutions.

Judging that most font require at most 16x16 (Latin and other western alphabet) or 32x32 (Logographic languages) per character,

how much space does it take to display all tags for one image?

>>5736

It was pretty damn ancient. I checked the version just before I updated it and it was 170-something.

>>5743

Oh, here:

>>5362

My first guess was a problem with the whole v245 thing.

>>5743

I am not the dev.

Okay, do you have a backup?

First thing to do is download the version you were on before and extract it over your current install. Then, launch it and see if it works.

If it does, upgrade in steps of 5 versions difference. Launch the client after every upgrade. If it needs to do any kind of maintenance, let it do that maintenance, quit the client, and launch the client again. If it tells you you need an older version (eg you're upgrading to 234 and it tells you it needs v234), do that. Do this till you arrive at v255.

If it does not, then IF AND ONLY IF you have a BACKUP, remove your install, install the version you used to have, and use it to restore the backup. But first check if that version is capable of importing a backup of course.

In the gap between 170 and 255, hydrus has undergone several big changes (I don't know which) that cause it to handle everything differently, I remember the whole way files where saved being changed, so even if you could have launched the client it might not recognize your database.

>>5635

I probably either SIGTERM'd or SIGKILL'd it one of the times it appeared to be hanging when I tried to exit, I'll have to make sure to not do that.

I'm running Void Linux, which is a rolling release distro like Arch but uses LibreSSL instead of OpenSSL.

iirc v255 started downloading updates from the PTR then gave me the message that it had nuked the local state for it and I should restart hydrus while it was downloading 5 updates (as opposed to the hundreds i would get if it were starting from scratch.) After restarting hydrus it knows the PTR exists, but doesn't have any tag mappings from it.

>>5745

Thanks, I'll try to get source install to behave. It doesn't look like it's going to be easy, though - I know 0 python and some pip modules are failing with compile errors.

Mint is based on Ubuntu (my version of Mint is specifically based on 14.04, and I can't update to Mint 18 because the LMDT FUBARed the GCC packages I absolutely need to do my work on), which is why the previous binary worked but maybe we're running different enough versions that the new one still fucked up.

>>5746

Thanks, I'll try that if the new Hydrus doesn't read my DB correctly. First I'll need to get it started up though, it still doesn't work even when I extract the tarball into a new folder.

>>5676

>Honestly I'd recommend just not scraping from gelbooru (pretty much all of their content is mirrored from danbooru anyway)

Problem is, hydrus does not scrape full-size images from danbooru properly.

>>5771

use tbib?

Alright, this is on v 255, on arch linux. Basically, I can't connect to hydrus tag server. Traceback:

NetworkException
Could not connect to hydrus.no-ip.org:
Traceback (most recent call last):
  File "/opt/hydrus/include/HydrusThreading.py", line 276, in run
    callable( *args, **kwargs )
  File "/opt/hydrus/include/ClientGUI.py", line 2788, in _THREADUploadPending
    service.Request( HC.POST, 'update', { 'client_to_server_update' : client_to_server_update } )
  File "/opt/hydrus/include/ClientServices.py", line 722, in Request
    ( response, size_of_response, response_headers, cookies ) = HG.client_controller.DoHTTP( method, url, request_headers, body, report_hooks = report_hooks, temp_path = temp_path, hydrus_network = True )
  File "/opt/hydrus/include/ClientController.py", line 388, in DoHTTP
    def DoHTTP( self, *args, **kwargs ): return self._http.Request( *args, **kwargs )
  File "/opt/hydrus/include/ClientNetworking.py", line 357, in Request
    ( response, size_of_response, response_headers, cookies ) = self._DoRequest( method, location, path, query, request_headers, body, follow_redirects = follow_redirects, report_hooks = report_hooks, temp_path = temp_path, hydrus_network = hydrus_network )
  File "/opt/hydrus/include/ClientNetworking.py", line 268, in _DoRequest
    connection = self._GetConnection( location, hydrus_network )
  File "/opt/hydrus/include/ClientNetworking.py", line 340, in _GetConnection
    connection = HTTPConnection( location, hydrus_network )
  File "/opt/hydrus/include/ClientNetworking.py", line 422, in __init__
    self._RefreshConnection()
  File "/opt/hydrus/include/ClientNetworking.py", line 885, in _RefreshConnection
    raise HydrusExceptions.NetworkException( text )
NetworkException: Could not connect to hydrus.no-ip.org:

('_ssl.c:645: The handshake operation timed out',)


('_ssl.c:645: The handshake operation timed out',)

>>5781

Same shit here.

>>5781

Anon should update/fix their ssl probably

the dns works fine, doesn't return ping but I don't expect it to

>>5786

Hmm. So this could be related to the new openssl update couple weeks back?

Hydrus give no problem after that, so I thought nothing is wrong. Lemme try changing the environment variable…

Okay, so I run hydrus with this and still got the problem: LD_LIBRARY_PATH=/usr/lib/openssl-1.0-compat/ hydrus-client

Anyway, traceback. I think it's similar?

NetworkException
Could not connect to hydrus.no-ip.org:
Traceback (most recent call last):
  File "/opt/hydrus/include/HydrusThreading.py", line 276, in run
    callable( *args, **kwargs )
  File "/opt/hydrus/include/ClientGUI.py", line 2788, in _THREADUploadPending
    service.Request( HC.POST, 'update', { 'client_to_server_update' : client_to_server_update } )
  File "/opt/hydrus/include/ClientServices.py", line 722, in Request
    ( response, size_of_response, response_headers, cookies ) = HG.client_controller.DoHTTP( method, url, request_headers, body, report_hooks = report_hooks, temp_path = temp_path, hydrus_network = True )
  File "/opt/hydrus/include/ClientController.py", line 388, in DoHTTP
    def DoHTTP( self, *args, **kwargs ): return self._http.Request( *args, **kwargs )
  File "/opt/hydrus/include/ClientNetworking.py", line 357, in Request
    ( response, size_of_response, response_headers, cookies ) = self._DoRequest( method, location, path, query, request_headers, body, follow_redirects = follow_redirects, report_hooks = report_hooks, temp_path = temp_path, hydrus_network = hydrus_network )
  File "/opt/hydrus/include/ClientNetworking.py", line 268, in _DoRequest
    connection = self._GetConnection( location, hydrus_network )
  File "/opt/hydrus/include/ClientNetworking.py", line 340, in _GetConnection
    connection = HTTPConnection( location, hydrus_network )
  File "/opt/hydrus/include/ClientNetworking.py", line 422, in __init__
    self._RefreshConnection()
  File "/opt/hydrus/include/ClientNetworking.py", line 885, in _RefreshConnection
    raise HydrusExceptions.NetworkException( text )
NetworkException: Could not connect to hydrus.no-ip.org:

('_ssl.c:645: The handshake operation timed out',)


('_ssl.c:645: The handshake operation timed out',)

>>5808

Network issue is indeed fixed now. Thanks, dev!

>>5756

>>5808

After I let hydrus (v255) get the updates it threw a traceback(again) immediately when it started processing, manually deleting the repo and re-adding it appears to fix it.

traceback:

DBException
DataMissing: Service tag map error in database
Traceback (most recent call last):
  File "/home/username/git/hydrus/include/ClientServices.py", line 1282, in SyncProcessUpdates
    ( did_some_work, did_everything ) = HG.client_controller.WriteSynchronous( 'process_repository', self._service_key, only_when_idle, stop_time )
  File "/home/username/git/hydrus/include/HydrusController.py", line 386, in WriteSynchronous
    return self._Write( action, HC.LOW_PRIORITY, True, *args, **kwargs )
  File "/home/username/git/hydrus/include/HydrusController.py", line 117, in _Write
    result = self._db.Write( action, priority, synchronous, *args, **kwargs )
  File "/home/username/git/hydrus/include/HydrusDB.py", line 824, in Write
    if synchronous: return job.GetResult()
  File "/home/username/git/hydrus/include/HydrusData.py", line 1668, in GetResult
    raise e
DBException: DataMissing: Service tag map error in database
Database Traceback (most recent call last):
  File "/home/username/git/hydrus/include/HydrusDB.py", line 517, in _ProcessJob
    elif job_type in ( 'write' ): result = self._Write( action, *args, **kwargs )
  File "/home/username/git/hydrus/include/ClientDB.py", line 9493, in _Write
    elif action == 'process_repository': result = self._ProcessRepositoryUpdates( *args, **kwargs )
  File "/home/username/git/hydrus/include/ClientDB.py", line 7249, in _ProcessRepositoryUpdates
    did_whole_update = self._ProcessRepositoryContentUpdate( job_key, service_id, content_update )
  File "/home/username/git/hydrus/include/ClientDB.py", line 6240, in _ProcessRepositoryContentUpdate
    tag_id = self._CacheRepositoryNormaliseServiceTagId( service_id, service_tag_id )
  File "/home/username/git/hydrus/include/ClientDB.py", line 697, in _CacheRepositoryNormaliseServiceTagId
    raise HydrusExceptions.DataMissing( 'Service tag map error in database' )
DataMissing: Service tag map error in database


Database Traceback (most recent call last):
  File "/home/username/git/hydrus/include/HydrusDB.py", line 517, in _ProcessJob
    elif job_type in ( 'write' ): result = self._Write( action, *args, **kwargs )
  File "/home/username/git/hydrus/include/ClientDB.py", line 9493, in _Write
    elif action == 'process_repository': result = self._ProcessRepositoryUpdates( *args, **kwargs )
  File "/home/username/git/hydrus/include/ClientDB.py", line 7249, in _ProcessRepositoryUpdates
    did_whole_update = self._ProcessRepositoryContentUpdate( job_key, service_id, content_update )
  File "/home/username/git/hydrus/include/ClientDB.py", line 6240, in _ProcessRepositoryContentUpdate
    tag_id = self._CacheRepositoryNormaliseServiceTagId( service_id, service_tag_id )
  File "/home/username/git/hydrus/include/ClientDB.py", line 697, in _CacheRepositoryNormaliseServiceTagId
    raise HydrusExceptions.DataMissing( 'Service tag map error in database' )
DataMissing: Service tag map error in database

>>5811

I found the bug that made me pkill hydrus, I paused the repo processing and told hydrus to exit late last night and it's been hanging since then. Here's the last few lines in the terminal:

2017/05/17 02:01:53: processed 30,662,109 content rows at 5,417 rows/s
2017/05/17 02:08:10:
2017/05/17 02:08:10: Exception:
2017/05/17 02:08:10: PyDeadObjectError: The C++ part of the _ServiceTagPanel object has been deleted, attribute access no longer allowed.
  File "/usr/lib/python2.7/threading.py", line 774, in __bootstrap
    self.__bootstrap_inner()
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/home/username/git/hydrus/include/HydrusThreading.py", line 286, in run
    HydrusData.ShowException( e )
  File "/home/username/git/hydrus/include/HydrusData.py", line 858, in PrintException
    trace_list = traceback.format_stack()
2017/05/17 02:08:10:
2017/05/17 02:08:10: Exception:
2017/05/17 02:08:10: PyDeadObjectError: The C++ part of the _ServiceTagPanel object has been deleted, attribute access no longer allowed.
  File "/usr/lib/python2.7/threading.py", line 774, in __bootstrap
    self.__bootstrap_inner()
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/home/username/git/hydrus/include/HydrusThreading.py", line 286, in run
    HydrusData.ShowException( e )
  File "/home/username/git/hydrus/include/HydrusData.py", line 858, in PrintException
    trace_list = traceback.format_stack()
2017/05/17 02:08:12:
2017/05/17 02:08:12: Exception:
2017/05/17 02:08:12: PyDeadObjectError: The C++ part of the _ServiceRepositoryPanel object has been deleted, attribute access no longer allowed.
  File "/usr/lib/python2.7/threading.py", line 774, in __bootstrap
    self.__bootstrap_inner()
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/home/username/git/hydrus/include/HydrusThreading.py", line 286, in run
    HydrusData.ShowException( e )
  File "/home/username/git/hydrus/include/HydrusData.py", line 858, in PrintException
    trace_list = traceback.format_stack()
2017/05/17 02:08:12: PTR sync: processing updates

>>5808

>>>5662

>

>That's odd–I assume those two files are the exact same thing, you just copied and renamed? I can't reproduce this problem, even with some complicated kana thrown in. Which version of Windows is that in your screenshot, 7? 8? Everything is explicitly converted to unicode inside hydrus, but I wonder if they are still dealing with it on their end in a different way. Is there any difference if you drop the unicode file on the main gui window vs dropping it on that import dialog?

Yep both files are the exact same thing just copied and renamed.

file names are:

Jessica Pare - Stardom (2000) GracefulSickIberianbarbel.webm

Jessica Paré - Stardom (2000) GracefulSickIberianbarbel.webm

I'm running Windows 7 SP1 x64.

I've got Python v2.7.10 installed (and on %path%) that I've been using for my own small programs.

Testing import methods with Hydrus v256 extractonly:

file>import files->add folder button->select directory

1 successful, 1 unsupported

drag drop both files onto main client window

1 successful, 1 unsupported

drap drop both files onto file->import files window

1 successful, 1 unsupported

I am unable to type or paste into the thread watcher URL field on v256 using Linux Mint 18. It seems like this problem is only on the thread watcher.

>>5872

See >>5874

So just how big is 368271232 pixels? I got a second decompression bomb warning in the past two days, took 10 minutes for my computer to react to ctrl alt del, then said it couldn't load the security shit, then another few minutes to just open the task manager, which showed hydrus using just under 2gb of ram (not bad) but the commit size was 12gb. I only vaguely understand what that means, and that it's virtual memory, but I've only got the 8gb of physical memory. Is there a better way to deal with these? Or a way to have actual information on what the file was in the log?

>>5948

>open back up

>go back to the search I was on

>there actually was a 37MB, 14,173x25,984 image grabbed from gelbooru

Well, fuck.

>>5979

https://gelbooru.com/index.php?page=post&s=view&id=444727

Didn't think there would be anything that big, especially not when grabbing everything from a series that began and fully ended pre-pixiv.

>>5985

The next largest files I've come across were a series of small poster scans, like the kind roughly a sheet of paper's width across, rolled up in a little blind packaged box. 2,145x6,075, png, 20 to 25 mb each and it's just stupid. I don't go around downloading huge single image files for fun or something.

hydrus (v258) threw a traceback on exit but nothing bad appears to have happened:

2017/06/04 03:23:39: waiting for daemons to exit
2017/06/04 03:23:39: Traceback (most recent call last):
2017/06/04 03:23:39:   File "/home/runningdroid/git/hydrus/include/ClientGUICommon.py", line 3812, in EventDirty
2017/06/04 03:23:39:     2017/06/04 03:23:39: self._func( *self._args, **self._kwargs )
2017/06/04 03:23:39:   File "/home/runningdroid/git/hydrus/include/ClientGUI.py", line 3406, in RefreshMenu
2017/06/04 03:23:39:     2017/06/04 03:23:39: ( menu, label, show ) = self._GenerateMenuInfo( name )
2017/06/04 03:23:39:   File "/home/runningdroid/git/hydrus/include/ClientGUI.py", line 1550, in _GenerateMenuInfo
2017/06/04 03:23:39:     2017/06/04 03:23:39: elif name == 'pages': return pages()
2017/06/04 03:23:39:   File "/home/runningdroid/git/hydrus/include/ClientGUI.py", line 1102, in pages
2017/06/04 03:23:39:     2017/06/04 03:23:39: gui_session_names = self._controller.Read( 'serialisable_names', HydrusSerialisable.SERIALISABLE_TYPE_GUI_SESSION )
2017/06/04 03:23:39:   File "/home/runningdroid/git/hydrus/include/HydrusController.py", line 290, in Read
2017/06/04 03:23:39:     2017/06/04 03:23:39: return self._Read( action, *args, **kwargs )
2017/06/04 03:23:39:   File "/home/runningdroid/git/hydrus/include/HydrusController.py", line 95, in _Read
2017/06/04 03:23:39:     2017/06/04 03:23:39: result = self._db.Read( action, HC.HIGH_PRIORITY, *args, **kwargs )
2017/06/04 03:23:39:   File "/home/runningdroid/git/hydrus/include/HydrusDB.py", line 813, in Read
2017/06/04 03:23:39:     2017/06/04 03:23:39: raise HydrusExceptions.ShutdownException( 'Application has shut down!' )
2017/06/04 03:23:39: include.HydrusExceptions2017/06/04 03:23:39: .2017/06/04 03:23:39: ShutdownException2017/06/04 03:23:39: : 2017/06/04 03:23:39: Application has shut down!

Slideshow isn't working for some reason.

hydrus does not ask if I want to run maintenance jobs on shutdown and does not run them.

hydrus version 258, happened when trying to add four tags in the same namespace, where three of the four where added with the keyboard hold ctrl+arrow keys, and one by scrolling down and CTRL clicking. three of the four where not on screen when trying to add by pressing enter.

SizeException
Received a zero-length tag!
Traceback (most recent call last):
  File "include\HydrusDB.py", line 534, in _ProcessJob
    result = self._Write( action, *args, **kwargs )
  File "include\ClientDB.py", line 9896, in _Write
    elif action == 'push_recent_tags': result = self._PushRecentTags( *args, **kwargs )
  File "include\ClientDB.py", line 7659, in _PushRecentTags
    tag_ids = [ self._GetTagId( tag ) for tag in tags ]
  File "include\ClientDB.py", line 5743, in _GetTagId
    HydrusTags.CheckTagNotEmpty( tag )
  File "include\HydrusTags.py", line 150, in CheckTagNotEmpty
    raise HydrusExceptions.SizeException( 'Received a zero-length tag!' )
SizeException: Received a zero-length tag!

>>6134

I should add, all the four tags did get added, and one extra tag in the namespace, which looks like it doesn't contain anything has been added

>>6134

>>6135

Thank you for this report and traceback. I will look at it next week!

If you petition a non-local tag on an item, that item will still show up when you search the tag you petitioned away from it until you actually commit the petition to the repo in question.

I don't know if it's a bug or working as intended, but I was checking for duplicates manually (checking artist tags and similar images from the menu, each opening a new tab) and after ~200 images I noticed my system slowing down big time. Hydrus was hoarding over 7GB of ram according to task manager.

For a couple of months now, the client has been crashing on and off for me when opening the "add sibling to tag" menu from the manage-tag dialog with a tag highlighted, I'm currently on win64 v258. Hydrus doesn't just hang, Windows almost immediately throws the "hydrus.exe has stopped working" box in my face, Hydrus doesn't write anything to any of the logs kept, not even a crash.log.

>>6206

Thank you for this report. I am afraid I cannot reproduce it on Win10. That dialog does take like 5 seconds to load now and is otherwise a big mess behind the scenes that can't deal with the current mass of pairs, so I can see how this could cause problems.

Do you get the crash if you open the dialog straight from services->manage tag siblings?

Are you asking to set siblings on tags that do or do not already have siblings?

I'll see about shifting the initialisation of the dialog around so it doesn't hang everything before it even exists. That sort of out-of-order event blocking tends to be the only thing that can cause a full gui crash for hydrus.

pixiv urls are http instead of https

>>6194

It's been a long time but I'll try to remember

>How many thumbnails, approx, were open across the program.

I'm assuming you mean how many images I had on tabs across the program, at one time probably ~2000 but in total during that session ~3000. Most of those were on tabs I didn't actually open that session so maybe they don't count for this? If that's the case, then it's less than a 1000.

>How many tabs were open.

7-10, maybe?

>Whether an import queue (even a silent one like an import folder) was happening, and what sort of files it was importing.

Nothing that left any notifications was happening

>How long the client had been open.

Like 6 hours or so.

>Whether you had been browsing many images or videos recently.

Nothing apart from the duplicate checking

>>6211

I just tried it out again, I loaded "manage tag siblings" a couple of times, one time opening an image in between. When I subsequently loaded another image and used "add siblings to *tag*" it immediately crashed.

After a couple of control runs it doesn't seem to matter whether the tag's already pending, what kind of tag I'm opening/what kind of namespace it's in or whether it already has siblings or not, the first couple of times the dialog is launched seem to succeed, and the crashes usually happen when I've opened the dialog for at least one *different* tag before.

Other bugs and circumstances I'm experiencing that I think might also be of note:

- I'm currently also having extreme slowdown issues with the tag autocompletion. Typing the first couple of letters in the tag: field when managing tags takes 10-60 seconds per symbol, while I'm using a Velociraptor disk for my DB, which is no SSD but should be fairly fast. Regenerating the cache doesn't seem to help all that much. This may be related to me running multiple tag repo's on a local server process to seperate collected tags from different sources and keep them from 'contaminating' the PTR when they're shitty (fucking tumblr blogs).

- I also have ~500k tags pending for the PTR because I pended the 'title:' tags from the partial FurAffinity dump.

And for more of a suggestion than a bug: I know you warned about trying out the Tumblr '_raw' URLs, but I think you might want to add an option to the downloader to it make fall back to trying out a non-raw version of a URL when downloading fails on a '_raw' one. It's currently failing the entire synchronization of some of my new subscriptions on the 403's Tumblr occasionally hands out.

http://cremechouette.tumblr.com/ is an example case that currently fails for me.

Now sure why, but I get some weirdness on closing hydrus in win 7 64 bit. It closes properly, the client.exe task clears from task manager but my system basically locks up for a minute or two afterwards. HDD light either stays on or flashes on/off for half a second each throughout the whole minute or two.

It doesn't effect anything currently going i.e. sound doesn't lag or lock up but I can't do anything besides move the mouse around the desktop.

>>6226

It's probably hydrus calling the Windows equivalent of fsync() on the databases after having filled the cache with writes.

I don't have access to a Windows box to test but you might be able to keep it from locking up your desktop by reducing hydrus' priority, I tend to set hydrus' io priority to idle when i'm doing something else and it seems to keep everything working fine.

When using the pixiv artist id downloader, the first page of images will get parsed properly but any pages after that report "0 urls found".

PyAssertionError
C++ assertion "(itemid >= 0 && itemid < SHRT_MAX) || (itemid >= wxID_AUTO_LOWEST && itemid <= wxID_AUTO_HIGHEST)" failed at ../src/common/menucmn.cpp(260) in wxMenuItemBase(): invalid itemid value
  File "/home/user/.git/hydrus/include/ClientGUIListBoxes.py", line 861, in EventMouseRightClick
    menu.Append( ClientCaches.MENU_EVENT_ID_TO_ACTION_CACHE.GetTemporaryId( 'new_search_page' ), 'open a new search page for ' + selection_string )
  File "/usr/lib/python2.7/dist-packages/wx-3.0-gtk2/wx/_core.py", line 12003, in Append
    return _core_.Menu_Append(*args, **kwargs)

got this error when right clicking on search result 'avengers' on ptr

>>6226

>>6274

I haven't tested it too extensively but I know for sure that it happens after vacuuming the db. It doesn't happen if I pop open hydrus for a small task and close but it does if I do an import of hundreds of images or do dupe processing and vacuum the db afterwards. (I manually do this regularly without issue and I don't think I've ever waited for a scheduled vacuum)

It says maintenance is complete before I close it so it's not like I'm cutting it off in the middle but if client.exe is closed I'm not sure any sort of db logs could catch what's going on.

>>6274

>This is the menu id overflow bug, which I am slowly fixing by replacing all the old menu code with a newer system. The fix for now is to restart the client. Please let me know if you are able to reproduce this soon after starting the client.

unfortunately i can't

one thing i can remember is that there is tag with many character, and the error happened.

after that right click will produce the error until application is restarted.

>>6374

It's been fixed, update.

A lil' bug report

1. Hydrus will silently crash if the custom thumbnail folder cannot be found. This is a pretty huge issue when migrating a DB to another installation. Possible solution would be to display a small dialog saying "could not find thumbnail folder", with options to abort startup or to unset old thumbnail folder.

2. On my linux installation (lubuntu, 1280 x 800 screen), most of the small dialog boxes (rating, add or remove tag, commit/confirm, etc) are slightly to short, and require dragging every time. I'm pretty sure WxPython has an option to wrap contents in this case.

If you make a shortcut for a tag that is shown as another tag, like 1girl, when you activate the shortcut, it will add the tag you specified to the image you're on, but the shortcut won't remove that tag. It only works in removing the tag if you specify a tag that won't be replaced.

>>6462

Thank you for this report. I will try to look at this tomorrow.

>>6395

>I will check on the custom thumbnail folder thing. I am not sure why it would cause a crash and not an error–was this on client boot, or did the folder dismount/whatever while the program was up?

It was on client boot.

Don't know if this is a bug or not, but everytime I run a DB vacuum(soft or full), I can never tell if it's actually working or not. I don't get any type of message.

>>6539

It might be that it is not running due to limited free space on your system partition/temporary folder. I think there are a couple of other checks it does as well. I will make a job to better inform the user when a manual attempt fails for this sort of unusual reason.

I recently updated to 270, and whenever I open it, it crashed at startup telling me that module 'cv2' doesn't have the 'CV_LOAD_IMAGE_UNCHANGED' field

I run from source, with dependencies installed separately

What can I do to fix it?

>>6622

I am sorry you are having this problem. I am not sure what is going on here. I juggled some of the opencv constants a little in v270, but that one remains the same as in v269. Was v269 the last version you were running, or a bit before?

Can you please try downloading the source for the last version you had running to a new location and seeing if a fresh client will boot?

Also, please open a terminal and go:

python
import cv2
cv2.__version__
cv2.CV_LOAD_IMAGE_UNCHANGED
cv2.IMREAD_UNCHANGED
exit()

What version does it think it is, and which of the constants failed?

If v269 was your last hydrus version, I believe that you can downgrade this week. If you try this, please let me know if it works, as this will show whether hydrus has changed or your opencv installation.

>>6630

My last installed version was v268, because I had forgot about the new release

I jumped to v270 from there, skipping a version

For a moment I thought it was because of python (I have both 2 and 3 installed), but clinent.py correctly calls python2, and python3 can't even find the cv2 module

Running the commands, these are the value I get:

cv2.version => 2.4.13

cv2.CV_LOAD_IMAGE_UNCHANGED => -1

cv2.IMREAD_UNCHANGED => -1L

The issue now is that trying to run hydrus again tells me the issue is on cv2.CV_LOAD_IMAGE_ANYDEPTH

>>6631

Not sure if it helps, but I just thought writing down the OS might be better

I run hydrus on GNU/Linux, as I said from source

The dependencies are installed through the package manager and are automatically updated when I update the system

So far I didn't have any problem though, this is the first time something like this happens

>>6632

>>6631

Thank you for this update. I have a temporary patch in place for tomorrow's release, so you should be able to boot.

Some of the constant names on the old version of OpenCV are unusual and badly documented, and I do not have a testing environment for it here. Please try this:

python
import cv2
cv2.CV_LOAD_IMAGE_ANYDEPTH
cv2.cv.CV_LOAD_IMAGE_ANYDEPTH
cv2.CV_LOAD_IMAGE_ANYCOLOR
cv2.cv.CV_LOAD_IMAGE_ANYCOLOR
exit()

If you would like to update to the new (3.3.0) version of OpenCV manually, you can with:

pip install opencv-python

The new version is great and has all kinds of GPU hardware acceleration for rendering, but if you have more programs that use 2.X, you may want to hold off unless you know they are 3.X compatible.

If your package manager has an option to update it, that might also be the better option.

>>6638

I'll just paste the console output:

>>> import cv2
>>> cv2.CV_LOAD_IMAGE_ANYDEPTH
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'CV_LOAD_IMAGE_ANYDEPTH'
>>> cv2.cv.CV_LOAD_IMAGE_ANYDEPTH
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'CV_LOAD_IMAGE_ANYDEPTH'
>>> cv2.CV_LOAD_IMAGE_ANYCOLOR
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'CV_LOAD_IMAGE_ANYCOLOR'
>>> cv2.cv.CV_LOAD_IMAGE_ANYCOLOR
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'CV_LOAD_IMAGE_ANYCOLOR'
>>> exit()

Apparently, hydrus is the only program using opencv, I upgraded it the last available through the PM, which is 3.2.0

Unfortunately the situation didn't change:

>>> import cv2
>>> cv2.CV_LOAD_IMAGE_ANYDEPTH
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'CV_LOAD_IMAGE_ANYDEPTH'
>>> cv2.cv.CV_LOAD_IMAGE_ANYDEPTH
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'cv'
>>> cv2.CV_LOAD_IMAGE_ANYCOLOR
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'CV_LOAD_IMAGE_ANYCOLOR'
>>> cv2.cv.CV_LOAD_IMAGE_ANYCOLOR
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'cv'
>>> exit()

>>6641

Nevermind, upgrading to 3.2.0 fixed the issue

Which makes sense, poking at the code, I see that the affected line happens only when OpenCV is at version 2

Still, you might look at it for those with version 2

Thanks for your help and keep the hard work

Opening webm files externally on Fedora 26 open up Notepad in Wine instead of mpv. Outside of Hydrus, all webms are set to open with mpv by default. This makes Hydrus useless for me, because I specifically got it to manage porn webms.

>>6642

>>6641

I am glad this got fixed. I have a patch in place for the 2.X users that just won't support some of the new EXIF rotation detection stuff. I will dive into the docs a bit more if there is demand but otherwise leave it alone.

>>6660

At the moment, for Linux, I run:

xdg-open "[file_path]"

I'm by no means an expert on this stuff, but could your xdg webm mapping for webm be unusual? What happens if you try that command in a regular terminal on an exported webm?

What happens if you try to open a jpeg or an mp4 externally in hydrus? Do they go to the right place?

I would like to add options for custom launch paths on a per mime basis in the future, but for the moment, I just pass it off to the OS to figure out.

>What happens if you try that command in a regular terminal on an exported webm?

Using xdg-oen on regular webms opened mpv. Exported webm from hydrus, used xdg-open to open it, still loaded mpv.

>What happens if you try to open a jpeg or an mp4 externally in hydrus? Do they go to the right place?

Jpegs were set to show in the in-built viewer by default, which worked fine. After changing them to be opened externally, it opened it up in Wine Internet Explorer instead of my image viewer. Also used xdg-open on jpegs, worked as expected, opening the image viewer. Mp4, when set to open externally, opened up Wine Notepad. This is pretty weird.

>>6673

After some searching, I found that sometimes wine notepad is set for unknown mimetypes. I wonder if the hydrus terminal is giving file info in a different way, or looking up the association in a different way. Is there any way that hydrus is running as a different user with different lookup table?

I'll reiterate that I'm not good at this, so please correct me if you know better, but please go:

xdg-mime query filetype [webmpath]
xdg-mime query default video/webm

I presume that will give us nothing surprising, but maybe one is unmapped.

You should have one or more mimeapps.list files as described here:

https://wiki.archlinux.org/index.php/Default_applications#XDG_standard

If you grep or just manually skim those for 'notepad', do you find anything? Are they for 'application/octet-stream' or any other 'default' mimetypes? Or can you find wine's internet explorer, whatever its .desktop file would be called?

>>6677

Different anon with same problem here. Don't have mime(1) on my system and couldn't find the package for it, but file -i gives the expected values for webm with binary charset. Running hydrus from console gives an undefined symbol gvfs_is_ipv6 for libgvfsdbus.so, which I have as version 1.32.1+5+gf0d758df-1 in the expected location. After that there's an error on failing to load that module and then there's the console output for the incorrect program. Not sure if it's noteworthy or not, but poking around in the file system shows half of the imported files as being zlib compressed data with no .watever extension. For mimeapps.list, I only have a user override file and the incorrect program is associated with text/plain. Interestingly there wasn't an explicit webm association, but adding one didn't change anything.

Thanks for the work so far, it's a pretty comfy program you have here.

Anyone else having issues with GIF's speeding through all their frames in 1-2 seconds? They then remain on their last frame while the seekbar at the bottom works through the time they're actually supposed to take to view these.

i.e. a 10 second GIF will speed through all its frames in the first 2 seconds and then remain frozen on the last frame for the remaining 8 seconds. This happened to most of my GIF's, old and new after an update some weeks ago.

>>6724

Thank you for this information. I will add the per-mime launch path options next week to fix this completely. Please let me know then how it works for you.

The zlibbed extensionless files are the cached repository update files your client has downloaded. They are stored and managed in the same file system as everything else but obviously aren't user facing. They are all json if you are interested in looking at them closer.

>>6725

Can you post or point me at some of these broken gifs, so I can check them my end?

>>6728

>>6730

And again, these did work correctly previously and I'm not on Linux or OSX

>>6734

That did the trick.

Old

1,086 frames; 18.1 seconds

43 frames; 2.1 seconds

197 frames; 2.0 seconds

256 frames; 2.6 seconds

256 frames; 2.6 seconds

New

700 frames; 18.1 seconds

8 frames; 2.1 seconds

50 frames; 2.0 seconds

52 frames; 2.6 seconds

52 frames; 2.6 seconds

ffmpeg version matches as well. Thanks for your help

>>6734

On another note, is there any way to make hydrus recover more gracefully from decompression errors like these?

2017/09/13 14:32:21: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (137578293 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:32:30: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (137457593 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:33:21: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (136245474 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:35:06: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (137959668 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:36:41: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (136247176 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:36:48: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (137620860 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:36:55: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (136816125 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:38:08: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (137647785 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:41:10: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (275015787 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:43:14: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (276887313 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:50:49: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (226451652 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.

It's not my first time with a decompression bomb error but before that only happened while trying to download a file. These managed to succeed and only cause the error when attempting to open them in the media viewer. Result is basically that the computer locks up due to Hydrus using large amounts of memory that increase rapidly; it jumped from ~800MB's to ~1.8GB's roughly 10 seconds into trying to view the file and then jumped to 3.8GB's after another 20-30 seconds or so. Hydrus stays locked up and while you can get taskmanager to pop up (I have a desktop shortcut for it) & try to kill the task, by that point it's pretty much too late since it would take so long to go through.

This batch of recent Ke-Ta uploads are the culprits:

https://gelbooru.com/index.php?page=post&s=view&id=3872394

https://gelbooru.com/index.php?page=post&s=view&id=3872396

https://gelbooru.com/index.php?page=post&s=view&id=3872424

https://gelbooru.com/index.php?page=post&s=view&id=3872426

https://gelbooru.com/index.php?page=post&s=view&id=3872460

https://gelbooru.com/index.php?page=post&s=view&id=3872465

https://gelbooru.com/index.php?page=post&s=view&id=3872468

https://gelbooru.com/index.php?page=post&s=view&id=3872469

https://gelbooru.com/index.php?page=post&s=view&id=3872483

https://gelbooru.com/index.php?page=post&s=view&id=3872498

https://gelbooru.com/index.php?page=post&s=view&id=3872543

https://gelbooru.com/index.php?page=post&s=view&id=3872545

https://gelbooru.com/index.php?page=post&s=view&id=3872547

https://gelbooru.com/index.php?page=post&s=view&id=3872549

https://gelbooru.com/index.php?page=post&s=view&id=3872550

https://gelbooru.com/index.php?page=post&s=view&id=3872554

https://gelbooru.com/index.php?page=post&s=view&id=3872556

https://gelbooru.com/index.php?page=post&s=view&id=3872557

https://gelbooru.com/index.php?page=post&s=view&id=3872558

https://gelbooru.com/index.php?page=post&s=view&id=3872634

The largest ones are 13859x9927 pixels

switched to running from source in a virtualenv on win7 and got an error on start up

Traceback (most recent call last):

File "client.pyw", line 20, in <module>

from include import ClientController

File "C:\Coding\python\hydrus\include\ClientController.py", line 21, in <module>

import ClientGUI

File "C:\Coding\python\hydrus\include\ClientGUI.py", line 11, in <module>

import ClientGUIManagement

File "C:\Coding\python\hydrus\include\ClientGUIManagement.py", line 14, in <module>

import ClientGUICanvas

File "C:\Coding\python\hydrus\include\ClientGUICanvas.py", line 15, in <module>

import ClientGUIShortcuts

File "C:\Coding\python\hydrus\include\ClientGUIShortcuts.py", line 8, in <module>

import wx.lib.flashwin

File "C:\Python27\Lib\site-packages\wx-3.0-msw\wx\lib\flashwin.py", line 20, in <module>

cc.GetModule( ('{D27CDB6B-AE6D-11CF-96B8-444553540000}', 1, 0) )

File "C:\Coding\python\hydrus\hydrus_env\lib\site-packages\comtypes\client\_generate.py", line 101, in GetModule

tlib = comtypes.typeinfo.LoadRegTypeLib(comtypes.GUID(tlib[0]), *tlib[1:])

File "C:\Coding\python\hydrus\hydrus_env\lib\site-packages\comtypes\typeinfo.py", line 478, in LoadRegTypeLib

_oleaut32.LoadRegTypeLib(byref(GUID(guid)), wMajorVerNum, wMinorVerNum, lcid, byref(tlib))

File "_ctypes/callproc.c", line 950, in GetResult

WindowsError: [Error -2147319779] Library not registered

figured out it was because i dont have flash installed. curiously it wasnt a problem when i was running one of dev's compiled releases

ended up writing this to get going again

diff –git a/include/ClientGUICanvas.py b/include/ClientGUICanvas.py

index 9c369aa..781a158 100755

— a/include/ClientGUICanvas.py

+++ b/include/ClientGUICanvas.py

@@ -30,7 +30,8 @@ import wx

if HC.PLATFORM_WINDOWS:

- import wx.lib.flashwin

+ if HC.FLASH:

+ import wx.lib.flashwin

ID_TIMER_SLIDESHOW = wx.NewId()

ID_TIMER_CURSOR_HIDE = wx.NewId()

diff –git a/include/ClientGUIShortcuts.py b/include/ClientGUIShortcuts.py

index 34861b0..8003f44 100644

— a/include/ClientGUIShortcuts.py

+++ b/include/ClientGUIShortcuts.py

@@ -5,7 +5,8 @@ import wx

if HC.PLATFORM_WINDOWS:

- import wx.lib.flashwin

+ if HC.FLASH:

+ import wx.lib.flashwin

def IShouldCatchCharHook( evt_handler ):

@@ -13,9 +14,10 @@ def IShouldCatchCharHook( evt_handler ):

window = wx.FindWindowAtPointer()

- if window is not None and isinstance( window, wx.lib.flashwin.FlashWindow ):

+ if HC.FLASH:

+ if window is not None and isinstance( window, wx.lib.flashwin.FlashWindow ):

- return False

+ return False

diff –git a/include/HydrusConstants.py b/include/HydrusConstants.py

index de276e9..1b50ba7 100755

— a/include/HydrusConstants.py

+++ b/include/HydrusConstants.py

@@ -667,3 +667,13 @@ sqlite3.register_adapter( bool, int )

sqlite3.register_converter( 'BLOB_BYTES', str )

sqlite3.register_converter( 'INTEGER_BOOLEAN', integer_boolean_to_bool )

sqlite3.register_converter( 'TEXT_YAML', yaml.safe_load )

+

+

+FLASH = True

+

+# running from source on windows without flash installed causes an error in comtypes library that isnt handled so catch and disable flash code

+if RUNNING_FROM_SOURCE and PLATFORM_WINDOWS:

+ try:

+ import wx.lib.flashwin

+ except WindowsError:

+ FLASH = False

fuck i hope this comes out formatted properly.

>>6741

of course this formatted like shit… i wish i knew how to dump text into a preformatted box

>>6742

code tags "code" "/code" with brackets instead of quotes

Tag parents and siblings do not get along well.

An example is character:shinon (sao-alo) being changed to character:sinon (sao-alo). When character:shinon (sao-alo) is entered and shows it will be displayed as sinon (sao-alo), it doesn't add the tag parents from shinon (sao-alo) or sinon (sao-alo). They are both tag parents, but it doesn't input the other tags associated with them if it is displaying one of the tags as another tag.

>>5115

On Linux, when opening Services>Review Services, the window that appears *really* wants to take up the whole screen. It will increase in size until it covers the whole screen, but not beyond.

Saving the session while searching for duplicates crashes the client.

>>6765

it's intentional

>>6767

Oh.

Any way to enable the audio again?

>>6786

>counter clockwise

Clockwise, I mean

>>6787

>>6786

Thank you for these examples. Support for EXIF-rotated images is not complete yet. Part of the rendering engine does it, but the file parsing doesn't, so they disagree. I hope to have complete support for different cases in the coming weeks, and then I will write a maintenance routine to re-parse existing files for rotation (and hence update their caches resolutions, which is part of the problem you are seeing) retroactively.

Tell me if I am being stupid, but hasn't 8chan/my browser made a similar mistake here? The thumbs look correct, but they load +90, -90, +90 degrees when I expand them. The stated 3000x1740 res is also incorrect in the same way, wew.

>>6824

The thumbs here actually aren't correct, the expanded images are(see the text). Opening in a new tab doesn't have the correct orientation again. Hydrus thumbnails are correctly oriented though.

Also this wasn't an issue in Hydrus when I imported them a month and a half ago, it's more recent.

When I select gifs, the client crashes with a segmentation fault before it can render the first frame in the preview box

I can't open them in any other way, as selecting them shows the preview and hydrus crashes as soon as the first frame is loaded

This actually happened in 274, but when I finally managed to upgrade to that version and discover the issue, 275 was released already

The crash still persists though

I've installed hydrus on win10 and it simply refuses to run. I can open the the server console but attempting to start the client does nothing. It appears in the task manager for a moment, then disappears, without any feedback. I did run it as an administrator. What could be at fault here?

>>6845

>>6825

Thanks. I get different rotations depending on whether I 'expand' the images inline vs opening them in a new tab, wew!

I will try to get all the pipelines doing what I think is correct and then we'll reparse this stuff and iron out lingering mistakes at that point. Thank you for the examples, as they will make testing on my end that much easier.

>>6853

Thank you for this report. I am sorry about your problems. Please flip file->options->media->disable OpenCV for gifs to whatever it currently isn't and let me know if that changes anything.

What happens if you open a webm or mp4?

And what version of PIL, Pillow and OpenCV do you have listed under help->about?

And which OS are you on?

>>6854

Hey, I am sorry you are having problems. Please check your install_dir/db folder for a crash.log file. If it exists, what does it report inside?

If no such file exists, please go back to install_dir, and then shift+right-click on some whitespace->open command window here and then type 'client' and hit enter on the terminal window that will pop up. If it gives an error traceback there, please take a screenshot and post it here or send it to my mail or whatever.

Some users with similar symptoms have also been hit by over-zealous anti-virus programs. If you run third-party anti-virus, please check that it hasn't quarantined part of the hydrus zip or installer or your install directory.

Also, is hydrus installed to a protected location like C:\Program Files? It needs to write to its own directory on a clean first boot. Running as admin might get around this, although it is obviously not ideal.

>>6858

>Thank you for this report. I am sorry about your problems. Please flip file->options->media->disable OpenCV for gifs to whatever it currently isn't and let me know if that changes anything.

Disabling OpenCV for gifs fixed it

>What happens if you open a webm or mp4?

Everything works fine, with and without OpenCV for gifs

>And what version of PIL, Pillow and OpenCV do you have listed under help->about?

OpenCV: 3.2.0

PIL: 1.1.7 (though I have the option to load images with it disabled)

Pillow: 3.4.2

>And which OS are you on?

I'm the same person of

>>6631

>>6632

>>6641

OpenCV keeps being an issue

>>6863

>>6862

Thank you for the update. I'm not familiar with powershell, but I guess you have to .\ to execute, wew! Apparently this is a new change, and if you would like to roll back to cmd, you can set it under Windows' settings->personalization->taskbar.

Anyway, please download this debug build and do the same thing:

https://www.mediafire.com/file/v1ugfjwyt3ulzfv/Hydrus%20Network%20276%20debug%20-%20Windows%20-%20Extract%20only.zip

It will explicitly write a lot of stuff to the terminal about how the boot is going, and should give us more info.

>>6866

Thank you for the follow-up. I am not sure what is doing this, but OpenCV has been a common pain, despite how good it is otherwise. Some Windows users had problems with OpenCV a while ago due to a bad set of Nvidia drivers–OpenCV uses OpenCL to do some rendering hardware acceleration, and a bug in Nvidia's OpenCL code was causing blue screen!

It might just be possible that you have a similar problem? If your GPU driver allows it, maybe try turning off OpenCL/CUDA support for client.exe. This may permit OpenCV to work again.

>>6899

>It might just be possible that you have a similar problem? If your GPU driver allows it, maybe try turning off OpenCL/CUDA support for client.exe. This may permit OpenCV to work again.

I have AMD hardware inside my laptop, and the "GPU" is the integrated card (it's a relatively old machine)

I can't really turn off support for anything, especially since I run hydrus from source (i.e. client.py)

OpenCV used to work fine, at least on version 2

I'll see if I can downgrade it and give results

>>6905

Hydrus 276, OpenCV 2.4.13, enabled for gif

Everything works fine, the client starts and selecting gifs shows them with no issues

Anyone else having this issue? I can't see the checkmarks… I thought the removal of the buttons was annoying but at least they still had text… with this I can't do anything…

>>6903

Thank you. I believe we had a similar problem a while ago with a different user who had a unicode character in his PATH environment variable, which the version of python I currently use cannot deal with. I think it was something including his username.

If you are not familiar with your PATH, just google it a bit. It is basically a list of places windows checks when it runs things. You can see how to find it here:

https://www.computerhope.com/issues/ch000549.htm

If you are not very familiar with what it is, be very careful. It is a critical part of Windows and should not be edited randomly. Please just check it for now–are there any unusual characters in your PATH?

If not your PATH, are there any other system level paths on your machine that might have a unicode character? Your username or anything else?

>>6905

>>6906

Great. I am not sure what happened here then. Maybe the new OpenCV hooks were interacting badly with your graphics stuff. Please let me know if you discover anything new.

>>6911

>>6912

I do not know what is going on here, and I don't think I have seen it before either. I don't think I changed anything on my end (with wxPython) that determines how these things are shown. Which OS are you on? Do you have any window managers active that alter gui elements? Has hydrus ever displayed correctly for you, or has it always had some corruption of some sort? If you open up the options menu, do the checkboxes and other elements on that also display badly, or is it just limited to the pages' left-hand management panels?

>>6936

meant to reply to >>6930

>>6967

Tumblr changed their URLs for raw images, they're now on data.tumblr.com instead of media.tumblr.com

>>6968

with https errors to boot

>>6968

As of…today? I had shit working yesterday

>>6973

Sometime within the last week

>>6969

Apparently s3.amazonaws.com/data.tumblr.com works correctly

>>6975

>>6973

two days ago

>>6980

Hey I have the same problem, here >>6912 and >>6936

Also Mint…

>>6980

>>6936

Can you please try extracting a fresh empty v268 and then comparing the library version numbers in help->about to your v277? I am guessing that wx or something got updated, perhaps in my recent Linux dev environment rebuild. If so, that at least gives us some information about this.

Then, please open a terminal and go:

python
import wx
wx.__version__
exit()

It may be the wx import statement fails, but it would be useful to know if Mint has a system wx as well that is perhaps conflicting here.

>>6963

Thank you, I will see if I can fix this for next week.

I agree about the dupe stuff. This first version I made has a lot of limitations, and most of my time went into the db-end of doing discovering dupes. The actual ui to make decisions about dupes is a lot less sophisticated, and I re-used existing ui where I could to save time. I hope a subsequent iteration of the system will have more specialised ui. I'd love to write some drag-and-drop/draw-arrows-between ui controls in future to help this sort of stuff, rather than always dealing with grids and lists. (drawing linked networks dynamically would be great for tag siblings and parents as well!)

>>6967

>>6968

>>6969

>>6973

>>6975

>>6979

Thank you for this report. Yeah, as 9c435e says, they rearranged their CDN and have a mickey-mouse ssl cert going on as well. I have this fixed for v278, please let me know if it gives you any more trouble.

Hydrus 277:

This client is the media management application of the hydrus software suite.

FFMPEG: 2.8.11-0ubuntu0.16.04.1

OpenCV: 3.3.0

openssl: OpenSSL 1.0.2g 1 Mar 2016

PIL: 1.1.7

Pillow: 4.2.1

python: 2.7.13

sqlite: 3.16.2

wx: 3.0.2.0 gtk2 (classic)

Hydrus 268:

This client is the media management application of the hydrus software suite.

FFMPEG: 2.8.11-0ubuntu0.16.04.1

OpenCV: 3.2.0

openssl: OpenSSL 1.0.2g 1 Mar 2016

PIL: 1.1.7

Pillow: 4.0.0

python: 2.7.13

sqlite: 3.16.2

wx: 3.0.2.0 gtk2 (classic)

Python wx test:

cfalcon@House-of-Hype ~ $ python

Python 2.7.12 (default, Nov 19 2016, 06:48:10)

[GCC 5.4.0 20160609] on linux2

Type "help", "copyright", "credits" or "license" for more information.

>>> import wx

>>> wx.version

'3.0.2.0'

>>> exit()

cfalcon@House-of-Hype ~ $

On a probably unrelated note, my version of OpenSSL is not the version listed in Hydrus. My version is 1.1.0f, the latest version on OpenSSL.org. This is probably just because I download the tarball on Github. I think 1.0.2g is still installed somewhere on my computer, but the if multiple exist then the default is the new version.

On a possibly related note, other programs (more specifically, a few source-installed emulators) have given me error messages that tell me that the version of wx the program was compiled with is not the same as my library's current version. Reinstalling the offending libraries with apt-get and recompiling the program did not stop the messages, but the programs all appear to run as expected anyways.

>>6991

>>6980

Also, right click->open externally now tries to open pngs, jpegs, and animated gifs in Wine's Internet Explorer instead of the default xviewer (which looks like a recent fork of eog (eye of gnome), the default image viewer in Mint 17.x).

I get a wine error (something about no Windows program configured to open this file type) when I try to right click-> open externally for webms. On 268 and earlier, I usually opened webms externally so that the sound would work.

>>6987

hydrus client 268, using network version 18:

FFMPEG: 2.8.11-0ubuntu0.16.04.1

OpenCV: 3.2.0

openssl: OpenSSL 1.0.2g 1 Mar 2016

PIL: 1.1.7

Pillow: 4.0.0

python: 2.7.13

sqlite: 3.16.2

wx: 3.0.2.0 gtk2 (classic)

hydrus client 276, using network version 18:

FFMPEG: 3.2-static

OpenCV: 3.3.0

openssl: OpenSSL 1.0.2g 1 Mar 2016

PIL: 1.1.7

Pillow: 4.2.1

python: 2.7.13

sqlite: 3.16.2

wx: 3.0.2.0 gtk2 (classic)

Python:

~ $ python

Python 2.7.12 (default, Nov 19 2016, 06:48:10)

[GCC 5.4.0 20160609] on linux2

Type "help", "copyright", "credits" or "license" for more information.

>>> import wx

>>> wx.version

'3.0.2.0'

>>> exit()

~ $

>>6992

>Also, right click->open externally now tries to open pngs, jpegs, and animated gifs in Wine's Internet Explorer instead of the default xviewer

I used to have this too for both hydrus images and other images a while ago, so I uninstalled Wine. I just now noticed that I can't open externally from v276 at all.

>>6987

>>6993

I just redownloaded them all and the problems appear in version 271.

Regarding the thread watchers auto naming:

Regardless of the settings, opening an old session with lots of thread watchers will cause them to lose my custom names for them. I would love it for this to be fixed, as I have a lot of sessions with old thread watchers, many of which the names are not obvious.

>>6993

>>6991

>>6995

Thank you for this. Looks like wx isn't it, but I will check the v271 changes to see what I did.

>>6992

>>6993

Please check the new overrides for 'open externally' under options->files and trash. Seems like some flavours of Linux are all over for this stuff, so you can now set it yourself. Please let me know if you run into any more trouble with this.

>>6996

I will see if I can do this simply, thank you for the report.

>>6987

>Thank you for this report. Yeah, as 9c435e says, they rearranged their CDN and have a mickey-mouse ssl cert going on as well. I have this fixed for v278, please let me know if it gives you any more trouble.

Will this essentially reset URLs? Just curious

>>7000

>just as it did when %path% was in double quotes

correction

>>7000

Apparently I can't post the error logs. I'm not allowed to attach .txt files to my post and if I post the error logs here, 8chan experiences a buffer overrun and doesn't post my reply.

Speaking as someone who doesn't know Python, but does know a few other languages, there seems to be some interesting information contained in the 846kB console output log (I had to copy-paste, since the >> operator didn't catch most of the actual errors).

Possibly even more interestingly, 268 gives errors too, but they seem to be different errors and they only show up on program startup.

>>7002

not dev ofc, but maybe you could upload the logs here: https://pastebin.com

you can set pastes to private so that they are only find-able with the link

>>6993

>>6991

Ok, I had a look at the commit here:

https://github.com/hydrusnetwork/hydrus/commit/e7449281e6f66400ac3ada9658ec48d7cb46f30c

There is nothing too special in the code, but the actual Linux release went down from 140MB to 102MB. Checking my release post, this is indeed the week when I had to rebuild my Linux machine.

Afaik, I now use a clean Ubuntu 17.10, whereas the old system, while I think it was 17.10, was updated from 15.something originally. I presume that all the library updates I got with a fresh build of my dev environment is now clashing with your Mint (16.04). It could also be a new version of PyInstaller packnig things differently, as I see a different format for the cv2 .so stuff as well in the two releases. In any case, I am guessing some GTK-something isn't presenting borders to your older window system properly.

I assume in your tests you have been making fresh installs. I was going to suggest trying a 'clean' install, in case the spare .so files from the v270 were confilcting with the v271, but if a fresh extract doesn't work either, that can't be true.

I do not know much about Linux, and I know less about Mint. Are they bringing out a new version based on the new Ubuntu? Do my thoughts on this likely being an old and new Ubuntu conflict sound reasonable?

I believe I know a Mint user who runs hydrus from source. If you are comfortable running pip a bunch of times, you could try this yourself as well. You already have wx, and that is usually the biggest pain. Here is the guide:

http://hydrusnetwork.github.io/hydrus/help/running_from_source.html

>>7008

Hey, >>6995 here. I have the same 'default:xdg-open "%path"' and stuff as >>7000, I didn't try all those things they did.

I don't know where I can find an error log, but launching from terminal with ./client gives me:

>(client:8643): Gdk-CRITICAL **: IA__gdk_window_get_origin: assertion 'GDK_IS_WINDOW (window)' failed

a couple of times and then things like:

>(client:8583): GdkPixbuf-CRITICAL **: gdk_pixbuf_get_pixels_with_length: assertion 'GDK_IS_PIXBUF (pixbuf)' failed

>(client:8583): GdkPixbuf-CRITICAL **: gdk_pixbuf_get_rowstride: assertion 'GDK_IS_PIXBUF (pixbuf)' failed

>(client:8583): GdkPixbuf-CRITICAL **: gdk_pixbuf_get_n_channels: assertion 'GDK_IS_PIXBUF (pixbuf)' failed

>(client:8583): GdkPixbuf-CRITICAL **: gdk_pixbuf_get_width: assertion 'GDK_IS_PIXBUF (pixbuf)' failed

>(client:8583): GdkPixbuf-CRITICAL **: gdk_pixbuf_get_height: assertion 'GDK_IS_PIXBUF (pixbuf)' failed

literally hundreds of times, occasionally interrupted by the message:

> (client:8583): WARNING : Invalid borders specified for theme pixmap:

> /usr/share/themes/Mint-X/gtk-2.0/images/notebook/tab-top-active.svg,

>borders don't fit within the image

or:

> (client:8583): WARNING : Pixbuf theme: Cannot load pixmap file /usr/share/themes/Mint-X/gtk-2.0/images/notebook/notebook.svg: Couldn't recognize the image file format for file '/usr/share/themes/Mint-X/gtk-2.0/images/notebook/notebook.svg'

In response to your post:

- Yes I have been making fresh installs. Well, "install", I just download and extract.

- I don't know much about Mint either. Afaik it is based on Ubuntu. The thing with Mint is that they are always very slow to update anything. So your thoughts may be correct. But I don't know anything about computer stuff so there's that.

- I will try running from source and report back.

- Thanks for all your attention to this problem.

>>7010

>spoilers

There were double asterisks there, guess that's how 8chan does spoilers…

>>7012

>– upgrade

this should be - and another - and then upgrade

>>7012

'open externally' works too on this install from source btw

(the media viewer still doesn't work though but that's unrelated to this)

>>6996

>>6997

The new option for not showing the 404 and dead marker doesn't seem to improve this.

When the old session is loaded, all the tabs have their old names, but as they initialize, they change name to "[dead] Thread Watcher"

>>7010

>>7012

>>7014

Hey, it's the other Mint user here. My error messages when running the tarball were slightly different, but the program seems to run correctly (correct icons, checkboxes work, open externally works) from source.

Running from source is a bitch on Mint. In addition to the BS that >>7012 did, I had to have pip upgrade lz4 after installing it or else it would fail to import lz4.block because _reasons_. I also had to sudo -H pip install a few of those modules. I didn't even know sudo _had_ flags, I always thought that the only arguments it takes are other commands.

Just an FYI, Mint 18 is pretty damn close to Ubuntu 16.04. If we're having trouble, they're probably having trouble. 18 seems to diverge a bit more from its Ubuntu base than earlier Mints did, but it's probably still close enough.

How exactly do you build the Linux version? I don't see a makefile or anything. Is it something I can do and share with my Mintian brothers?

>>7030

>How exactly do you build the Linux version? I don't see a makefile or anything. Is it something I can do and share with my Mintian brothers?

What do you mean? Have you downloaded and unpacked the source .zip or .tar.gz? Open terminal and cd to the directory that it is opened in, then just do python client.pyw, that's all there is to is (at least that worked for me). If you want to be able to open it from anywhere just make an alias.

>>7032

>the directory that it is opened in

I meant the directory that you get when you open the compressed thingy sorry.

>>7032

I haven't tested yet, as my hydrus folder is backing up, but I would assume that copying the source install over the old hydrus install wouldn't work right. I would guess that all of the so files in the regular hydrus folder would mask your system's shared objects. Yes, it can be fixed with one terminal command, but there are a bunch of other subdirectories which might give similar issues.

Also, when running from source the client would occasionally hang up for a few seconds. On the regular Linux install, I rarely see that except when I've been running hydrus for several days straight, trying to download all files from a huge tag on Gelbooru. Maybe it's just coincidence, but even if it's not, if I can make it easier for people to use Linux by taking literal seconds out of my week to run a single script or two, I don't see why I shouldn't do it if it's that fast.

>>7041

>I build using PyInstaller. This basically (…) per normal.

I must admit I don't understand anything about what it means to "build" it… I just executed the client.pyw script from the source zip with python… is that a bad idea?

I tried doing the same in the hydrus network folder you get from a normal install, and I got this error:

~ $ python client.pyw
Traceback (most recent call last):
  File "client.pyw", line 13, in <module>
    from include import HydrusData
  File "~/Pictures/hydrus network/include/HydrusData.py", line 8, in <module>
    import HydrusSerialisable
  File "~/Pictures/hydrus network/include/HydrusSerialisable.py", line 2, in <module>
    import lz4.block
ImportError: No module named block

My lz4 version is 0.10.1. If I try to import lz4 in a normal python shell, I can just do things like "from lz4 import block" or "import lz4.block" and I don't get error messages.

So this is a bit odd.

>>7041

Pics spoilered due to very NSFW thumbnails

All upcoming information comes from 277, since I have both the source and executable archives already downloaded.

Copying the source files over my old install doesn't work, but if I delete the old .so files in the main hydrus directory everything I've tested works as expected (this hold true for 278 as well).

Pyinstaller build from source mostly works (details below), but only if I copy the contents of the dist/client/ folder over either the source archive or the executable archive. Otherwise, it complains about missing icons and won't start.

I just tried a PyInstaller build of 277 (since I have both the source and executable archives already downloaded). It's still weird, but it works a hell of a lot better than the regular Linux executable, though not as well as running straight from source. The default xdg-open from Hydrus still opens in WINE IE, but manually setting image/jpeg (probably the others too) to xviewer will make it work right. Granted, some of xviewer's icons don't render if I do that, but that's probably xviewer's problem and not yours. Most UI elements render as expected, and checkboxes function again. File open icons are still using the weird ones I mentioned earlier. Duplicate processing no longer appears on the right click menu, for some reason. AFAICT, there's no way to set duplicate relationships with the build I just made, though the duplicates search page appears to function as expected. The dupe processing filter may work, but I have no idea how to control it. I got a few errors during the build, posting them below. I'll try to make a Mediafire account or something and post an alternate Linux build in the new version release threads (hence the tripcode).

cfalcon@House-of-Hype ~/.hydrus $ pyinstaller --windowed --hidden-import appdirs --hidden-import six --hidden-import packaging --hidden-import packaging.version --hidden-import packaging.specifiers --hidden-import packaging.requirements --name="client" hydrus-277/client.py
81 INFO: PyInstaller: 3.3
81 INFO: Python: 2.7.12
81 INFO: Platform: Linux-4.8.0-53-generic-x86_64-with-LinuxMint-18.2-sonya
83 INFO: wrote /home/cfalcon/.hydrus/client.spec
85 INFO: UPX is not available.
86 INFO: Extending PYTHONPATH with paths

...

14197 INFO: Processing pre-find module path hook   site
14197 INFO: site: retargeting to fake-dir '/usr/local/lib/python2.7/dist-packages/PyInstaller/fake-modules'
22114 INFO: Loading module hooks...

...

22150 INFO:   Removing import of PyQt4.QtCore from module PIL.ImageQt
22150 INFO: Loading module hook "hook-wx.lib.activex.py"...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/wx-3.0-gtk2/wx/lib/activex.py", line 35, in <module>
    import ctypes.wintypes as wt
  File "/usr/lib/python2.7/ctypes/wintypes.py", line 19, in <module>
    class VARIANT_BOOL(_SimpleCData):
ValueError: _type_ 'v' not supported
22243 INFO: Loading module hook "hook-lxml.etree.py"...

...

23287 INFO: Loading module hook "hook-setuptools.py"...
23288 WARNING: Hidden import "setuptools.msvc" not found!
23288 INFO: Loading module hook "hook-PIL.SpiderImagePlugin.py"...

If I download a picture from e.g. danbooru then the source url is not https.

>>7063

Some of the defaults are old and have not been updated to reflect https yet. I am not sure if danbooru has been updated yet, but I will check.

In any case, please go services->manage boorus then select danbooru and change the search url to 'https'. Just add the 's', hit ok, and it should thereafter fetch https urls. I just tested this on my own cleant and it seems ok, but let me know if it gives you any trouble.

>>7052

Thanks, this is really helpful. I will put some time aside to sort this out.

>>7059

>>7060

Yeah–there may be ways to inform pyinstaller about non-python stuf like the 'static' and 'help' folders that you eventually want in the build directory, but I just copy it over in a script in the same way. Make sure you don't ever copy a real database into your empty build db directory–I almost completely fucked myself once doing that.

Could the duplicate stuff be that you are not in advanced mode (help->advanced mode)? New clients won't start in advanced.

I get similar errors when I build. There are ten kinds of voodoo that pyinstaller has to do to get OS-specific stuff to work, and it logs all the problems it finds. So far, my rule has been that if it boots on the other side, I won't complain.

Grats for figuring this out, and thanks again for making your release available.

Just for shits and giggles, I tried manually setting the open externally program to xdg-open. Didn't fix anything, but I didn't expect it to.

>>7069

As it turns out, not being in advanced mode is exactly why dupe processing didn't work.

>>7069

>services->manage boorus

Thanks that worked !

With version 282, the main GUI (search, sort, tags) is resetting in size on each new tab. I changed no settings. This is also new behavior, and I also realize I have no clue how to edit the GUI elements, the changes I'm making don't effect the GUI options menu.

>>7261

this started working again, must have something to do with running off an external.

>>7286

>Please check options->gui->always embed a/c dropdown results window and restart the client.

That solved it, thanks!

I just upgraded to 284 (MintTeaFresh's Linux build). Hydrus then checked 3 of my booru subscriptions, and declared all of them dead. Two of them haven't found new images in over a year, so that's not surprising, but one grabbed something less than a month ago.

>>7372

Can you please check the respective subscription's 'edit check timings' button? Any subs that updated from the old system got different 'dead' timings based on the old static subscription period. It may be that has 'dead of < 1 file per 14 days' or something.

It seems like related tags isn't working right anymore. Now it just seems it picks a random image it knows and suggests all the tags from that image now.

>>7399

I don't think I changed any of that code. It has never been excellent, particularly if the search distances in options->tags are short. What happens if you open manage tags over and over on the same file–do you get substantially different results every time? What if you click the 'thorough' button? If so, you might like to try bumping up the search time, as your mappings have probably expanded and it might not be searching the whole search space deep enough to get good results.

Can you go into any more depth on the sorts of bad results you are getting? Different 'sample' tags on the file work better than others. Also, some big tags (like 'series:touhou') have such a large imprint on the db that they can muddy any other specificity. A future iteration of the system could add weights on input/output tags based on 'usefulness', although I am not sure what that exactly means in statistical terms atm.

Database Traceback (most recent call last):
  File "/include/HydrusDB.py", line 523, in _ProcessJob
    result = self._Read( action, *args, **kwargs )
  File "/include/ClientDB.py", line 7985, in _Read
    elif action == 'media_results_from_ids': result = self._GetMediaResults( *args, **kwargs )
  File "/include/ClientDB.py", line 5076, in _GetMediaResults
    hash_ids_to_hashes = self._GetHashIdsToHashes( hash_ids )
  File "/include/ClientDB.py", line 4900, in _GetHashIdsToHashes
    ( hash, ) = self._c.execute( 'SELECT hash FROM hashes WHERE hash_id = ?;', ( hash_id, ) ).fetchone()
TypeError: 'NoneType' object is not iterable

I imported a file without a hash. Things broke.

latest download, on linux, specifically slackware. every few minutes this happens.

DBException
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 55-56: ordinal not in range(256)
Traceback (most recent call last):
  File "include/HydrusThreading.py", line 146, in run
    self._callable( self._controller )
  File "include/ClientDaemons.py", line 46, in DAEMONCheckImportFolders
    import_folder = controller.Read( 'serialisable_named', HydrusSerialisable.SERIALISABLE_TYPE_IMPORT_FOLDER, name )
  File "include/HydrusController.py", line 370, in Read
    return self._Read( action, *args, **kwargs )
  File "include/HydrusController.py", line 120, in _Read
    result = self.db.Read( action, HC.HIGH_PRIORITY, *args, **kwargs )
  File "include/HydrusDB.py", line 846, in Read
    return job.GetResult()
  File "include/HydrusData.py", line 1774, in GetResult
    raise e
DBException: UnicodeEncodeError: 'latin-1' codec can't encode characters in position 55-56: ordinal not in range(256)
Database Traceback (most recent call last):
  File "include/HydrusDB.py", line 523, in _ProcessJob
    result = self._Read( action, *args, **kwargs )
  File "include/ClientDB.py", line 8004, in _Read
    elif action == 'serialisable_named': result = self._GetJSONDumpNamed( *args, **kwargs )
  File "include/ClientDB.py", line 5031, in _GetJSONDumpNamed
    return HydrusSerialisable.CreateFromSerialisableTuple( ( dump_type, dump_name, version, serialisable_info ) )
  File "include/HydrusSerialisable.py", line 100, in CreateFromSerialisableTuple
    obj.InitialiseFromSerialisableInfo( version, serialisable_info )
  File "include/HydrusSerialisable.py", line 156, in InitialiseFromSerialisableInfo
    self._InitialiseFromSerialisableInfo( serialisable_info )
  File "include/ClientImporting.py", line 1951, in _InitialiseFromSerialisableInfo
    self._path_cache = HydrusSerialisable.CreateFromSerialisableTuple( serialisable_path_cache )
  File "include/HydrusSerialisable.py", line 100, in CreateFromSerialisableTuple
    obj.InitialiseFromSerialisableInfo( version, serialisable_info )
  File "include/HydrusSerialisable.py", line 153, in InitialiseFromSerialisableInfo
    ( version, serialisable_info ) = self._UpdateSerialisableInfo( version, serialisable_info )
  File "include/ClientImporting.py", line 3334, in _UpdateSerialisableInfo
    if os.path.exists( seed_text ):
  File "genericpath.py", line 26, in exists
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 55-56: ordinal not in range(256)


Database Traceback (most recent call last):
  File "include/HydrusDB.py", line 523, in _ProcessJob
    result = self._Read( action, *args, **kwargs )
  File "include/ClientDB.py", line 8004, in _Read
    elif action == 'serialisable_named': result = self._GetJSONDumpNamed( *args, **kwargs )
  File "include/ClientDB.py", line 5031, in _GetJSONDumpNamed
    return HydrusSerialisable.CreateFromSerialisableTuple( ( dump_type, dump_name, version, serialisable_info ) )
  File "include/HydrusSerialisable.py", line 100, in CreateFromSerialisableTuple
    obj.InitialiseFromSerialisableInfo( version, serialisable_info )
  File "include/HydrusSerialisable.py", line 156, in InitialiseFromSerialisableInfo
    self._InitialiseFromSerialisableInfo( serialisable_info )
  File "include/ClientImporting.py", line 1951, in _InitialiseFromSerialisableInfo
    self._path_cache = HydrusSerialisable.CreateFromSerialisableTuple( serialisable_path_cache )
  File "include/HydrusSerialisable.py", line 100, in CreateFromSerialisableTuple
    obj.InitialiseFromSerialisableInfo( version, serialisable_info )
  File "include/HydrusSerialisable.py", line 153, in InitialiseFromSerialisableInfo
    ( version, serialisable_info ) = self._UpdateSerialisableInfo( version, serialisable_info )
  File "include/ClientImporting.py", line 3334, in _UpdateSerialisableInfo
    if os.path.exists( seed_text ):
  File "genericpath.py", line 26, in exists
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 55-56: ordinal not in range(256)

>>7435

also, hydrus seems to crash alot when minimized during these errors.

>>7395

Thanks, it looks like it'll behave again if I play around with those settings.

>>7433

I am sorry you are having this problem. Can you explain more what lead up to this? Was this a normal file import that somehow has broken things, or did you import a file by interacting with the database directly with a SQL program? Every file should have a hash.

Have you recently had any power outages or hard crashes of the client during importing?

>>7435

>>7438

Thank you for this report. I am sorry for the inconvenience. Please hit services->pause->import folders for now. This will stop the error from spamming you. I will look into the traceback and try to fix the error for next week. Please try unpausing your import folders then and let me know if it works again.

>>7473

error spam continues if I pause and unpause.

also, I think I may have mistook >>7438 for hydrus closing because of inactivity. the spam is still annoying though.

>>7473

>>7433 here

I'm not 100% sure what happened (only discovered it when a search failed) but I think my computer suspended/went to sleep while hydrus was importing a file causing an entry to be made without a hash included.

Checking database integrity found no problems. A quick file integrity check "corrected" the problem but that led to another issue where majority of new imports were failing because the db says it had already deleted that file from the db.

In the end it's not a big deal. I've created a fresh db.

Sometimes I turn off my remote file/tag server. If I accidentally try to manage the service, Hydrus becomes unresponsive and I have to wait for the connection to timeout to be able to do anything.

It's low priority, since it eventually recovers, but it would be nice to have a way to cancel the connection. Additionally, is the connection attempted on a separate thread? Why does the UI freeze?

not really a bug anyone will find unless you're adding your own mods to the source

in thread watcher page -> tag import options: the default namespaces (i.e. filename checkbox) are hardcoded and are not being pulled from ClientDefaults.GetDefaultNamespacesAndSearchValue()

where as in file -> options -> default tag options -> add/edit -> thread watcher: the namespaces are pulled from the defaults

to repro:

edit ClientDefaults.py line 372 and add another namespace

e.g.: namespaces = [ 'filename', 'title' ]

file -> options -> default tag options -> add/edit -> thread watcher : there are two checkboxes (filename and title)

thread watcher page -> tag import options : only one checkbox (filename), should be two

From 320573c024a8a3366df802a72942a844471a1a45 Mon Sep 17 00:00:00 2001
From: Anonymous <anon@anon.com>
Date: Tue, 26 Dec 2017 01:08:38 +1100
Subject: [PATCH] Unhardcoded namespace list in tag import options popup on the
 thread watcher page

---
 include/ClientGUIManagement.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/include/ClientGUIManagement.py b/include/ClientGUIManagement.py
index f0095f8..5d57742 100755
--- a/include/ClientGUIManagement.py
+++ b/include/ClientGUIManagement.py
@@ -2080,8 +2080,10 @@ class ManagementPanelImporterThreadWatcher( ManagementPanelImporter ):
         
         ( thread_url, file_import_options, tag_import_options ) = self._thread_watcher_import.GetOptions()
         
+        ( namespaces, search_value ) = ClientDefaults.GetDefaultNamespacesAndSearchValue( ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_THREAD_WATCHER ) )
+        
         self._file_import_options = ClientGUIImport.FileImportOptionsButton( self._thread_watcher_panel, file_import_options, self._thread_watcher_import.SetFileImportOptions )
-        self._tag_import_options = ClientGUIImport.TagImportOptionsButton( self._thread_watcher_panel, [ 'filename' ], tag_import_options, self._thread_watcher_import.SetTagImportOptions )
+        self._tag_import_options = ClientGUIImport.TagImportOptionsButton( self._thread_watcher_panel, namespaces, tag_import_options, self._thread_watcher_import.SetTagImportOptions )
         
         #
         
-- 
2.8.0.windows.1

i dont know if devbro likes patches or not but i find them easier for communicating and i hope it makes it easier for devbro to write his own patch

>>7559

tried running database -> check -> file integrity and seeing if any of your files are actually missing?

2018/01/06 11:18:08: Exception:
2018/01/06 11:18:08: AttributeError
'NoneType' object has no attribute 'shape'
Traceback (most recent call last):
  File "/opt/hydrus/include/HydrusThreading.py", line 242, in run
    callable( *args, **kwargs )
  File "/opt/hydrus/include/ClientRendering.py", line 402, in THREADRender
    frame = GenerateHydrusBitmapFromNumPyImage( numpy_image, compressed = False )
  File "/opt/hydrus/include/ClientRendering.py", line 25, in GenerateHydrusBitmapFromNumPyImage
    ( y, x, depth ) = numpy_image.shape
AttributeError: 'NoneType' object has no attribute 'shape'

when trying to open one webm

The default regex for "E:\my collection\author name - v4c1p0074.jpg -> author name" seems to be broken, at least in v287

>[^\\]+*(?=\s-)

Gives a "multiple result" error, and regexr.com just says error.

>>7519

I am always overwhelmed with things to do, so a lot of my code has been thrown together just to get something going. I don't always have great ui behaviour when things go wrong. It is a long-term project to rewrite older parts of the program to behave more neatly and put potentially blocking sections like this on their own thread. Thank you for reporting this–I will make a job to put some time into it.

>>7542

Thank you for this report. This is an interesting problem–a lot of the siblings and parent code is still first-version prototype and is basically magic tricks to make you think at the gui level that tags are one thing while the db retains the original truth of what they are. as a result, some db-level search code doesn't have a special hook to compensate. I will check the *anything* search code out to see if I can slip in easy siblings support, but if it is super complicated, it will have to wait for the next iteration over the whole siblings and parents system, which will likely extend sibling and parent awareness to a new db cache layer that can be efficiently searched without a bunch of CPU checking these links.

>>7559

Thank you for this report. This is fixed for v289–let me know if it gives you any more trouble.

>>7561

Thanks for this interesting report. I have done exactly as you suggest.

The whole shebang will be replaced in the downloader overhaul, with the namespace list being populated by what the parsing scripts say they support and everything running off the same system, so this will all disappear anyway in a bit. It sounds like you'll be well able to create your own parsing scripts when all that happens, so let me know if the new thread watcher parsing stuff doesn't do what you would like as I roll it out.

>>7599

Thank you for this report. There seems to be some kind of problem syncing files being deleted and similar file information in the duplicate search tree. Some other users have noticed this same 'file is remote, but shouldn't be seen in this local-file-only search context' after doing some dupe filtering.

I think I have identified how these spurious results are appearing, if not the cause. It might have been old code that I more recently fixed, so some users just have some orphans hanging around. In any case, running database->regen->similar files search tree in v289 should clear these bad results out. Please run it when convenient and let me know if this search still produces the bad result, and if these bad results reappear at all.

>>7634

Thank you. It looks like the client is having trouble rendering the first frame of that video, which is unexpected as it must have imported ok at one point, which would have needed the first frame to generate the thumbnail. I have improved this error text, but can you post/link/send me the webm so I can check it out on my end?

If you go help->about, what version of ffmpeg do you appear to have?

>>7642

Thank you. That looks like a typo. I updated some of this stuff recently, so I think I must have copied something wrong. I have corrected the default for new users, but you will have to fix it yourself with the 'manage favourites' part. I think this will do the job:

[^\\]+(?=\s-)

i.e. Just take out the asterisk and you should be good.

So I took a pause from hydrus, due to moving computers (Debian to Antergos/Arch). Today I decided I wanted to migrate, and obviously I run into a heap of bugs. Stuff I've noticed while attempting to fix my database:

- Crash on launch when one or two of client.mappings.db, client.master.db or client.db are missing. An error message would be nice.

- The database assumes thumbnails being in a specific location, when the custom thumbnail locations are specified. I needed to set all locations in client_file_locations to "client_files", and overwrite my options value, before it would work.

- The cache is awfully fragile. Some cache tables are not automatically regenerated if not found. Some can be manually created in the database menu, but at least one of them (shape_vptree) are deadlocked. (See below)

- I get notifications about some "BetterListCtrl" being deleted now and then. Not sure what that is about. (Again see below)

Traceback (most recent call last):
  File "include/HydrusDB.py", line 527, in _ProcessJob
    result = self._Write( action, *args, **kwargs )
  File "include/ClientDB.py", line 10699, in _Write
    elif action == 'regenerate_similar_files': result = self._CacheSimilarFilesRegenerateTree( *args, **kwargs )
  File "include/ClientDB.py", line 1835, in _CacheSimilarFilesRegenerateTree
    self._c.execute( 'DELETE FROM shape_vptree;' )
OperationalError: no such table: shape_vptree

RuntimeError
wrapped C/C++ object of type BetterListCtrl has been deleted
  File "include/ClientGUIListCtrl.py", line 1193, in EventContentChanged
    self._UpdateButtons()
  File "include/ClientGUIListCtrl.py", line 1132, in _UpdateButtons
    if enabled_check_func():
  File "include/ClientGUIScrolledPanelsReview.py", line 1349, in _FileLocationSelected
    locations = self._current_media_locations_listctrl.GetData( only_selected = True )
  File "include/ClientGUIListCtrl.py", line 807, in GetData
    indices = self._GetSelected()
  File "include/ClientGUIListCtrl.py", line 593, in _GetSelected
    i = self.GetFirstSelected()
  File "wx/core.py", line 2715, in _ListCtrl_GetFirstSelected
  File "wx/core.py", line 2722, in _ListCtrl_GetNextSelected