KeyError 13

Name
Email
Subject
Comment *
File	Select/drop/paste files here
* = required field	[▶ Show post options & limits] Confused? See the FAQ.

Embed	(replaces files and can be used instead)
Options	Do not bump (you can also write sage in the email field) Spoiler images (this replaces the thumbnails of your images with question marks)
Password	(For file and post deletion.)
Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf Max filesize is 12 MB. Max image dimensions are 10000 x 10000. You may upload 5 per post.

KeyError 13 Anonymous 07/15/16 (Fri) 23:33:01 a626c4 No.3180

Hey dev, I keep getting this error when hydrus is populating files from system:archive.

Side question, when clearing orphans, why do I have near twice as many thumbnails as actual pictures?

DBException

KeyError: 13

Traceback (most recent call last):

File "include\HydrusThreading.py", line 274, in run

callable( *args, **kwargs )

File "include\ClientController.py", line 1031, in THREADDoFileQuery

more_media_results = self.Read( 'media_results_from_ids', sub_query_hash_ids )

File "include\HydrusController.py", line 229, in Read

def Read( self, action, *args, kwargs ): return self._Read( action, *args, kwargs )

File "include\HydrusController.py", line 91, in _Read

result = self._db.Read( action, HC.HIGH_PRIORITY, *args, **kwargs )

File "include\HydrusDB.py", line 644, in Read

return job.GetResult()

File "include\HydrusData.py", line 1838, in GetResult

raise self._result

DBException: KeyError: 13

Database Traceback (most recent call last):

File "include\HydrusDB.py", line 462, in _ProcessJob

if job_type in ( 'read', 'read_write' ): result = self._Read( action, *args, **kwargs )

File "include\ClientDB.py", line 5985, in _Read

elif action == 'media_results_from_ids': result = self._GetMediaResults( *args, **kwargs )

File "include\ClientDB.py", line 3957, in _GetMediaResults

petitioned_file_service_keys = { service_ids_to_service_keys[ service_id ] for service_id in hash_ids_to_petitioned_file_service_ids[ hash_id ] }

File "include\ClientDB.py", line 3957, in <setcomp>

petitioned_file_service_keys = { service_ids_to_service_keys[ service_id ] for service_id in hash_ids_to_petitioned_file_service_ids[ hash_id ] }

KeyError: 13

Anonymous 07/16/16 (Sat) 05:07:05 a626c4 No.3183

I also just noticed my server is giving an error:

2016/07/16 01:05:58: Server is running. Press Ctrl+C to quit.

2016/07/16 01:06:00: Daemon DeleteOrphans encountered an exception:

2016/07/16 01:06:00:

2016/07/16 01:06:00: Exception:

2016/07/16 01:06:00: TypeError: Traceback (most recent call last):

File "include\HydrusDB.py", line 463, in _ProcessJob

elif job_type in ( 'write' ): result = self._Write( action, *args, **kwargs )

File "include\ServerDB.py", line 2687, in _Write

elif action == 'delete_orphans': result = self._DeleteOrphans( *args, **kwargs )

File "include\ServerDB.py", line 807, in _DeleteOrphans

thumbnails_hashes = ServerFiles.GetAllHashes( 'thumbnail' )

File "include\ServerFiles.py", line 7, in GetAllHashes

def GetAllHashes( file_type ): return { os.path.split( path )[1].decode( 'hex' ) for path in IterateAllPaths( file_type ) }

File "include\ServerFiles.py", line 7, in <setcomp>

def GetAllHashes( file_type ): return { os.path.split( path )[1].decode( 'hex' ) for path in IterateAllPaths( file_type ) }

File "encodings\hex_codec.py", line 42, in hex_decode

TypeError: Non-hexadecimal digit found

File "threading.py", line 774, in __bootstrap

File "threading.py", line 801, in __bootstrap_inner

File "include\HydrusThreading.py", line 167, in run

HydrusData.ShowException( e )

File "include\HydrusData.py", line 835, in PrintException

trace_list = traceback.format_stack()

hydrus_dev ## Board Owner 07/16/16 (Sat) 21:00:10 26412c No.3189

File: 1468702812446.jpg (91.68 KB, 1071x894, 357:298, a1084353eff286310616c5ea6e….jpg)

Thank you for both of these reports. I have fixed both of them for next week. Let me know if they give you any more trouble.

For the thumbnails, the client generates a 200x200 'master' thumbnail of every thumbnailable file on import and then generates a smaller thumbnail from the master the first time you load it in the gui. It keeps the master around so it can quickly generate (or regenerate, if you change resolution) the smaller on demand. If you browse to install_dir/db/client_files/whatever/, you should see a bunch of file groups named:

[hash].jpg

[hash].thumbnail

[hash].thumbnail.resized

If you copy the thumbs somewhere and rename them .jpg, you should be able to load them in any image viewer.

Anonymous 07/24/16 (Sun) 00:11:12 a626c4 No.3249

Those errors are gone! Thanks. But I found a new one.

I got this when clicking "process" on a file petition while the only petition was unchecked.

2016/07/23 20:08:06: Traceback (most recent call last):

File "threading.py", line 801, in __bootstrap_inner

File "threading.py", line 754, in run

File "site-packages\twisted\_threads\_threadworker.py", line 46, in work

File "site-packages\twisted\_threads\_team.py", line 190, in doWork

— <exception caught here> —

File "site-packages\twisted\python\threadpool.py", line 246, in inContext

File "site-packages\twisted\python\threadpool.py", line 262, in <lambda>

File "site-packages\twisted\python\context.py", line 118, in callWithContext

File "site-packages\twisted\python\context.py", line 81, in callWithContext

File "include\ServerServerResources.py", line 510, in _threadDoPOSTJob

HydrusGlobals.server_controller.WriteSynchronous( 'update', self._service_key, account_key, update )

File "include\HydrusController.py", line 341, in WriteSynchronous

return self._Write( action, HC.LOW_PRIORITY, True, *args, **kwargs )

File "include\HydrusController.py", line 139, in _Write

result = self._db.Write( action, priority, synchronous, *args, **kwargs )

File "include\HydrusDB.py", line 685, in Write

if synchronous: return job.GetResult()

File "include\HydrusData.py", line 1838, in GetResult

raise self._result

exceptions.TypeError: Traceback (most recent call last):

File "include\HydrusDB.py", line 463, in _ProcessJob

elif job_type in ( 'write' ): result = self._Write( action, *args, **kwargs )

File "include\ServerDB.py", line 2696, in _Write

elif action == 'update': result = self._ProcessUpdate( *args, **kwargs )

File "include\ServerDB.py", line 2057, in _ProcessUpdate

for hashes in update.GetContentDataIterator( HC.CONTENT_TYPE_FILES, HC.CONTENT_UPDATE_DENY_PETITION ):

File "include\HydrusData.py", line 1464, in <genexpr>

else: return ( { self._hash_ids_to_hashes[ hash_id ] for hash_id in hash_ids } for hash_ids in ( data, ) )

File "include\HydrusData.py", line 1464, in <setcomp>

else: return ( { self._hash_ids_to_hashes[ hash_id ] for hash_id in hash_ids } for hash_ids in ( data, ) )

TypeError: unhashable type: 'list'

Anonymous 07/24/16 (Sun) 00:31:52 a626c4 No.3250

>>3249

If you could explain the petition process in general, I'd appreciate it. Your manual only mentions to "set up a server and play with it" or "connect to my server"

Except, I can't tell if the logic behind petitions is unintuitive, or if it's simply broken.

I created a user for both file/tag services with permissions: get data, post petitions, post data

tag service:

Without 'post data', access is denied. With post data, any tag the user uploads is just accepted, and I do not see their uploads in the tag petition page.

file service:

Things deleted by the user appear on the petition page. This leads me to believe the tag logic is broken. However, as an admin I am only able to accept (process) the petition, or… ignore it. I can't reject it. I thought rejecting might be to "process" with the box unchecked, but it results in the previous post's error on the server log.

What's going on here? Are my suspicions about how it is supposed to work correct, and I'm experiencing bugs? Or does petitioning tags work differently than petitioning files, and I just don't know the proper way to do it?

hydrus_dev ## Board Owner 07/24/16 (Sun) 19:37:24 26412c No.3252

File: 1469389045015.jpg (330.35 KB, 1392x1774, 696:887, 42707c670a96cb9683964ab0e2….jpg)

>>3249

Thank you for this new report. I have fixed it for next week. You might be the first person to ever try denying a file petition! I didn't realise this basic data fetch wasn't tested, I'll write some unit tests to catch it in future.

>>3250

Permissions are a mess right now. I haven't touched it in ages because no one's mentioned it and I haven't had to change them on my PTR and example file repo, but if you are actively managing a server, I'll schedule some improvements.

The initial idea of permissions was to have a sliding scale, but it turned out a little too basic for what repos ended up being able to do. I'd like in future for there to be more descriptive and granular permissions, so you'll be able to say 'user can add tags, but it'll create a petition I have to approve' and 'user can petition files but not upload them' and any other combination, but for now some things are treated differently at the client and server.

Essentially, atm these combinations make sense:

get data

get data, post data

get data, post data, post petitions

get data, post data, post petitions, resolve petitions

Adding tags and files needs 'post data' and goes through without a petition. Tag siblings and parents need 'post data' and do create a petition.

Removing tags, files, siblings or parents needs both 'post data' and 'post siblings' and creates a petition unless the user has 'resolve petitions', in which case it goes through auto-approved.

In testing the above bug, I ran into some account desync, where clients wouldn't notice permission changes instantly. I've fixed that for next week.

Basically, this is all currently debug-tier. I apologise for the inconvenience. I'd appreciate your feedback on how you would like this stuff to work at some point.

Anonymous 07/25/16 (Mon) 01:43:01 a626c4 No.3268

File: 1469410983330.gif (759.1 KB, 160x160, 1:1, 139821f4d6741a2c441e728abf….gif)

>>3252

>You might be the first person to ever try denying a file petition!

Hah!

>I'll schedule some improvements.

Here was my train of thought, without documentation to guide me.

First guess:

get data = read access

post data = full write access

post petitions = moderated write access; to be used INSTEAD of post data. Anything the user submits must be approved

When I realized the user could not write at all without post data, my second guess:

post data = full write access

post data + post petitions = all submissions are marked as a petition

>unless the user has 'resolve petitions', in which case it goes through auto-approved.

I'm not informed enough to know if this will actually be easy; but perhaps you could implement my "second guess" in some fashion, to get granular user permissions with minimal effort.

Remove 'post data', for it will be redundant.

post +petitions = commits create a petition to add tag/file

post -petitions = commits create a petition to remove tag/file

resolve +petitions = self explanatory

resolve -petitions = self explanatory

Achieve the function of 'post data' by granting users 'post +petitions' & 'resolve +petitions' & 'post -petitions'

Where the functionality, as I understand it, is that anyone can freely add, but cannot freely remove files/tags.

>if you are actively managing a server

More specifically, it's me, the guy who keeps trying to use server.exe network functions on a smaller-than-intended scope xD

I convinced a small number of friends to use hydrus, and they are interested in having access to my library. They are also interested in helping me tag my library. Since this is still my personal collection, I want to manually approve any and all file/tag commits from them.

>In testing the above bug, I ran into some account desync

Yes, I noticed this, or something like it. When changing around account types, to get a user to notice the change I had to modify the user from the server administration, and apply the new account type.

>Basically, this is all currently debug-tier. I apologise for the inconvenience.

No worries; I assumed that, since you run a public server, this was all implemented and I lacked the knowledge to use it.

thanks!

hydrus_dev ## Board Owner 07/26/16 (Tue) 18:24:12 26412c No.3290

File: 1469557452641.jpg (1.79 MB, 1966x2691, 1966:2691, e48aa0a7864b34ef439a863ba3….jpg)

>>3268

Thank you for this feedback. I've created a new superjob to overhaul how permissions work. I will try to fold it into the next time I work on the network.

I've considered combining the file and tag repos and having individual content types selectable/permissable at the admin level, which might go along with this. It might make things more complicated clientside, though, so I'm still thinking about it.