[ / / / / / / / / / ] [ dir / cute / egy / fur / kind / kpop / miku / waifuist / wooo ]

/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Catalog

Name
Email
Subject
Comment *
File
* = required field[▶ Show post options & limits]
Confused? See the FAQ.
Embed
(replaces files and can be used instead)
Options
Password (For file and post deletion.)

Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf
Max filesize is 12 MB.
Max image dimensions are 10000 x 10000.
You may upload 5 per post.


New user? Start here ---> http://hydrusnetwork.github.io/hydrus/

Current to-do list has: 714 items

Current big job: finishing off duplicate search/filtering workflow


YouTube embed. Click thumbnail to play.

718fbd No.4856

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v239/Hydrus.Network.239.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v239/Hydrus.Network.239.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v239/Hydrus.Network.239.-.OS.X.-.App.dmg

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v239/Hydrus.Network.239.-.OS.X.-.Extract.only.tar.gz

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v239/Hydrus.Network.239.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v239.tar.gz

I had a really great week. A lot of different things are fixed, updated, or do something new.

duplicate files improvements

I have finished some 'similar files' maintenance code and added it to the normal idle/shutdown maintenance routines. You'll see it pop up every now and then–I wouldn't expect more than a couple of minutes a week for most users. It just keeps your searches fast after you import or delete a lot of files.

I also improved the search tree generation algorithm, so searches now operate ~10-40% faster. This better tree will come in branch by branch as the maintenance routine works, but if you currently do a lot of similar files searches, you can get the full benefit now by forcing a full regen with database->regenerate->similar files search data.

And the normal thumbnail right-click menu now has four types of similar search, depending on exactly how similar a match you are looking for. The more exact the search, the faster it will run. The 'similar' entry is good for most 'have I got this file already?' purposes.

hydrus network is now https

This only affects hydrus clients talking to hydrus servers like my public tag repository. Hydrus client continue to talk to normal websites with http or https depending on how their url starts.

As I mentioned in my v236 post, I have now flipped the switch on https for the hydrus server code, and I updated my public tag repository server today. If your client is at least v236, you do not have to do anything–it will naturally upgrade the connection to https when it talks to a v239-or-later hydrus server.

If you have a v235 or earlier client, it will no longer be able to talk to my PTR server! You will likely get ugly and unhelpful error messages, too. If you want to sync with an updated server, you will have to update your client.

If you run a hydrus server, the grace period to update to >=239 will be another three weeks. In v242, I will remove the old http support from the client.

misc stuff

Some highlights from the rest of the changelog, which this week is not brief:

I've fixed the more complicated lists, like in manage subscriptions, to sort by the underlying data rather than the presentation text, so '2 hours ago' is now correctly sorted before '1 day ago'.

Export and import folders also got this treatment, and while I was at it, I added name support to export folders. As the path is no longer the unique identifier of an export folder, you can now have multiple exports to the same path.

All the system predicate panels have had a makeover. Mostly, the old dropdowns have been replaced with quicker radio buttons.

I've split the database->maintenance menu to declutter it and help avoid costly misclicks.

full list

- finished up similar files search data maintenance code

- similar files search data maintenance will now run during idle time

- similar files search data maintenance can be called from database->maintain menu

- the crowded database->maintenance menu is now split into maintain, regenerate, check

- improved the similar files tree generation code, speeding searches significantly

- wrote a new listctrl class to handle more complicated objects and also sort by underlying data

- the new listctrl now handles object name non-duplication

- cleaned a bunch of crap old listctrl code

- manage export folders now uses the new listctrl

- manage import folders now uses the new listctrl

- manage subs now uses the new listctrl

- manage scripts now uses the new listctrl

- options media viewer options now uses the new listctrl

- file import status panel now uses the new listctrl

- the new listctrl can now quickly fetch item index from the underlying object

- the file import status panel should now cope with extremely huge lists a bit better now!

- multiple parsing child nodes can now import from/export to clipboard as lists

- export folders now have names, so you can have multiple export folders pointing to the same path! existing export folders will get their path as their name, but this can be changed no prob

- cleaned import/export folder dialogs

- cleaned import/export folder dialog workflow

- several misc import/export folder improvements

- hydrus servers are now exclusively https with self-signed certificates

- hydrus servers now create server.crt and server.key in their db folders for SSL–these files will be backed up along with everything else on an admin backup command

- system:hash now ignores the file domain and any other predicate. it now returns very quickly, no matter the context

- improved system:hash search logic

- all the awkward choice dropdowns in system predicate panels are replaced with radioboxes

- improved system:rating panel grid layout

- wrote a better subclass of radiobox to handle more data

- moved first half of thumbnail menu to new menu system

- cleaned up a little thumbnail menu logic

- improved the different ways services are added to thumbnail menu

- thumbnail 'select' menu is logically cleaned up and allows for better file domain selection

- the thumbnail menu's copy files and copy hashes to clipboard will now send them ordered (they were previously pseudo-random)

- added 'paths' to the share->copy thumbnail menu for copying multiple files' paths. these are also ordered.

- if the popup message manager does not have any errors, it will no longer unhide (which can annoyingly raise the main gui window) when the gui window does not have focus

- removed some old redundant error reporting stuff in popups

- improved and quietened some some mime detection failure code, sped up mime failure loop in all cases

- massively simplified and atomised how new serialisable-object management panels can save their data

- the manage subscription dialog now saves in a single, faster transaction

- the manage script dialog now saves in a single, faster transaction

- reduced redundant index work from analyze jobs

- improved tag parse error handling

- fixed media removal rules when deleting from the 'all local files' domain

- polished and clarified some of the help's tag schema

- misc cleanup

- more misc cleanup

- misc refactoring

- more misc refactoring

next week

I am just this afternoon going down with something, but if I'm not in bed all week, I'll write some code to start automatically detecting pairs of similar files, getting ready for the gui workflow I'll write to deal with them all.

There's also a maintenance tweak I'm pretty sure I need to make to speed up newly-ptr-synced users' dbs. If you recently synced with the ptr and things are running much slower than before, please run database->maintain->analyze->full and let me know if that speeds things up. It may take ten or twenty minutes to run.

284fc2 No.4860

Thanks, finally made the update from v233 and I have to say the new subscriptions interface is really useful.


657b23 No.4862

Boss, could you elaborate what each maintenance option does exactly?


50dc7a No.4863

Take vitamins, make sure to wash your hands and don't trim your nose hair too short. If someone is coughing near you, shove them.

Feel better Hydrus, patch the bug in the developer.


718fbd No.4874

>>4862

The new menu system requires me to add a status bar description for all menu entries. As I replace the old menus, you should get better descriptions in the status bar of the main gui when you hover over each entry. I believe this is true of the maintenance menu, and they'll all also give yes/no dialog windows with longer explanations when you click on them. Don't be afraid to click on them.

Let me know if any of those explanations are confusing or not enough.


60e1e9 No.4875

I think this is new with this version. It comes up when trying to copy the external share url:

WindowsError

[Error 2] The system cannot find the file specified

File "include\ClientGUIDialogs.py", line 1460, in EventCopyExternalShareURL

external_ip = HydrusNATPunch.GetExternalIP() # eventually check for optional host replacement here

File "include\HydrusNATPunch.py", line 34, in GetExternalIP

p = subprocess.Popen( cmd, stdin = subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.PIPE, startupinfo = HydrusData.GetSubprocessStartupInfo() )

File "subprocess.py", line 711, in init

File "subprocess.py", line 959, in _execute_child


718fbd No.4876

>>4875

In your install folder, do you have install_dir/bin/upnpc_win32.exe? That's the file being called there. It is a tool for UPnP stuff with your router–in your case the client needs to ask your router what the external ip is.

Some virus scanners block it accidentally because trojans and whatever use similar code. Maybe your virus scanner quarantined it?

A new copy is in any release, so just download/extract again if you need to track down a new version.


60e1e9 No.4877

>>4876

This was indeed the issue. I thought I had already set the folder as white listed.


1331ff No.4886

Are there any plans to add a night mode or any basic theming of the window? I'm using the Windows build.

And the ratings in the full screen view in top-right, there's no text displaying to show what each label is for me, not sure if this is a bug.

Otherwise, thank you so much, such a great program.


a89ca1 No.4888

File: f79f3f6f01c6d97⋯.jpg (15.02 KB, 361x161, 361:161, error.JPG)

I may have updated from some ancient release to v239 and now I am getting a "Fatal Error! Failed to execute pyi_rth_win32comgenpy" on startup and it won't run. Pic related. How fucked am I?


a89ca1 No.4890

>>4888

More specifically whatever release I was on before was at least one and a half years old. I never thought to update it until I started getting errors.


60ff55 No.4891


a89ca1 No.4892

>>4888

>>4890

Solved by deleting everything in the install directory except the db folder, trying to install the latest release, and then following the python error messages backward. Updated to 183>228>239, database is intact and my tabs are even still open. That is a well-designed piece of software. Thanks anyway!


8d2f37 No.4895

>>4856

Hey. First of all I want to say thank you for this wonderful program. In the new version featuring a piece to find not only exact copies, but simply similar images. That's cool. But there is a small bug with the menu. Webmrelated.

https://webmshare.com/0rPZK

If I come to the point of "find similar files", but did not choose, and close it by clicking the mouse in any other place, it comes out that's the error.

PyAssertionError
C++ assertion "m_menuDepth > 0" failed at ..\..\src\msw\toplevel.cpp(1544) in wxTopLevelWindowMSW::DoSendMenuOpenCloseEvent(): No open menus?
File "include\ClientGUIMedia.py", line 3039, in EventShowMenu
HydrusGlobals.client_controller.PopupMenu( self, menu )
File "include\ClientController.py", line 770, in PopupMenu
window.PopupMenu( menu )
File "site-packages\wx-3.0-msw\wx\_core.py", line 11144, in PopupMenu

Actually not especially important, just report about bugs.


6c283c No.4904

File: cc815d44df8ed7a⋯.jpg (59.25 KB, 685x638, 685:638, cc815d44df8ed7ac85ab818957….jpg)

>>4856

Thanks for adding the ordered paths and improving import speed. It is much comfier to use now.

I have a noticed a few bugs with file services.

When I try to commit new files to a file repository I get the following error:


AttributeError
'module' object has no attribute 'WSAEACCES'
Traceback (most recent call last):
File "/opt/hydrus/include/HydrusThreading.py", line 276, in run
callable( *args, **kwargs )
File "/opt/hydrus/include/ClientGUI.py", line 2468, in _THREADUploadPending
service.Request( HC.POST, 'file', { 'file' : file } )
File "/opt/hydrus/include/ClientData.py", line 1772, in Request
( response, size_of_response, response_headers, cookies ) = HydrusGlobals.client_controller.DoHTTP( method, url, request_headers, body, report_hooks = report_hooks, temp_path = temp_path, hydrus_network = True )
File "/opt/hydrus/include/ClientController.py", line 389, in DoHTTP
def DoHTTP( self, *args, **kwargs ): return self._http.Request( *args, **kwargs )
File "/opt/hydrus/include/ClientNetworking.py", line 347, in Request
( response, size_of_response, response_headers, cookies ) = self._DoRequest( method, location, path, query, request_headers, body, follow_redirects = follow_redirects, report_hooks = report_hooks, temp_path = temp_path, hydrus_network = hydrus_network )
File "/opt/hydrus/include/ClientNetworking.py", line 273, in _DoRequest
( parsed_response, redirect_info, size_of_response, response_headers, cookies ) = connection.Request( method, path_and_query, request_headers, body, report_hooks = report_hooks, temp_path = temp_path )
File "/opt/hydrus/include/ClientNetworking.py", line 900, in Request
( response, parsed_response, size_of_response ) = self._SendRequestGetResponse( method, path_and_query, request_headers, body, report_hooks = report_hooks, temp_path = temp_path )
File "/opt/hydrus/include/ClientNetworking.py", line 529, in _SendRequestGetResponse
( response, attempt_number ) = self._GetInitialResponse( method, path_and_query, request_headers, body, attempt_number = attempt_number )
File "/opt/hydrus/include/ClientNetworking.py", line 589, in _GetInitialResponse
if e.errno == errno.WSAEACCES:
AttributeError: 'module' object has no attribute 'WSAEACCES'

It seems to affect only new uploads since I have no problem to add & sync new repository or petition existing files.

It starts working if I press the 'refresh account' button in 'review services' window or if I commit petitions for some files first. This looks like a new bug in v239.

There are also some errors with file permissions when removing remote file / tag repository:


2017/01/06 18:46:27: Trying to delete /media/hydrus/client_updates/53f4148070f4ae16e2786da0ba33e9548ee3d8044209b10bff053bd0b00001bd caused the following error:
2017/01/06 18:46:27: Exception:
2017/01/06 18:46:27: OSError
[Errno 13] Prístup odmietnutý: '/media/hydrus/client_updates/53f4148070f4ae16e2786da0ba33e9548ee3d8044209b10bff053bd0b00001bd/1483621632_metadata.json'
Traceback (most recent call last):
File "/opt/hydrus/include/HydrusPaths.py", line 193, in DeletePath
shutil.rmtree( path )
File "/usr/lib/python2.7/shutil.py", line 252, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File "/usr/lib/python2.7/shutil.py", line 250, in rmtree
os.remove(fullname)
OSError: [Errno 13] Prístup odmietnutý: '/media/hydrus/client_updates/53f4148070f4ae16e2786da0ba33e9548ee3d8044209b10bff053bd0b00001bd/1483621632_metadata.json'

2017/01/06 18:46:27: /opt/hydrus/include/ClientGUICommon.py:3810: UnicodeWarning: Unicode unequal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
if self._tb_text.GetLabelText() != text:

It probably removes the access permission (x) for the directory somewhere in the code. It also affects v238 so it's probably an older bug.

Speaking of file permissions the new https public/private keys generated by server have very generous file permissions in Linux which could potentially allow read access to any user of that system. Not exactly a good thing in multi-user environment. It is good practice to set the permissions to 400 to allow read-only access only to the file owner. Ssh for example completely ignores private keys with too open permissions.

System: Arch Linux, x86_64, using AUR package


b7778c No.4905

>>4856

Trying to install this for the first time, Avast tells me that it contains Malware after the download reaches 40MB for both the .zip and .exe windows files. Is it a false positive?


575f63 No.4907

>>4905

Yes it is.


f4edc6 No.4909

Keep up the godly work, anon.


718fbd No.4910

File: 06f097623e83a61⋯.jpg (618.17 KB, 1024x1884, 256:471, 06f097623e83a61de69f188093….jpg)

>>4886

Other than the colours in file->options->colours, I pull everything from system defaults. Adding a night mode to automatically shade everything or similar is on my todo list, although you may have more luck in the meantime with an OS-level night-mode that hydrus can inherit from, or even something like f.lux that might do it better without my intervention.

>>4888

>>4890

>>4892

Great. You hit the 32->64bit changeover I did I think around v186. The solution then was to do a clean install just as you figured out. As all releases are 64bit now, I don't expect any more so traumatic updates in the future.

>>4891

The github is a mirror of my home work environment, so my workflow doesn't really support sucking code back from it. I'm happy to make very similar changes on my end to make packaging an app easier, though–can you describe exactly what USERDATA_DIR and LICENSE_DIR represent, so I can double-check my new db_dir stuff and whatever else doesn't mess any of this up for you?

>>4895

Thanks for this. I get similar, on any nested menu, but only sometimes. This bug is coming from an area of code I don't think I have control over. As it doesn't always happen, I suspect it may be an afteraffect caused by the other longstanding menu id overflow bug. I am slowly replacing all that bad menu code, so I'm going to see if this menuDepth > 0 one goes away with that.

>>4904

Thank you for these detailed reports. I will check this stuff today and update the permissions issue.

>>4905

This happens from time to time. It is often upnpc_win32.exe throwing the false positive–which I use to do UPnP stuff at the user's request–but I'd be interested to know if you get anything different.

>>4909

𝕯𝖊𝖚𝖘 𝖛𝖚𝖑𝖙, 𝖔𝖕𝖚𝖘


1331ff No.4916

File: bf8b3ccb78093e0⋯.jpg (35.9 KB, 553x463, 553:463, hydrus.jpg)

>>4910

re: colors, in the end I did go with an OS-level custom styling, which looks fantastic.

Thank you again


60ff55 No.4939

>>4910

>>4891 here, USERDATA_DIR just represents the directory that hydrus writes to (which appears to just be the database) and LICENSE_DIR is just the directory containing the license file (which I probably should have made into just a path to the file.)


718fbd No.4946

File: 70cfb836ff7c645⋯.jpg (412.43 KB, 1600x1318, 800:659, 70cfb836ff7c645a321054f933….jpg)

>>4939

I've made a similar change as your pull request for tomorrow's release. There's now LICENSE_PATH as you suggested and I've made a single DEFAULT_DB_DR, which is base_dir/db (i.e. install_path/db). If you want to change where a fresh client writes its stuff by default, you can set DEFAULT_DB_DIR to something like os.path.expanduser( '~/.hydrus_network' ), and I think you'll get what you want. Hydrus will try to create the directory if it doesn't exist. LICENSE_PATH is install_dir/license.txt by default. Let me know if I messed it up or misunderstood what you wanted to do here.

I haven't made all the changes you did as the other stuff needs to refer to the install path explicitly. Some of it is the update code that converts from the old fixed-under-install_dir system to the new one that can run the db from anywhere.

Note also there is the -d launch parameter. You could create a shortcut in your package that goes:

./client -d "muh_db_path"

And it'll also work.




[Return][Go to top][Catalog][Post a Reply]
Delete Post [ ]
[]
[ / / / / / / / / / ] [ dir / cute / egy / fur / kind / kpop / miku / waifuist / wooo ]