windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v341/Hydrus.Network.341.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v341/Hydrus.Network.341.-.Windows.-.Installer.exe
os x
app: https://github.com/hydrusnetwork/hydrus/releases/download/v341/Hydrus.Network.341.-.OS.X.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v341/Hydrus.Network.341.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v341.tar.gz
I had a great week after being ill at the end of the previous week. There is a bit more than usual in this release because of the extra time. The Client API has some new commands, and there are a variety of fixes and stability improvements.
client api
The Client API now does some fun stuff. If you would like to play with one user's browser extension, which recognises hydrus-compatible URLs–including whether you have the page's file already–and lets you send them straight to hydrus for downloading, please check it out here:
https://gitgud.io/prkc/hydrus-companion
And the Client API help itself is here:
https://hydrusnetwork.github.io/hydrus/help/client_api.html
There is a little info at the top telling you how to turn the API on.
This release brings some more API commands. You can now edit a file's tags, edit a file's URLs, and the 'add_url' command now lets you attach tags for any files coming from the given URL and also lets you determine which page (by page name) the URL will go to (for instance if you have multiple watcher pages working on different check timings for different boards).
the rest
I spent some time this week cleaning things up. Repository update processing uses less memory and now regularly saves its progress on big jobs, reducing transaction bloat, improving error recovery, and giving control back to you much faster when it finishes. Memory use all around should be reduced.
I also replaced several areas of messy UI code, which has resulted in improved stability, particularly for Linux. Some CRITICAL logspam is gone, and I believe I have finally fixed a lingering ui crash on the duplicates processing page (at least for search distance 0–I will still work on >0, which seems to be a db-level problem for linux). Several boot crashes due to incorrect boot-error handling (such as when catching that some client_files directories are missing) are also fixed. I would appreciate your continued feedback here.
Duplicate processing now treats ratings copy/move/merge actions a little different: Before, it would only make a change if the 'destination' file had no rating set–now it will do it if it has no rating or if the rating is smaller than what the 'source' has. For instance, merging two files that have rating 2/5 and 3/5 will result in both files having rating 3/5.
After talking with a user about different ways of storing and presenting thumbnails, I put some work into a thumbnail drawing experiment that only needs one set of thumbs to draw on screen (the current thumbnail system stores two copies for different purposes). If you would like to try it temporarily, please turn it on under help->debug->gui actions->thumbnail experiment mode. Instead of using your 'resized' thumbs, it will only use the 'master'. It makes no changes to your file structure, and you can turn it off at any time. Try loading up some random files in your client and let me know if you get any errors or if any thumbs take an extremely long time to load. I've wanted to drop the two-thumb storage system for some time but never had good numbers to feel good about it, but my tests here are promising–the experiment mode only takes a few hundred µs more per initial thumb load for me, and if I were to switch over to it permanently, I could reduce it even further. Assuming there are no big problems here, I expect to chip away at this problem over the coming weeks and hopefully make a significant simplification in hydrus file storage.
full list
- client api:
- added /add_tags/add_tags, which does several kinds of tag content updates
- added /add_tags/clean_tags, which shows how hydrus will handle potential tags
- added /add_urls/associate_url, which allows you to associate urls with files
- added 'destination_page_name' to /add_urls/add_url, which will choose which destination watcher/url importer to place the url (or create a new one with that name)
- updated client api version to 2
- updated client help and unit tests for the above
- added a linked contents to the client api help
- improved some server error handling, mostly moving 403s to more correct 400s
- improved how missing parameter 400 errors are reported from the server vs deeper keyerrors that should be 500
- .
- the rest:
- tag repository update processing now saves progress to disk every million rows or every minute, whichever comes first. this reduces journaling bloat, improves recovery when the process quits unexpectedly, and makes for significantly faster cancel when requested by the user
- when processing duplicates and copying/merging/moving ratings, the 'source' file will now also overwrite the 'destination' file's rating if that destination rating is lower (previously, the rating would only go over if the dest had no rating set)
- added a new 'thumbnail experiment mode' under help->debug->gui. this will load fullsize thumbs and resize them in memory, please see release post for more details
- reduced menubar replacement flicker while, I believe, keeping and strengthening recent menubar indexing stability improvements
- the tag autocomplete dropdown will now always embed (instead of floating) in non-Windows
- when data seems non-decodable, the fallback encoding format is now that given by chardet, rather than utf-8
- improved serialisability of some pending tag data
- watchers can now hold and pass on fixed pending tag data
- gallery log objects can now hold and pass on fixed pending tag data
- file import objects can now hold and action fixed pending tag data
- hard drive imports now store their paths-to-tags info in this new format, directly in the file import objects
- improved some url-import page drop-target-selection logic
- improved error reporting when dropping/api-adding urls
- adjusted some url import workflow so big 'already in db' download lists should work a bit faster
- attempting to start the program with some external database files but not the main 'client.db/server.db' file will now cause a boot-fail exception with an explanation before any stub db files can be made
- tightened up some hydrus service login-capability-testing code that was previously stopping certain error states from recovering promptly, even on a force account refresh, while the service was maxed on bandwidth
- fixed a source of linux CRITICAL logspam related to several common dialogs
- improved ui stability on boot when file folders are missing (particularly for linux)
- improved stability for the various async tasks on the duplicates processing page, particularly for linux. I am not sure I got everything here, but it is definitely better
- did some more misc stability improvements, particularly in various boot fail scenarios
- completely removed an ancient and janky focus catcher widget from the main gui frame
- now various db caching is improved on the python side, removed a sqlite instruction to force temp information to always stay in memory–hot data returns to staying mostly in memory to start and then spools to disk if the transaction gets too large
- fixed approx bitrate sorting for malformed video files with explicitly '0' duration
- daemon_profile_mode now spams some more info about export folders
- fixed an issue that meant client db maintenance was firing its jobs too aggressively, regardless of idle status
- updated windows build to cv 4.0
- misc refactoring and fixes
next week
We are in the home stretch of the Client API now. I just want to write some simple file searching and it'll be done. I do not expect to be done in one week, but I think I should have something working for 342.
It felt good this week to clear out some long-time bugs doing code cleaning, so I'd like to focus on that as well.