windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v398/Hydrus.Network.398.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v398/Hydrus.Network.398.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v398/Hydrus.Network.398.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v398/Hydrus.Network.398.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v398.tar.gz
I had a good work week. Tag autocomplete gets some new search options, and advanced users who make downloaders get some new text processing tools.
tag autocomplete
When I recently overhauled the tag autocomplete pipeline, I eliminated some unusual logical hoops where you could accidentally fire off expensive searches that would fetch all tags. Now the code is clean, I am adding them back in as real options.
The main thing here is that services->tag display is now services->tag display and search. It has several new options to change search based on what the autocomplete's current 'tag domain' is (i.e. what the button on the dropdown says, "all known tags" or "my tags" or whatever else). The options are available for every specific tag domain and the "all known tags" domain, and only apply there.
There are three new search options: You can have full namespace lookup, so an input of 'ser' also finds 'series:metroid' and all other series tags; you can have an explicit input of 'series:*' show all 'series' tags; and you can have '*' show all tags. These queries are extremely expensive for a large service like the public tag repository (they could take minutes to complete, and eat a ton of memory and CPU), but they may be appropriate for a smaller domain like "my tags". Please feel free to play with them.
There are also a couple of clever options setting how 'write' autocompletes (the ones that add tags, like in the manage tags dialog) start up, based on the tag service of the page they are on. You can set them to start with a different file or tag domain. Most users will be happy with the defaults, which is to stick with the current tag domain and "all known files", but if you want to change that (e.g. some users like to get suggestions for "my tags" from the PTR, or they don't want tag counts from files not in "my files"), you now can. The old option under options->tags that did the "all known files" replacement for all write autocompletes is now removed.
I have optimised the database autocomplete search code to work better with '*' 'get everything' queries. In the right situation, these searches can be very fast. This logic is new, the first time I have supported it properly, so let me know if you discover any bugs.
string processing
This is only important for advanced users who write downloaders atm. It will come to the filename tagging panel in future.
I am plugging the new String Processor today into all parsing formulae. Instead of the old double-buttons of String Match and String Converter, these are now merged into one button that can have any combination of ordered Matches and Converters, so if you want to filter after you convert, this is now easy. There is new UI to manage this and test string processing at every step.
The String Processor also provides the new String Splitter object, which takes a single string like '1,2,3' and lets you split it by something like ',' to create three strings [ '1', '2', '3' ]. So, if your HTML or JSON parsing provides you with a line with multiple things to parse, you should now be able to split, convert, and match it all, even if it is awkward, without voodoo regex hackery.
I also did some background work on improving how the parsing example/test data is propagated to different panels, and several bugs and missed connections are fixed. I will keep working here, with the ideal being that every test panel shows multiple test data, so if you are parsing fifty URLs, a String Processor working on them will show how all fifty are being converted, rather than the current system of typically just showing the first. After that, I will get to work on supporting proper multiline parsing so we can parse notes.
the rest
Double-clicking a page tab now lets you rename it!
system:time imported has some quick buttons for 'since 1/7/30 days ago'.
I cleaned out the last of the behind-the-scenes mouse shortcut hackery from the media viewer. Everything there now works on the new shortcuts system. There aren't many front-end changes here, but a neat thing is that clicking to focus an unfocused media window no longer activates the shortcut for that click! So, if you have an archive/delete filter, feel free to left-click it to activate it–it won't 'keep and move on' on that first click any more. I will continue to push on shortcuts in normal weekly work, adding mouse support to more things and adding more command types.
You can now enter percent-encoded characters into downloader queries. A couple of sites out there have tags with spaces, like '#simple background', which would normally be broken in hydrus into two tags [ '#simple', 'background' ]. You can now search for this with '#simple%20background' or '%23simple%20background'. Generally, if you are copy/pasting any percent-encoded query, it should now work in hydrus. The only proviso here is %25, which actually is %. If you paste this, it may work or not, all bets are off.
I am rolling out updated Gelbooru and Newgrounds parsers this week. Gelbooru searching should work again, and Newgrounds should now get static image art.
full list
- new tag search options:
- there are several new options for tag autocomplete under the newly renamed _services->tag display and search_:
- for 'manage tags'-style 'write' autocompletes, you can now set which file service and tag service each tag service page's autocomplete starts with (e.g. some users have wanted to say 'start my "my tags" service looking at "all known files" and "ptr"' to get more suggestions for "my tags" typing). the default is 'all known files' and the same tag service
- the old blanket 'show "all known files" in write autocompletes' option under _options->tags_ is removed
- you now can enable the following potentially very slow and expensive searches on a per-tag-domain basis:
- - you can permit namespace-autocompleting searches, so 'ser' also matches 'ser*:*', i.e. 'series:metroid' and every other series tag
- - you can permit 'namespace:*', fetching all tags for a namespace
- - you can permit '*', fetching all tags （╬ಠ益ಠ)
- '*' and 'namespace:*' wildcard searches are now significantly faster on smaller specific tag domains (i.e. not "all known tags")
- short explicit wildcard searches like "s*" now fire off that actual search, regardless of the 'exact match' character threshold
- queries in the form "*:xxx" are now replaced with "xxx" in logic and display
- improved the reliability of various search text definition logic to account for wildcard situations properly when doing quick-enter tag broadcast and so on
- fixed up autocomplete db search code for wildcard namespaces with "*" subtags
- simplified some autocomplete database search code
- .
- string processing:
- the new string processor is now live. all parsing formulae now use a string processor instead of the string match/transformer pair, with existing matches and transformers that do work being integrated into the new processor
- thus, all formulae parsing now supports the new string splitter object, which allows you to split '1,2,3' into ['1','2','3']
- all formulae panels now have the combined 'string processing' button, which launches a new edit panel and will grow in height to list all current processing steps
- the stringmatch panel now hides its controls when they are not relevent to the current match type. also, setting fixed match type (or, typically, mouse-scrolling past it), no longer resets min/max/example fields)
- the string conversion step edit panel now clearly separates the controls vs the test results
- improved button and summary labelling for string tools across the program
- some differences in labelling between string 'conversion' and 'transformation' are unified to 'conversion' across the program
- moved the test data used in parsing edit panels to its own object, and updated some of the handling to support passing up of multiple example texts
- the separation formula of a subsidiary page parser now loads with current test data
- the string processing panel loads with the current test data, and passes the first example string of the appropriate processing step to its sub-panels. this will be expanded in future to multiple example testing for each panel, and subsequently for note parsing, multiline testing
- added safety code and unit tests to test string processing for hex/base64 bytes outcomes. as a reminder, I expect to eliminate the bytes issue in future and just eat hashes as hex
- cleaned up a variety of string processing code
- misc improvements to string processing controls
- .
- the rest:
- double-clicking a page tab now opens up the rename dialog
- system:time imported now has quick buttons for 'since 1/7/30 days ago'
- all hydrus downloaders now accept percent-encoded characters in the query field, so if you are on a site that has tags with spaces, you can now enter a query like "simple%20background red%20hair" to get the input you want. you can also generally now paste encoded queries from your address bar into hydrus and they should work, with the only proviso being "%25", which is "%", when all bets are off
- duplicates shut down work (both tree rebalancing and dupe searching) now quickly obeys the 'cancel shutdown work' splash button
- fixed a signal cleanup bug that meant some media windows in the preview viewer were hanging on to and multiplying a 'launch media' signal and a shortcut handler, which meant double-clicking on the preview viewer successively on a page would result in multiple media window launches
- fixed an issue opening the manage parsers dialog for users with certain unusual parsers
- fixed the 'hide the preview window' setting for the new page layout method
- updated the default gelbooru gallery page parser to fix gelb gallery parsing
- updated the newgrounds parser to the latest on the github. it should support static image art now
- if automatic vacuum is disabled in the client, forced vacuum is no longer prohibited
- updated cloudscraper for all builds to 1.2.38
- .
- boring code cleanup:
- all final mouse event processing hackey is removed from the media viewers, and the shortcut system is now fully responsible. left click (now with no or any modifier) is still hardcoded to do drag but does not interfere with other mapped left-click actions
- the duplicates filter no longer hardcodes mouse wheel to navigate–whatever is set for the normal browser, it now obeys
- cleaned up some mouse move tracking code
- clicking to focus an unfocused media viewer window will now not trigger the associated click action, so you can now click on archive/delete filters without moving on!
- the red/green on/off buttons on the autocomplete dropdown are updated from the old wx pubsub to Qt signalling
- updated wx hacks to proper Qt event processing for splash window, mouse move events in the media viewer and the animation scanbar
- cleaned up how some event filtering and other processing propagates in the media viewer
- deleted some old unused mouse show/hide media viewer code
- did some more python imports cleanup
- cleaned up some unit test selection code
- refactored the media code to a new directory module
- refactored the media result and media result cache code to their own files
- refactored some qt colour functions from core to gui module
- misc code cleanup
next week
I will be taking my week vacation after next week, and I don't want to accidentally create any big problems for the break, so I will try to mostly do small cleanup work and bug fixes.
