windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v376/Hydrus.Network.376.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v376/Hydrus.Network.376.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v376/Hydrus.Network.376.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v376/Hydrus.Network.376.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v376.tar.gz
I had an ok to good week. Subscriptions run better and there are more UI fixes and basic theming for Qt.
subscriptions
Subscriptions have always been resource heavy. As last year's downloader overhaul extended their capabilities, we saw increased CPU lag with many subscription operations. I had hoped to completely fix this this week, even permitting subs with >250,000 items cached, but the surgery needed to figure this out was more complicated than I expected. While I otherwise prepped for this next step, I also tightened up subscription timing code and wrote a new scheduler that reduces overall subscription resource needs.
Subscriptions will now try to load when they are due, and only then. So, if a query in your sub is due to check in 45 minutes, or if it has file to download but has run out bandwidth for 45 minutes, it should now run, typically, in 45-60 minutes. They will also respond better to cancelling and pause events, and if only 30 of 100 queries are due for a check in this run, or have outstanding files to download, you will now see x/30 on the popup progress text and gauge, just as better indication of how much work remains.
Previously, subs checked every four hours. It meant a bunch of wasted CPU for subs that were not due, late checks for subs that were due just moments after the check time, and also a delayed 'clumping up' of work. Today's change is subtle but significant, so I am interested in feedback from all users of subs–if you notice anything odd, bad, or great, please let me know. Ideally, subs will be quieter for you, running more often when you are not looking.
Although the data overhaul was too much work to do in one week, I now have a good plan and am in better position to get it done. I really want subs to load and save their metadata in milliseconds so the manage subs dialog can launch equally quickly and the client api can start talking to them. The year is fading, so I am not sure when this will get done, but hopefully this week's scheduling will at least relieve some of the work involved.
Qt
Hydrus now has very simple support for Qt styles. Please check the new options->style page. All users are encouraged to try out the 'style' option, which should have one or more flavours for your OS and also 'Fusion'. These will change how buttons and other widgets are drawn across the program.
Advanced users may wish to play with 'stylesheet' support, which uses css/qss files to change colours and layouts using portable files. This is still prototype, just rough importing of stylesheets from a barebones folder in the install dir, and is likely to be replaced with something more user-friendly and technically sensible in the coming weeks. If you play with this, I am interested in areas where hydrus's old colour system conflicts foolishly with a qss stylesheet (such as some panel backgrounds and the green/red yes/no buttons on many dialogs), and also recommendations for stylesheets you would like bundled as future hydrus defaults.
The macOS build can now do page tab drag and drop again! Media drag and drops between tabs should also work. Everything should line up right and work like the Windows and Linux builds, but please let me know if it does not. A tip here, if you still have trouble with a centered page tab bar, is just to set the Fusion style as above, which will make hydrus look more like a standard PC application. You may prefer the macOS look, but if not, you now have the option.
Beyond a bunch of general bugfixes, I also improved the stability of some review bandwidth panels and some bad 'should I close/exit?' dialogs that were either not respecting 'no' answers or causing unusual behaviour as 'yes' closes were going on.
the rest
Pixiv tag search and twitter username lookup get updated default downloaders this week. Pixiv should be fixed, and twitter should work more efficiently, only getting 'media' tweets.
Client shutdown should now be faster and snappier for most users.
File export filename phrases now support a new '{file_id}' term that substitutes a unique permanent numerical identifier. If you need to create guaranteed-unique filenames, this is easier than using the '{hash}' term.
full list
- subscriptions:
- wrote a new subscription manager to better look after subscription scheduling
- rather than checking every four hours or after manage subs dialog close, subscriptions now record an indication of when they are next due for work, whether that is the estimated next check time or when bandwidth is free on remaining file downloads, and launch in a fifteen-minute window around that time. delays due to previous errors or user cancels are also taken into account. this reduces background cpu and i/o greatly for clients with large subs
- if a sub is paused, or all its queries are paused, it will now never be reloaded after first load until a change via the manage subs dialog
- furthermore, if a single sub takes a very long time to work, the whole sublist can re-cycle if they come up due for more work before it is finished
- if a sub query is DEAD but still has outstanding files to download, it will no longer automatically pause
- subs now clean up more tidily if they are running on a program exit
- the subscription popup now shows check/file progress based on the number of queries that appear to have pending work. instead of 'query 300/450' with 420 that aren't due, you'll get 'query 12/30'. if a query becomes due during a round of checking, another round of checking will run
- if a subscription fails to load from the db, the error is handled better and no more subs will run in that boot
- improved subscription startup checking logic, tightening up various paused/dead/cansync tests
- improved subscription interrupt checking logic, tightening checks on global network pause and various shutdown scenarios
- cleaned up some more subscription code in prep for data storage breakup
- .
- qt:
- added experimental Qt style settings to the new options->style page! all users should now be able to set Fusion style, and perhaps some alternate OS styles. advanced users are invited to play around with QSS stylesheets (although be warned that some of hydrus's custom colour system overrides QSS, so more work is needed here), which will be extended and made user-friendly in coming weeks
- fixed tab position calculations for all tab/media drag and drops for tab bars that are centered or otherwise positioned far off top-left alignment
- fixed tab drag and drop event object handling for macOS. tab and media DnD is now enabled for macOS
- the popup toaster can now unhide if an on-top-of-parent non-modal frame (like review services) is focused (so hitting 'process now' should show you the work)
- fixed a variety of old hacky wx close-window veto tests. the 'close client?' confirmation dialog will now reliably veto a close requent on 'no'/cancel, dialog close events that are vetoed (such as closing the manage tags dialog with pending tags) will now veto more than just the first time, and several bad media viewer archive/dupe filtering cancel and end-of-window events should now work more cleanly and correctly. users who had crashes at the end of filtering may find they are stable again
- as a quick patch against some multiline notes and statuses, list controls now force single-line text in all cells
- list controls now tooltip all cells
- fixed the shutdown splash not updating after the daemons shut down (lmao)
- 'modal' message dialogs, which are created by blocking maintenance tasks such as vacuum, will no longer raise the program to the foreground on creation
- should have fixed the taglist vertical positioning jank that could occur in the row after a tag with a tall emoji unicode character (and also sometimes kanji/hangul)
- fixed a typo bug that was throwing an error for the upnp port widget in the local client server management panel when 'allow non local connections' was checked
- improved stability of bandwidth review panel bandwidth rules refresh
- improved stability of review services bandwidth rules refresh
- improved some dialog cleanup code
- reverted a bad environment-setting change put in last week that was causing some running-from-source users trouble
- misc qt code cleanup
- .
- the rest:
- updated the default pixiv tag search downloader to one submitted by a user. it now uses their api
- updated the default twitter username lookup to a downloader submitted by a user. it fetches just the media tweet feed, making it more efficient. also added (but not linked by default) is a new tweet parser that can fetch most videos using a third-party site, advanced users may wish to play with this
- added a {file_id} term for file export phrases that substitutes a unique and permanent numerical file identifier
- fixed an issue where idle maintenance jobs could sometimes sneak in a few milliseconds of work during certain long shut down pauses, such as while waiting for a 'should I do shutdown work?' dialog to return. program shutdown should be snappier for many users as forced startup delays in these calls will no longer trigger
- added a date 'encode' string transformation rule, which takes an integer timestamp and converts it to a pretty date string. the date rules are now renamed to the clearer 'datestring to timestamp' and vice versa
- fixed page parser edit panel's 'test parse' when string transformations perform pre-parsing conversion. the handling and passing of test data for all the panels here is cleaned up throughout
- system:limit predicate edit panel now has a small label describing its sampling behaviour
- updated the various 8chan links in the client and help to 8kun, let me know if I missed any, and added Endchan bunker link to help menu
- improved some misc status text handling across the program
- refactored cache and manager code into different, simpler files
- updated sqlite on windows build to 3.30.1
next week
It is already December! For me, and I know for many of you, 2019 has just disappeared. Christmas is exactly three weeks away, so I am afraid some of my hopes for significant tag overhaul like tag siblings or processing filters may have to be delayed a little. I have a bit more Qt work to do, but also plenty of regular small jobs and bugfixes, so for next week I will take a simple bit of misc work, nothing huge, just to catch up on some things. I also have messages to catch up on.
If you were an 8chan user and missed the news, the /hydrus/ board is back on 8kun.top. The site is still unstable, and reception from the community is mixed, but against all odds they are back up and improving. I will keep posting release posts on Endchan and 8kun simultaneously for the time being, so please use whichever you feel better about, or just hit me on discord/tumblr/email if you prefer.
