Version 296

Name
Email
Subject
Comment *
File	Select/drop/paste files here
Password	(Randomized for file and post deletion; you may also set your own.)
Archive	Archive [500 char limit]
* = required field	[▶ Show post options & limits] Confused? See the FAQ.

Embed	(replaces files and can be used instead)
Voice recorder	Show voice recorder (the Stop button will be clickable 5 seconds after you press Record)
Options	Do not bump (you can also write sage in the email field)
Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf Max filesize is 16 MB. Max image dimensions are 15000 x 15000. You may upload 5 per post.

Version 296 hydrus_dev ## Board Owner 02/28/18 (Wed) 23:16:16 ca7a44 No.8169

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v296/Hydrus.Network.296.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v296/Hydrus.Network.296.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v296/Hydrus.Network.296.-.OS.X.-.App.dmg

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v296/Hydrus.Network.296.-.OS.X.-.Extract.only.tar.gz

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v296/Hydrus.Network.296.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v296.tar.gz

I had a great week. You can now edit the text on thumbnails, and there are some new file import options.

file import options

I have moved the 'allow decompression bombs' option from the global options dialog down to regular file import options. So, if you encounter a suspected decompression bomb that you would still like to import, you can now much more quickly just set the allow state on that queue and try again. Or if one artist you like tends to make big pngs that false-positive detect as decompression bombs, you can just allow them on their subscription and disallow them otherwise. This value will default to False, so please check the default file import options under options->importing if you prefer to always have it on.

Also, you can now set maximum file size and resolution in file import options. They both default to 'no limit'.

And I have added a specific 'maximum gif size' option. This defaults to 32MB and is an attempt to strike a blow against THE SFM COMMUNITY, which has a habit of duplicating webms into full-size gifs that can end up being 60MB, 150MB, or even higher, just for a few seconds of limited-colour janky-rendering animation. These gifs apparently have value for some users (on older phones, I understand), but they are almost completely useless for our purposes (especially as many downloads we do will get the much better webm/mp4 original in the same download queue). In any case, this option should stop the worst offenders without you having to pick through them. Feel free to set it even lower, although looking at my own stuff and asking other users suggests there are still some decent non-sfm-dupe gifs about 20MB. If you do want these files, feel free to set 'no limit' here, and please let me know what purpose you find in them.

tag summaries

I have written a new object, the 'tag summary generator', to allow user-editable tag 'banners' on thumbnails and the media viewer. Please check out the new 'tag summaries' options page to set what will display on the thumbnail top (which defaults to 'creator - series - title'), the thumbnail bottom-right (which defaults to 'volume-chapter-page'), and the media viewer top (which defaults to 'creator - series - title - volume - chapter - page').

I hope the ui here is clear. You choose the namespaces you want, set an optional prefix (like 'p' for pages), and set the separators to say how they are all joined up. There is a preview so you can see what you are making.

I expect to use these for file export filenames in future. Let me know how they work out for you.

misc

Some error popups non-Windows users were getting related to the tag autocomplete box should be fixed.

The client should be more aggressive about liberating memory and threads. Idle CPU, memory, and thread count should all be reduced.

I finished the 'formulae' downloader help. If you expect to write some downloaders in the new engine, please check it out here:

https://hydrusnetwork.github.io/hydrus/help/downloader_parsers.html

https://hydrusnetwork.github.io/hydrus/help/downloader_parsers_formulae.html

But I still have more to do. Anyway, let me know if you find anything confusing.

I also updated to a new version of wx, 4.0.1, for all platforms. It promises a little more stability. If you've been having any odd problems with your window managers that I haven't been able to figure out, please let me know if they are better or otherwise different in this release.

full list

- the 'allow decompression bombs' option is now moved to 'file import options'. it defaults to False

- file import options now allow max size and max resolution rules. they default to None

- file import options now allows a max gif size rule to deal with THE SFM COMMUNITY. it defaults to 32MB

- file imports will give better quality errors if they fail due to file import option exclusion rules

- file import fail errors will now also specify the incorrect amount–e.g. if a file is 64KB but the minimum limit is 256KB, it will state this in the error

- wrote a bunch of unit tests to make sure the new complicated file import options object works good

- finished the tag summary edit ui and added a new 'tag summaries' page to the options

- you can set the thumbnail top, thumbnail bottom right, and media viewer top summaries here, and everything should link up right!

- the old options to hide thumbnail top and thumbnail bottom right text are removed from the options->gui page

- tag summaries now include pending tags again

- tag summaries will order tags according to hydrus numeric tag sorting rules (0 < 0a < 0b < 1 < 9 < 10)

- tag summaries will collapse namespace groups that all start with decimals into a "min-max" phrase instead of the whole list (so 'collecting' a chapter will list p1-23 for the page phrase instead of each number in turn)

- fixed some potential error popups from the autocomplete dropdown when not in float mode (this hit non-windows users more often)

- split the parser help pages up a bit more and updated the client ui html links

- finished off the formulae help pages

- updated to wx 4.0.1 on all platforms. it promises increased stability!

- default tag import options are now moved to the 'importing' options panel, alongside the default file import options

- the database will deal with and recover from rollback exceptions more reliably. rollback-loops should be fixed, but the original event will now also produce a request to restart the client (which will safely reset everything) anyway

- improved how I create and layout the 'help for this panel →' help buttons

- the job scheduler now regularly checks for and clears out dead wx-related jobs (this saves a bit of memory)

- cleaned up an old tag retrieval method that was wasting CPU. some larger pages with many thousands of tags should do tag calculation after selection and so on a bit faster

- fixed a thread shutdown test that wasn't working until app shutdown time

- improved some more thread shutdown code

- deleted some old thread-inefficient calllater and wx-safe timer code

- emergency server errors that occur during the response writing stage should be handled better by the client

- emergency server errors are more robust in general serverside

- improved the quality of the errors thrown by the new network engine's login system

- improved some test timing and key press simulation code

- misc refactoring

- misc improvements

next week

I have a few more parsing help pages to finish before I get back to the downloader code properly. I would also like to look at some unusual subscription behaviour (mostly just not starting when told to) and maybe slip in something else slightly fun and new, like the tag summaries, this week.

____________________________

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/01/18 (Thu) 00:31:52 7813f4 No.8170

File: 5788e437194595c⋯.png (114.21 KB, 1123x2040, 1123:2040, explorer_2018-02-28_18-18-….png)

File: af5ff3d1c3922d6⋯.png (14.85 KB, 240x240, 1:1, client_2018-02-28_18-25-06.png)

File: d9c8b1542035c2c⋯.png (135.45 KB, 1803x1878, 601:626, client_2018-02-28_18-27-01.png)

File: 57c37e8bc7ac891⋯.png (120.55 KB, 1803x1878, 601:626, client_2018-02-28_18-27-55.png)

got a fun and annoying issue

long and short, got a 4k monitor, had the preview window set up to give me a very big image (around 1100 x 1700) and while it was minimized it decided no, every single open tab will now have this small ass preview image till you restart, right now the client is dealing with 5gb+ of ram and 200k images, so you can imagine what it takes to fully load it.

Is there any way to make the image preview default to a larger size if the screen is large? what is in this image is clearly a minimum size given a restore, the second image is the minimum window size possible

when I reopen the page it looks like the third image when it started out looking like the 4th image.

there is clearly a resize order and what preferences it has, personally I would rather have it prioritise the preview image over the tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/01/18 (Thu) 02:58:19 ce22f8 No.8172

Based Dev

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/01/18 (Thu) 05:07:18 ddee25 No.8173

>>8169

>I expect to use these for file export filenames in future. Let me know how they work out for you.

How exactly do I work the file export parameters? Let's say I want all files to export with whatever their original filename was; how do I get it to read the "original filename:" namespace and export the file with whatever comes after that?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/01/18 (Thu) 10:37:57 b1cb9c No.8174

How do I disable tag summaries? I prefer just thumbnails.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/01/18 (Thu) 17:43:10 ab1a9d No.8176

Ok, back again with that random subs freeze.

No errors pop up. It simply freezes when importing files from gelbooru using the tag "ao_usagi". It's the 444th image out of 476 that causes the freeze.

Feels like the software is tetraphobic.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/01/18 (Thu) 22:30:23 3e3d08 No.8180

Thanks a lot for this update my man I will most certainly be looking around the new download engine to see if I can be of use. One thing though, duplicate pags don't update tgemselves anymore after doing something. branges rebalancing nor duplicate searching

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/01/18 (Thu) 22:45:59 ad7ff4 No.8181

>>8169

Is it possible to use ratings in the summary generator? I'd like a star or something in the corner of a thumbnail

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/02/18 (Fri) 06:50:32 ff6d10 No.8182

File: f5bdfbdcba76c44⋯.png (27.72 KB, 152x127, 152:127, I he saddest.png)

File: b2bdd6bb45b90ac⋯.png (29.13 KB, 152x127, 152:127, the transparentest.png)

File: b63791bd36b72eb⋯.jpg (33.41 KB, 600x600, 1:1, tfw jpg artifacts.jpg)

>'banners' on thumbnails

Could these be made transparent so they don't block the images? If the visibility hit is too much you could have them become opaque when selected.

ideally with opacity adjustment for both modes but I know you're busy

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/02/18 (Fri) 18:18:51 18c02d No.8183

solid update.

longshot: any way we could get a more advanced set of tools for tag relationships?

e.g. for siblings:

red hair -> hair:red

typing this out for every variant of "___ hair" is cumbersome.

Imagining something using regex groups:

* hair -> hair:*

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

hydrus_dev ## Board Owner 03/03/18 (Sat) 21:57:26 ca7a44 No.8257

File: 8fdf4fe192d3c45⋯.jpg (347.48 KB, 682x1010, 341:505, 8fdf4fe192d3c450e73481113c….jpg)

>>8170

Thank you for this report. The preview window messing up is probably the longest-term bug I have. Trying to get the 'splitter' windows to have the correct gravity (how much of a resize event they consume) and boot coordinates can be a real headache. I tried to be clever, when I first wrote it–it saves the preview window size for the current page on client shutdown–but it messes up in too many instances. Since you've got a weighty client, telling you 'set the size good on one page then restart' isn't a great fix.

I will have a think about this this week and figure out a better fix. Maybe just an override in the options that says 'make the preview window 640x480' or explicitly override its gravity. Let me know how it works for you.

>>8173

It works off tags, so at the moment, you'd do something like select and right-click a bunch of thumbs->share->export->files and then setting a filename phrase of:

[filename]

Which would look for filename namespace tags and try to put those as the filename. So if your file has the tag 'filename:help_compute', then the file might get 'help_compute.jpg' as its ultimate name. Export folders use the same filename phrase system.

This phrase system has worked ok, but I want something richer and easier and more robust.

>>8174

Try editing the tag summary in the new options page and just removing all the namespaces. If the summary generated is an empty list, no banner will be displayed.

I'll probably add on/off checkboxes as well next week though, so you can also just wait for that.

>>8176

Thank you for this report. That file and its neighbours import fine for me, and they look ok (they aren't like 10,000x12,000 pngs that'd give render lag), so I suspect something else occured while your sub was processing. Was your machine idle at the time (so maybe some client maintenance routine might have kicked in for 30s?), or were you using the client while the sub was going on?

When it freezes, which part does it freeze on–'downloading file' or 'importing file' or a different step? Does the bottom-right of the client say 'db read locked' or 'db write locked' or is it blank? If you open task manager (Ctrl+Shift+Esc on Windows), is hydrus using a bunch of CPU/HDD? Are any other programs?

And if you leave it for a riduculous time, like ten minutes, does it eventually free up?

>>8180

Thank you for this report, I will check it out this week.

>>8181

This is a nice idea! I will write this down in my to-do.

>>8182

I think this is doable. I will check, and if it is, I'll add a slider.

>>8183

Yeah, I would love to add something like this. Unfortunately, the guts of the tag parent/sibling system is such a mess that I will have to put a whole bunch of time just fixing everything behind the scenes before I can add anything clever to it.

Another priority to add will be simple namespace switching, so you can convert all 'creator:' tags to be 'artist:' tags or whatever.

This is on my mind though. I think a database update to how siblings are stored and presented may be in the near future. The CPU weight of the whole system is starting to pile up.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/04/18 (Sun) 00:32:24 7813f4 No.8262

>>8257

If you are able to specify a size target, as in, lets say I want 1100x1700 and just get as close to that as you can, I think that would be perfect.

If its possible, having multiple size targets would be the best solution, for me right now, I have to much shit open and the client is laggy so I made the preview window fuckhuge so I can look at 39 images at a time between scrolls, and click the ones I want to add a rating too normally I would likely keep this around 600x600 - 600x900 and actually use it as a preview window rather then full image viewer.

On that note, I noticed that if I minimize the program to the smallest window size, then expand it, every single window is affected by this, regardless of focus. If there was a way to have every windows keep the same dimensions, this would also be a fix of sorts, not exactly the best it doesn't happen fix, but a good enough it cuts down the 'i have 108 pages that all need seperate re adjusting' annoyance of it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/04/18 (Sun) 02:01:14 61e7fa No.8263

Thank you for the good update. Regarding the thread watcher, could you look into adding support for 155chan.top? To be frank, I'm not knowledgeable about web design so I don't know how hard that would be. It also has two sister sites, 144chan.ws and 180chan.info, which both have the exact same url and website structure. Sample links here:

https://155chan.top/sw/res/6521.html

https://144chan.ws/sw/res/8496.html

https://180chan.info/sw/res/10628.html

Again, thanks for your great work; I can't wait for your next update!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/04/18 (Sun) 04:38:48 1039b0 No.8267

Eh? I can't find the 'tag summary' option.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/04/18 (Sun) 08:36:56 53004d No.8276

File: 213a1ae36d0a4a2⋯.png (40.57 KB, 1035x390, 69:26, ClipboardImage.png)

>>8267

it's in the options menu

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/04/18 (Sun) 17:41:19 6eecbd No.8279

File: 7d21864462d697c⋯.jpg (1.55 MB, 1430x2183, 1430:2183, 7d21864462d697cd5f1c70f69c….jpg)

Is it currently possible to store the post ID and the rating of an image (safe, questionable, explicit) when importing from the boorus?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/05/18 (Mon) 13:24:06 c06703 No.8285

>>8279

The ID should be in the source URL. Speaking of, I wonder if Hydrus will ever be able to get the source URLs from boorus that have them.

For ratings I use tag parents, but it's not perfect.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/05/18 (Mon) 19:56:05 3e3d08 No.8287

Just had a crash whilst using the duplicates filter, I just had my import folder import two new webms, and when I first updated (via the cog icon) one duplicates page, and after that a second one which was not in a page of pages but in the root page. just after pressing the update menu-item, and when the play icon on the search for duplicates had turned white and I pressed it, and then it searched. After completing the search it crashed, it did not yet show any new potential pairs, but when booting it again I now do see the four new potential pairs. I also have some thread watchers running in the background on 8chan and 4chan pages.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/06/18 (Tue) 13:06:00 7813f4 No.8291

Hey dev, I just had a thought. I have been going through some of the image options, there are some options that would make images look better as they zoomed in. while this is good for just browsing, would there be a way to have a seperate 'not adding anything to make this shit look better' option for the duplicate filter? I don't have time to really look into it and compare images, and it would be a bit difficult to do, but something tells me stripping anything that makes the images look better would be helpful for finding out the differences between them.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/06/18 (Tue) 15:40:27 b1cb9c No.8292

For files downloaded from HentaiFoundry with a HF "known url", before when you viewed them in the media viewer it said "Hentai Foundry url" (or similar) in the upper right corner of the screen. But since a couple of versions back it doesn't say anything there anymore.

I'm guessing maybe the urls don't match the "hentai foundry file page" url class? Here's a typical url from one of my subscriptions "http://www.hentai-foundry.com/pictures/user/Sparrow/579179/Assuka-Sucks". But the only difference I can see is the url class uses https.

That little line of text was useful when using the dupe filter…

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 03/07/18 (Wed) 17:58:33 18c02d No.8297

>>8257

>8183 again

another thought would be being able to paste them in more than 1 at a time–vim/emacs macros would make the repetition bearable, and I'm probably not alone in pre-staging them there to begin with so I don't forget any.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

hydrus_dev ## Board Owner 03/07/18 (Wed) 19:10:59 ca7a44 No.8299

>>8262

I've done a bit for this in today's v297 release. You can now save and spam preview size settings a bit easier–let me know how it works out and if you would like more fine control of specific pixel values or whatever. Setting multiple options might be easily doable–I'll check it next week.

In working on this this week, I was also surprised that hidden pages still do re-layouts on parent window size events. Unfortunately, I don't control that stuff at my level, but it better explains why having 100 windows open can make ui resizes laggy (especially a few weeks ago, when file and tag import options were embedded on every page). I will consider this in some other ways going forward.

>>8263

Thanks for these examples–I expect to write a generic html imageboard parser in the coming weeks for rarer/bunker chans, so I will see if I can wrap support for these into that and write some URL Classes to capture them for the thread watcher as well. I'll roll it all into a normal update, so it should just start working one week. Let me know how they work for you as I roll this out.

>>8279

>>8285

Not easily yet, but the ongoing downloader overhaul will expand our options greatly here. It'll be easy to add 'rating:safe'-type tags and I think we should be able to do known url parsing as well.

>>8287

Thank you for this report. I happen to have fixed a duplicate filter crash for today's release (as well as a fix for the page not refreshing after a dupe files maintenance job is done). Please let me know how v297 works for you.

>>8291

This is an excellent idea. I don't have much time to put into the duplicate filter workflow at the moment, but I will write this down for the next time I look at it.

>>8292

Thank you for this report. I have fixed this for today's v297 release–it was an issue with the order in which I was caching URL Classes for comparison. Let me know if it gives you any more trouble.

>>8297

Thanks. Adding a paste button is something I can do easy, although I'll have to be careful with the workflow if there are errors (which is the real terror of this whole issue–how to determine what to do on conflicts/loops/etc…)–I'll see if I can have it in for next week.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.