/hydrus/ -Version 336

Name
Email
Subject	REC STOP
Comment *
File	Select/drop/paste files here
Password	(Randomized for file and post deletion; you may also set your own.)
Archive	Archive [500 char limit]
* = required field	[▶Show post options & limits] Confused? See the FAQ.

Embed	(replaces files and can be used instead)
Options	Do not bump (you can also write sage in the email field)
Allowed file types:jpg, jpeg, gif, png, webp,webm, mp4, mov, swf, pdf Max filesize is16 MB. Max image dimensions are15000 x15000. You may upload5 per post.

YouTube embed. Click thumbnail to play.

Version 336 hydrus_dev ## Board Owner 01/16/19 (Wed) 23:07:56 06617c No.11355

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v336/Hydrus.Network.336.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v336/Hydrus.Network.336.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v336/Hydrus.Network.336.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v336/Hydrus.Network.336.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v336.tar.gz

Last week, v335, was an important update. If you are updating from v334 or earlier, please go read that release post for your special one-time update instructions and update to v335 before you try this week. Once you have updated to v335, updating to anything newer is back to normal.

I had a good week. A bunch of py3-related bugs are fixed, and I improved some heavy-load throttling.

py3 fixes

Overall, last week's update went well. CPU and memory use and overall jank is down for all users, and afaik there weren't any permanent disasters in the update. This was a big relief for me.

That said, as expected, several unusual bugs related to py3's new unicode handling slipped through. Thank you for the reports. A common one was related to windows logging, another was in pixiv searching with kanji tags, another with videos with unicode metadata. I believe I have cleaned most if not all of it up. There may be a couple more out there, so please let me know if you get anything new. Otherwise, please resume any broken subs or retry any broken webm imports.

I also fixed the numerical tag ui, which was coercing all clicks to to zero or max stars, and which I messed up by accident!

better heavy workflow

The client now has a strict thread-level throttle on the number of importers that can work at once. It seperates the different sub-parts into their own max limits, but the overall total limit is about 20-40. This is in an effort to stop ui hang after accidental heavy import situations such as pasting 100+ queries into a gallery downloader. This is a 1.0 throttle and doesn't have any ui to report 'waiting on a slot' or anything. I don't expect 99% of users will even notice it, but if it affects you seriously, please let me know.

In a related note, for advanced users, gallery downloaders now have the ability to 'bundle' mass-pastes into one downloader, queueing up all the different gallery queries sequentially in the same object (rather than in many in parallel, which is what previously cluttered the ui and boshed the CPU). If you sometimes paste a bunch of single-file md5-queries at once, please check this new check menu item under the gallery downloader's cog menu.

full list

- fixed an issue where the numerical rating control was coercing all clicks to either the minimum or maximum allowable rating (e.g. 3/5 stars wasn't working)

- fixed some text file and process i/o, which was handling some unicode decoding/encoding incorrectly. it now mandates utf-8 in all cases

- fixed a referral url encoding problem that was stopping pixiv from downloading when the gallery page url had kana/kanji characters (from a search term)

- fixed a str vs bytes issue when loading the filename tagging panel

- fixed the delete button on the filename tagging quick namespaces panel (the edit and delete buttons are also now 'live' and will disable when nothing is selected)

- improved some json dump deserialisation code

- fixed a data-sorting issue that would appear with certain parsers in the edit parsers panel

- improved video metadata parsing, fixing an issue when the video has a 'title' row containing inconvenient data

- fixed some hex character processing for system preds

- added an advanced check item to the gallery downloader cog icon menu that will 'bundle' multiple query-pastes to the same single gallery downloader (this is helpful if you are pasting a whole bunch of md5 queries in one go and would rather one downloader work through them sequentially than 50+ separate ones blat your CPU simultaneously)

- the different kinds of importer worker threads now have several limits on the max number that can be working at once, to stop accidental ui overload when a hundred or more are in memory and all want to work at once (like after a big paste event or resuming after computer sleep). during periods of heavy import activity, the importers will now naturally space themselves out to smooth out the spike. the limits are hardcoded for now, let me know if it noticeably bottlenecks your situation

- made some menubar update code a bit less complicated and reduced how often it'll spam during heavy update

- the 'what to do?' buttons that appear in manage tags sometimes on a tag action got a simplification pass and are now on the new dialog system

- simplified my new dialog code significantly, clearing out redundant code and classes and pushing all okable/cancellable/vetoable closing checks through one single method

- wrote some new help.txt in the db dir about hanging startups

next week

I still have a bunch of small stuff going on, but I'd like to start on the Client API, the next 'big thing', as well. I expect this will involve several weeks of ui and clientside server prep before anything exciting happens.

____________________________

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 01/19/19 (Sat) 06:28:41 d0d841 No.11376

Finished updating from v334>v335>v336 and the client runs so much faster now, holy shit.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 01/19/19 (Sat) 08:34:31 bbad73 No.11377

File: 1ef5174aedae52c⋯.jpg (194.43 KB,1344x630,32:15,45745747.jpg)

>>11355

Its me again from the issue about my anti-virus slowing down Hydrus. Just updated to v366 and its even worse now than before. Kaspersky is going nuts over this version and I have no option to ignore the message on the bottom right. I can try to ignore some of this stuff on the left though and see how that turns out. Worse case, I may need to send a copy of Hydrus to their anti-virus labs and beg for them to white list Hydrus or something.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 01/19/19 (Sat) 08:37:24 bbad73 No.11378

>>11377

Also forgot to mention that everything from 3AM onward is from everything related to Hydrus.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 01/19/19 (Sat) 09:53:02 bbad73 No.11379

>>11355

>>11377

>>11378

Welp looks like can't even use Hydrus anymore, Kaspersky keeps deleting the client.exe no matter what I whitelist, fucking shit. It just seems to be the client.exe that's triggering my anti virus while its giving me the same list of detected objects as the pic posted earlier.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 01/19/19 (Sat) 12:55:30 5560d6 No.11381

>>11355

Who did this (thanks anyway) https://alternativeto.net/software/hydrus/ be sure to upvote this all the way <3

P.S. someone please make a Hydrus page for https://www.slant.co/ and https://www.producthunt.com/ please

(maybe https://software.informer.com/ https://www.openhub.net/ or http://www.alternative.to/ or https://appcrawlr.com/)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 01/19/19 (Sat) 16:40:34 1437b9 No.11388

Guy that had dialog issues: Began running from source as suggested; stability issues have completely disappeared, and the UI fits my OS. Thanks for the help.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 01/19/19 (Sat) 20:12:24 d3a2d0 No.11389

>>11377

>>11378

>>11379

My Webroot spazzes about Hydrus every update, and will select 1-2 components to highlight as "omg virus" for each daily scan. The most retarded part is that even if I whitelist them all, after a few days it'll go back to flagging them again. And of course any client update resets all the whitelists.

Can't believe Kaspersky is worse. Literally nigger tier.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

hydrus_dev ## Board Owner 01/19/19 (Sat) 20:37:25 06617c No.11390

>>11378

>>11377

>>11379

Thank you for this update. I just heard from another user with Kaspersky problems this week. I have heard from some users before that their anti-virus was intercepting files the installer/extractor wanted to add as they were being unpacked in the OS's temp folder. Hence a whitelist on the installer file or destination folder wasn't quite doing it.

It sounds really ugly to say, but you may have luck disabling the anti-virus completely for the 60 seconds it takes to extract/install and then coming back up with the install dir completely whitelisted. I know that has worked for other users.

I wonder what Trojan Generic really means? Maybe the combination of sqlite and network dlls in there? There's some folder scanning code of course, for the file importing workflow, but afaik I don't do any 'gimme the content of system32 m8' kind of calls anywhere. A quick search suggested a beta of OpenOffice once caused the same flag. I presume it is a catch-all definition. Ironic that it would say Win32, since afaik everything is now 64. Maybe the InnoSetup installer is 32-bit.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

hydrus_dev ## Board Owner 01/19/19 (Sat) 20:40:58 06617c No.11391

>>11376

>>11388

Thanks for letting me know lads. I am really pleased we are finally boshing some of these long-term issues on the head. Py3 is working out, and running from source isn't the huge headache it used to be.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 01/19/19 (Sat) 21:06:49 bbad73 No.11392

>>11390

I'll try asking on their antivirus lab if they can look at Hydrus then. I was doing some looking around on their site and found this whitelisting project where people are trying to get more legit programs to be whitelisted since trying to run legit software has become harder and harder over the years. I guess they're trying to create a database of trusted software or something.

whitelisting.kaspersky.com/whitelist_program

I haven't really looked too much into this as I'm not really sure if this is for ALL antivirus or just for kaspersky but if its the later I wouldn't bother but if its for all antivirus, it might be something to look into. It doesn't seem like something a normal user can use and from what little I've seen, it looks like its asking for dev's to sign up. I don't know, I'll ask on their forums to take a look.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 01/19/19 (Sat) 21:43:17 08bd95 No.11396

On osx, is there a way to specify the location of your db folder instead of just putting it in the hydrus folder in your library? I run 2 copies of hydrus each with different files and I would like to be able to give a path to each hyrdrus telling them to look in two separate folders in my documents folder as to not mix them up.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

hydrus_dev ## Board Owner 01/19/19 (Sat) 22:02:58 06617c No.11402

>>11396

There isn't, but I am open to it. I don't know much at all about OS X, so how might you like this to happen? Is it possible to launch an App with a command switch (like -d="muh_db_location"), or set up some sort of shortcut that does the same thing? Or is the practical answer here some kind of 'on program launch, recognise multiple installs/profiles and allow user selection'?

If I added some hackery-dackery-doo where you put in some 'db_location' file to ~/Library/Hydrus that redirected your install to multiple locations somehow, would that do the trick?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 01/19/19 (Sat) 23:03:06 08bd95 No.11403

>>11402

I think the latter hackery-dackery-doo would do. Just selecting in hydrus "go here, or go here, or add a new location to go" or whatever. Previously I had one hydrus on my computer and one on an external hard drive, but now I could just have the one on my computer and have one db in my documents and the other db on the external. I think you can use command switches for opening apps on mac but I've never done it and tbh I like the other solution more. Thanks for the quick response

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 01/20/19 (Sun) 15:11:09 cc2505 No.11409

up to v336, still getting this error when importing a folder

UnicodeEncodeError

'charmap' codec can't encode character '\u014d' in position 71: character maps to <undefined>

Traceback (most recent call last):

File "include\HydrusData.py", line 854, in Print

print( str( text ) )

File "include\HydrusLogger.py", line 130, in write

self._log_file.write( message )

File "c:\python36\lib\encodings\cp1252.py", line 19, in encode

UnicodeEncodeError: 'charmap' codec can't encode character '\u014d' in position 66: character maps to <undefined>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "include\HydrusThreading.py", line 307, in run

callable( *args, **kwargs )

File "include\ClientGUIDialogs.py", line 1031, in THREADParseImportablePaths

HydrusData.Print( 'Unparsable file: ' + path )

File "include\HydrusData.py", line 858, in Print

print( repr( text ) )

File "include\HydrusLogger.py", line 130, in write

self._log_file.write( message )

File "c:\python36\lib\encodings\cp1252.py", line 19, in encode

UnicodeEncodeError: 'charmap' codec can't encode character '\u014d' in position 71: character maps to <undefined>

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

hydrus_dev ## Board Owner 01/20/19 (Sun) 22:58:17 06617c No.11412

>>11403

Great, I'll do something like this then.

>>11409

Thank you for this report. I missed some of the unicode stuff last week. I have it fixed for 337.

Note that this is a logging problem, so that particular file it wants to complain about (which probably has some unicode in its filename) may still be broken in 337–it'll just be able to correctly report that it is 'unparseable'. If you get a better error here next week and the file nonetheless seems good in an external media viewer, please submit it and I'll have a look on my end!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 01/21/19 (Mon) 01:47:17 bcc9f9 No.11418

>>11390

when it comes to anti virus, they have 2 methods of working

1) knowing the behavior of the program, if its doing something iffy, they may catch it

2) knowing the exact exe

other then this, anti viruses are kind of worthless. you are honestly better off working in a vm, and becaues these things dont typically catch new things, only old ones, by the time antiviruses catch and patch for things, your os is usually already patched, or will be patched shortly, and for the few things that would still hit you that you don't ask for, it would be better to have a hardware based firewall.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 01/21/19 (Mon) 06:47:28 bcc9f9 No.11422

https://i.4cdn.org/gif/1548042014339.webm

float division by zero… (Copy note to see full error)

Traceback (most recent call last):

File "include\ClientImportFileSeeds.py", line 1211, in WorkOnURL

self.DownloadAndImportRawFile( file_url, file_import_options, network_job_factory, network_job_presentation_context_factory )

File "include\ClientImportFileSeeds.py", line 544, in DownloadAndImportRawFile

self.Import( temp_path, file_import_options )

File "include\ClientImportFileSeeds.py", line 763, in Import

( status, hash, note ) = HG.client_controller.client_files_manager.ImportFile( file_import_job )

File "include\ClientCaches.py", line 1088, in ImportFile

file_import_job.GenerateInfo()

File "include\ClientImportFileSeeds.py", line 289, in GenerateInfo

self._thumbnail = HydrusFileHandling.GenerateThumbnail( self._temp_path, mime, percentage_in = percentage_in )

File "include\HydrusFileHandling.py", line 113, in GenerateThumbnail

renderer = HydrusVideoHandling.VideoRendererFFMPEG( path, mime, duration, num_frames, cropped_dimensions )

File "include\HydrusVideoHandling.py", line 598, in init

self.fps = self._num_frames / self._duration

ZeroDivisionError: float division by zero

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

hydrus_dev ## Board Owner 01/22/19 (Tue) 20:40:31 06617c No.11428

>>11422

Thank you for this report. Someone actually sent me a similar unusual-frame-count music vid the other day, so this is fixed for tomorrow's release already! Let me know if you still have any trouble in v337.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.