Version 280

Name
Email
Subject
Comment *
File	Select/drop/paste files here
* = required field	[▶ Show post options & limits] Confused? See the FAQ.

Embed	(replaces files and can be used instead)
Options	Do not bump (you can also write sage in the email field) Spoiler images (this replaces the thumbnails of your images with question marks)
Password	(For file and post deletion.)
Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf Max filesize is 16 MB. Max image dimensions are 15000 x 15000. You may upload 5 per post.

YouTube embed. Click thumbnail to play.

Version 280 hydrus_dev ## Board Owner 11/01/17 (Wed) 21:55:18 bb3a29 No.7132

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v280/Hydrus.Network.280.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v280/Hydrus.Network.280.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v280/Hydrus.Network.280.-.OS.X.-.App.dmg

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v280/Hydrus.Network.280.-.OS.X.-.Extract.only.tar.gz

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v280/Hydrus.Network.280.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v280.tar.gz

I had a great week. I have finished adding EXIF image rotation support and fixed a bunch of other stuff besides.

rotated images and stretched videos

Images with EXIF rotation (this is usually camera photos that have an orientation flag set by a sensor inside the camera, but it comes up in some other contexts as well) should now be fully supported in the client. Thumbnails and regular media viewer rendering should all display correct now. Unfortunately, this only affects newly imported images, so if you know you have some bad ones, you can either manually delete and reimport them or you can wait for me to write a maintenance routine that will do this automatically.

Also, some videos that imported with stretched dimensions should now be fixed. In the same way, I'll be writing a maintenance routine to look for and fix these over time, or you can force them now.

misc

Like 'file import options' some weeks ago, all 'tag import options' collapsible panels are now buttons that launch a small dialog. As well as saving screen space, this also reduces the total count of in-memory ui objects. I have increased the maximum page limit to 200 as a result, but haven't tested it thoroughly–please let me know if you still run into program instability with page counts above 175 or so.

The 'page of images' downloader now has two separate pause buttons, if you would like to control its file and page queues separately. It should also be a little less janky overall.

After the hydrus switch to the new network engine, I have improved the default hydrus bandwidth rules. You will get them on update, so if you care about this stuff, please check them (it now just defaults to 64MB/day, which is fine for hydrus and really just a stopgap to stop unforeseen problems), and if you don't care about bandwidth rules at all, you can just leave it alone–hydrus stuff should just work better now!

full list

- the client should now recognise EXIF file rotation and flipping in resolution discovery, thumbnail generation, and image rendering. existing files will be borked, so will have to either be manually deleted&reimported, or wait for a maintenance routine that will retroactively fix all these

- videos with a non 1:1 sample aspect ratio (wew lad) should now import with correct resolutions

- all 'tag import options' are now managed with the new button

- deleted all old tag option management gui, including deleting ClientGUICollapsible entirely

- with the simpler tag import options, increased max page count to 200

- page of images downloader now has two pause separate buttons for queue and files processing

- the page of images downloader should no longer too-quickly blank out its parser status at the end of queue processing

- the default hydrus bandwidth rules now no longer has the choking 'requests per day' limitation that was messing up large mappings uploads. (it is now just 64MB/day)

- your hydrus default bandwidth rules will reset to this on update

- POST requests (which in for our purposes are always user-driven) no longer obey bandwidth rules

- fixed a bug when sometimes hitting the file limit or hitting a siteside 404 in the gallery downloader

- a rare bug due to empty string menu labels is now no longer possible. empty string labels, if they slip through, will be replaced with 'invalid label'

- you can no longer choose the empty string as a gui session name

- improved how network jobs keep track of all applicable network contexts vs the preferred session and login contexts

- cleaned up a bunch of login code, improved verification testing

- fleshed out domain login system

- in the networking engine, the read timeout is now six times the connect timeout–we'll see if this reduces some hydrus and other read timeout problems we've seen

- increased the server transaction period from 10s to 120s–we'll see if this helps reduce some spiky server POST lag or what

- the default tag import options panel now edits on double-click, rather than deletes. it also yes/no dialogs on a delete action. this control is still trash, but it now works more like the rest of the program

next week

I did some good work on the login system this week, so I will keep pushing that. I would also like to see about moving subscriptions to support multiple queries, but this might be a two-week job.

Anonymous 11/01/17 (Wed) 22:30:42 af9e22 No.7133

>>7132

Ok, most of the fucking videos you post I can at least gleam context from

this one

here is what google translates the name to

Option: Left breast breast milk video

what the actual fuck am I watching.

Is he playing a game or is it a daw he is fucking around with.

Why did he tape the ruler to his nipple, there has to be a simpler way.

Anonymous 11/01/17 (Wed) 23:39:32 1f7963 No.7134

>>7133

he's playing beatmaniaIIDX, and using his nipple to 'scratch', i assume 'for the lulz'

MintTeaFresh !!2icG5cjEeM 11/02/17 (Thu) 05:24:27 e75df8 No.7135

Well, now is the time I would normally post the alt linux build, but Mediafire won't work. The progress bar goes up to 17%, starts dropping, and then starts going up before dropping to 13%, 10% 6%, 3%, and finally to 0% and saying the upload failed. I've contacted support to see if they can't unfuck themselves.

Anonymous 11/02/17 (Thu) 09:54:29 cb93a6 No.7136

File: 21a618cdd94e5f3⋯.jpg (464.7 KB, 3000x1740, 50:29, 21a618cdd94e5f3415ffb1eaca….jpg)

File: d75a2c4a00f81b2⋯.jpg (481.1 KB, 3000x1740, 50:29, d75a2c4a00f81b2ae13b687956….jpg)

File: 12994622930b9c0⋯.jpg (581.21 KB, 3000x1740, 50:29, 12994622930b9c07b5833eb8e9….jpg)

The thumbnails and actual image still mismatch on these. Thumbnail is landscape, image is portrait. The stretching is gone though.

Anonymous 11/02/17 (Thu) 11:49:13 4b1c1b No.7137

>>7136

That's odd, when you post them, the thumbnail is portrait, when I click on the image to expand it it becomes landscape, but when I open it in a new tab it is a portrait.

Anonymous 11/02/17 (Thu) 12:33:01 cb93a6 No.7138

>>7137

These three are weird. I've already rotated them and imported them back in as new images so I don't have to deal with it anymore. Just reporting that things are not completely fixed yet.

MintTeaFresh !!2icG5cjEeM 11/02/17 (Thu) 18:36:23 e75df8 No.7140

YouTube embed. Click thumbnail to play.

Apparently the Mediafire issue was something on their end, and nobody was able to upload last night.

I made an alternate Linux build for people having trouble with the official build. A few things are still weird, but overall it works better for me than the official. Run from source works slightly better, but pip is a headache. Built on Linux Mint 18, so it should also work on Ubuntu 16.04.

If you want open externally to work, you will have to manually set the open program for each mimetype in options->files and trash. For some reason, when Hydrus invokes xdg-open it opens a program in WINE.

https://www.mediafire.com/file/du2xbkkyh0d3c13/hydrus-280.tar.gz

I'll post one of these every week until the official Linux build works on my machine. If anything is broken, tell hydrus_dev (and thank him while you're at it). He was kind enough to give me his build command, but I don't know Python. If something is broken, I'm probably just as clueless as you are.

Anonymous 11/02/17 (Thu) 19:26:38 2ff078 No.7141

>>7136

>>7137

>>7138

We already went over this >>6845

go get the :orig files from the twitter post and they'll no longer be messed up.

Anonymous 11/02/17 (Thu) 19:37:39 cb93a6 No.7142

>>7141

I don't have issues with this anymore. As I said I just reimported the the edited(correctly rotated) image and now it works as it should. But if anyone else downloads these from gelbooru, like I originally did, they'd still get the wonky versions.

Anonymous 11/03/17 (Fri) 14:30:13 ae4abc No.7143

Hey boss, I'm getting a few error messages in the import status window when I scrape the boorus complaining of truncated images and whatnot, i.e.

image file is truncated (31 bytes not processed)… (Copy note to see full error)

https://chan.sankakucomplex.com/post/show/692565

https://chan.sankakucomplex.com/post/show/674828

https://chan.sankakucomplex.com/post/show/678839

or something like

Image seems to be a Decompression Bomb!… (Copy note to see full error)

http://konachan.com/post/show/252706/animal-bikini-bird-black_hair-blonde_hair-blue_eye

http://konachan.com/post/show/252705/animal-black_hair-blonde_hair-blue_eyes-bow-braids

Checking the source, there usually is indeed something wrong with the images. Usually a bit of corruption near the bottom, but most of the time it is unnoticeable. Anyway, is there some way we can overwrite the command to abandon the download of slightly corrupted images, or at least give us an option to do so in the future?

Anonymous 11/04/17 (Sat) 12:35:37 eabc1f No.7147

Got a question

Is there a way to add a duplicate search option on a set of images?

Here let's get the scenario laid out.

right now my db for dup searches is kind of messed up so giving accurate numbers as of now is not really easy and until some things change which I have mentioned in the past to dup filtering and delete tagging, getting my db fixed up for dups is a low priority. That said I was thinking of how to make duplicate searching more useful.

I typically import 30-200 files a day from my main import folder, basically images I liked enough to make me save them.

so now they are in my db, If I go to the dup detector, I have 1.3 million files and when it was working 120,000 potential duplicates on an exact match. Now as you may imagine, thats a fucking daunting task.

however, if I could take a page, and use the images in the page for the basis of finding duplicates, now I could take those 30-200 images and run the dup detector first, then go through the images and see if anything is already in my database in a higher quality, or possibly already tagged but lower and deal with it that way, or I could get a random sample of images,

filter

everything

rated - not rated save/suggestive/explicit

limit:100

that should give me a random assortment of 100 images I can go though. rating/tagging after I feed the list through the dup filter rather than dup filters current random 300 making the overall process less daunting.

hydrus_dev ## Board Owner 11/04/17 (Sat) 18:29:38 8a9836 No.7148

File: 057346286c30b35⋯.jpg (956.82 KB, 1280x1775, 256:355, 057346286c30b3544f296d25d2….jpg)

>>7143

Thank you for these examples. The 2hu one actually imported ok for me, but I get the same problem with the others.

This has been a long-running error, but in searching for it again, I have found a way to override it. The images seem to render in the client, albeit with their glitches, although their thumbnails are completely black. I will work on the thumbnail issue, but let me know how v281 works for you with these images.

The old advice on this, btw, was to run the images through GIMP or Photoshop. Just load/save, and because they are so much cleverer, they'll generally heal the damage so even if the file still looks borked, its new file structure is valid.

The decomp bomb is a different issue, and I am not sure what to do about it. Even though those files are not actively malicious, they are gigantic by today's standards, and I've noticed in my tests that they often consume like 2GB and a ton of CPU to render (when by their sheer size it should often be about 400MB or so), so the 'holy shit' detection isn't completely wrong. I presume png compression can get a bit too good for its own good on these 71MB (wew) files. If you like, you can turn off the error hook under options->media and try them again. I recommend just doing one, manually, from your hard drive and watching your resource use under task manager.

>>7147

Thank you for these thoughts. I agree that the current dupe filter is too broad. I will be adding a way to say 'launch the filter but only show matches where one/both files are within this search' in my normal weekly work. I don't know when it will be ready, but I will be adding saved 'favourite' searches soonish, which will give me some extra tools to throw regular search objects around a bit easier, so I expect it will be rolled into that.

Anonymous 11/04/17 (Sat) 23:18:38 eabc1f No.7149

>>7148

playing around with the dup detector a bit more I have a suggestion.

allow remembering zooms and possibly relative positions of all image not just same ratio

possibly add in a lower right or left hand corner A/B watermark, Going from black to gray in the background is helpful, but the problem that I have is no matter what the images I'm zooming into it 200 to 800% so I can see the compression artifacts. The program on my system also has a tendency to hang going from image to image so it makes it really hard to use a duplicate the finder easily.

I'm trying to think of ways that you could use a duplicate Finder to spotcheck images and make the process go a lot faster for manual operation but I'm coming in at a loss outside of my idea for having a picture-in-picture and remembering two different image areas.

On the topic of that how easy what a side-by-side be?

Open up the two images at once side-by-side and manipulating them on one window manipulates on the other image to, you zoom in the other one zooms in you move the one on the right down the one on the left moves down with it the one on the left gets moved to the right the one on the right gets moved to the right. That would possibly save a lot of time, but I'm not sure how viable an option that is. If you took the viewer out of full screen and had two viewers open at a time adapting more possible to do it that way?

Anonymous 11/05/17 (Sun) 03:51:58 99eda9 No.7150

>>7148

Thanks boss, the decomp bomb issue was resolved.

Anonymous 11/05/17 (Sun) 05:52:39 0a3168 No.7151

Just updated the client back from when sankaku stopped working. Seems I lost all my tags and images, not too big of a deal. But I can't for the life of me figure out how to get the tags back and have the booru downloader auto tag the images it downloads.

Someone give me a hand please? I've forgotten how to do everything.

Anonymous 11/05/17 (Sun) 05:55:31 0a3168 No.7152

>>7151

Alright hold up, seems I didn't lose everything, Hydrus just doesn't know its there for some reason. "E:\Hydrus Network\db\client_files" still has all my images and I'm going to assume the tag database is somewhere. How can I restore the database?

Again, thanks for any help.

Anonymous 11/05/17 (Sun) 06:22:48 0a3168 No.7153

File: 19d74549c6400b4⋯.jpg (33.74 KB, 500x375, 4:3, 1507089173148.jpg)

>>7151

>>7152

I realise im just talking to myself at this point but I figured out the tagging problem. Just want my old pictures and tags back if possible.

Anonymous 11/05/17 (Sun) 06:50:00 99eda9 No.7154

>>7153

Yeah, devanon said something about older versions not compatible due to a change he made. If hydrus hasnt rewritten over your old database, I suggest you make a backup of it, then try and update incrementally, say 2-3 versions at a time until you catch up. Good luck

Anonymous 11/05/17 (Sun) 06:53:13 99eda9 No.7155

>>7148

In the interest of providing extra data-points, decided to update my yande.re collection. Got these errors

Could not parse a download link for http://yande.re/post/show/413982!… (Copy note to see full error)

http://yande.re/post/show/413360

http://yande.re/post/show/413975

http://yande.re/post/show/413976

http://yande.re/post/show/413977

http://yande.re/post/show/413978

http://yande.re/post/show/413979

http://yande.re/post/show/413980

http://yande.re/post/show/413981