[ / / / / / / / / / / / ] [ dir / 8teen / asmr / games / gdpe / newbrit / ss13 / tijuana / tk ]

/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Catalog

Name
Email
Subject
Comment *
File
* = required field[▶ Show post options & limits]
Confused? See the FAQ.
Embed
(replaces files and can be used instead)
Options
Password (For file and post deletion.)

Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf
Max filesize is 12 MB.
Max image dimensions are 10000 x 10000.
You may upload 5 per post.

New user? Start here ---> http://hydrusnetwork.github.io/hydrus/

Current to-do list has: 951 items

Current big job: downloader engine overhaul

YouTube embed. Click thumbnail to play.

00c563 No.6221

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v262/Hydrus.Network.262.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v262/Hydrus.Network.262.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v262/Hydrus.Network.262.-.OS.X.-.App.dmg

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v262/Hydrus.Network.262.-.OS.X.-.Extract.only.tar.gz

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v262/Hydrus.Network.262.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v262.tar.gz

I had a great week. I fixed and improved some things, and I added apng support!

apngs

The apng format is animated-PNG:

https://en.wikipedia.org/wiki/APNG

It is pretty neat! It isn't useful for messy lossy IRL video, but for pixel-perfect animations (usually anime, pixel-art or digital renders with large smooth surfaces and sharp edges), it is excellent. Roughly speaking, apng is to webm/mp4 as png is to jpeg.

There aren't a lot of apngs out there, but when they are done correctly, they are great. Here are some neat examples:

http://apng.onevcat.com/demo/

http://www.littlesvr.ca/apng/samples.html

I am now in a good position to add pixel-perfect and rich-colour ugoira support (ugoiras are just a series of pngs shown in a slideshow by javascript). When the new downloader engine is done, I will see about writing an ugoira->apng converter and hook it into the new pixiv downloader (and any other sites that do the same!).

To programs that don't understand apng (including old hydrus), they look and render like static pngs. I will in future schedule all existing pngs for a rescan to see if they are truly apngs.

network engine overhaul

I spent a lot of time on the new network engine this week. The critical parts are mostly ready, and I am now writing tests to make sure it all works correctly. I had a couple of breakthroughs this week that simplified a lot of what I want to do, so I am feeling very optimistic about it.

None of this is active in the program yet, but I expect to start phasing it in next week. I will start with better bandwidth management.

full list

- added apng support!

- sessions will now append from the default insertion index (e.g. right of current page)

- sessions will now load in the correct order if the default insertion index is to the left of the current page!

- fixed an issue with custom 'favorites' gelbooru parsers

- manage parents/siblings dialogs now load their larger datasets on a thread, after initialising

- sped up how some default thumbnails are generated on client boot

- sped up how resized thumbnails are generated

- refactored ffmpeg video parsing to be simpler and easier to maintain

- improved some ffmpeg parsing to be much faster

- ffmpeg can now parse video that reports no duration

- fixed some more bad video framerate parsing

- the 4chan/8chan thread watcher will now always generate https urls (even if you enter an http thread url)

- pixiv will now produce https urls

- the known url file status system will now check both http and https versions of an url it is given

- refactored some core elements to simplify common controller requests

- started on some database migration help and ui–should be finished next week

- flushed out network engine a little

- more refactoring and simplification of new network engine object coupling

- flushed out bandwidth manager

- misc logic tweaks in bandwidth management

- wrote a network session manager

- wrote a new class for identifying all future 'network contexts'

- employed NetworkContext everywhere in the new engine, massively simplifying several things

- wrote the first batch of tests for my new network engine and its network jobs

- wrote tests for the bandwidth rules object, including covering the new short time delta support

- improved reliability and speed of bandwidth unit tests

- wrote tests for the new domain-based bandwidth manager

- misc improvements

- more misc improvements

next week

Network engine stuff is top priority, but I would also like to finish off a neat gui 'database migration' tool I started this week that'll let you move your database and files around without it being a giant headache.

00c563 No.6222

File: 9fe500844262276⋯.png (596.07 KB, 500x500, 1:1, 9fe50084426227645e8e726406….png)

File: 9e7b8b5abc7cb11⋯.png (602.5 KB, 500x500, 1:1, 9e7b8b5abc7cb11da32db05671….png)

Here are some nice apngs I've been hanging onto for a while. I've got a bunch more in my client, but I'd never be able to find them manually–I'll have to wait for the maintenance that'll rescan them.


1be428 No.6225

>>6221

I was just about to import hunderds of apngs, I come here to check the latest release and what do i see? apng support.

You're awesome, man. Thank you so much for hydrus.


046a40 No.6227

File: 2656dee17269631⋯.png (64.02 KB, 100x100, 1:1, Animated_PNG_example_bounc….png)

WHO APNG TEST BEACH BALL HERE?

Animated SVG support when? (kidding)

I need to remember to complain on /sudo/ about .apng not being accepted as a file extension.


00c563 No.6228

File: 33b4bf5f0cb1834⋯.png (4.3 MB, 512x384, 4:3, 33b4bf5f0cb183437e8483b134….png)

File: 8bdd8ee90debb28⋯.png (25.19 KB, 376x304, 47:38, 8bdd8ee90debb282c562132eb6….png)

I think the VW one has a black background in hydrus as the video pipeline doesn't yet support transparency. I've got a job to figure it out and see if ffmpeg actually delivers the correct alpha channel data.

It seems some apngs aren't supported by current ffmpeg yet at all, so this will be an ongoing job, I think, involving waiting for newer versions and updating the parsing, like we already do for weirder webms and so on. Please submit your apngs that won't import!


00c563 No.6229

>>6228

Also note the massive disparity in filesizes between these two–the first has a bunch of jpeg artifacts, so I suspect it is an apng conversion of a jpeg ugoira, and would do better as a webm/mp4. The latter has hard pixels and flat colour and so compresses very well as png.


3174cd No.6230

>manage parents/siblings dialogs now load their larger datasets on a thread, after initialising

Any way the controls that suggest/looks up tags as you type them in could be threaded? Could prevent the way the GUI locks up as Hydrus is doing the search, and make it more responsive or reduce the need for a delay after you stop typing etc. Typing something like "creator:" will freeze the GUI for 15+ seconds for me, which happens accidentally sometimes as I'm typing in tags in the manage tags window. But even something with less hits can take a second or so.


3174cd No.6231

For the "system:number of tags" predicate, could you add the option to limit that to a specific namespace?

It would be useful for finding files with more than one "title:" tag. There exist a lot of files in the PTR with title tags that have been cut off at the first space through a parsing error. So many of them now have multiple title tags, one correct and one cut off after the first space. Would be easier to clean that up with this addition.


084a36 No.6232

Got a question, I have been useing thread watcher quite a bit as of late, and adding new threads to watch is kind of tedious due to the program downloading an image, locking up for a few seconds, then going to the next when there are up to 200~ images to go through.

So I have a question, is there a way to currently make this process not suck, or would that be a future update?

Also on a potential fix, I use an import folder that imports daily, but I tested it with a 50k import once, it was relatively fast compared to drag and drop that took I think 3 weeks for 120k~ images. So in the same way, would it be possible to have all the images for a specific page download, then once the download is finished (sans failed images) it imports it in one go?


bc50fa No.6233

are tumblr subscriptions dead for anyone else?

mine all fail all files now.


00c563 No.6272

Sorry for late replies, lads.

>>6225

Thanks! If you had any of those fail to import, please do send them in so I can give them another look and try to figure out a fix.

>>6230

Yeah, I think this is increasingly important. The hangs are really annoying, and I think I can alleviate them significantly. I have longer-term plans to completely eradicate the source of these problems, but I would like in the next few weeks to threadify the basic autocomplete request as you suggest.

>>6231

Yes, absolutely. I have a plan to do this, and it is near the top of my todo list.

>>6232

It'll have to be a future update. Most of the new lag from importing is due to the more accurate video parser. This works great, but it blats your CPU and can't help but hang the client a little. I will be writing a wrapper for this code and moving it to an external process that will run completely in the background without affecting the client gui at all.

Please let me know how it works for you as I roll it out!

>>6233

This should be fixed in v263. All your existing links will be updated automatically. Please let me know if you still have problems.



[Return][Go to top][Catalog][Post a Reply]
Delete Post [ ]
[]
[ / / / / / / / / / / / ] [ dir / 8teen / asmr / games / gdpe / newbrit / ss13 / tijuana / tk ]