/hydrus/ - Hydrus Network

>>10806

that solved the issue though it also takes away 10 potential button presses, I don't think it matters much as I can change out the shortcuts i'm using.

with that said, in the custom shortcuts, would it be possible to have a drop down menu for them and a check mark next to them when selected, kind of like the pause menu stuff, at least this would be a nice in ui way to switch to what you are using without going full screen, loading your short cuts, and needing to do this everything you exit full screen.

granted just a note this isn't something I would use immediately, but it would be a nice feature when I finally get to a second phase of rating images into categories.

I have one issue with saves, a while back you found out they caused a crap ton of hang for me, I have also noticed that when the client goes into what I think is a save, it eats 4-6gb of ram bringing the running ram use of hydrus from 12 to 18

not sure if there is much you can do, but a less ram intensive way to save would be nice.

Any chance you could raise the thumbnail width and height limits? 200x200 is too small for my eyesight.

I never got around to testing the new login scripts for rule34hentai back when it was added to the new login scripts so I'm not sure if this was recently or always been this way but the downloader for that site still doesn't pick up hidden images where you need an account to see. Login works fine I guess, it knows I'm logged in. I'm just not sure if its the downloader or the login script that's causing Hydrus to ignore hidden images when downloading from rule34hentai.

Ok, while its not as big of a problem as it use to be, tabs are moving around fairly regularly in the program.

The biggest problem for me is that all my top level things are page of pages, and those are starting to move around, making it difficult to put the program back in order when things want to move.

Is it possible to get tabs to either lock adding new things to it or lock tab positions?

Hell, if you have a tab/thing I can put in that just acts as a spacer between tabs, that would help immensely with re organizing because i'm not putting shit into tabs when trying to put everything back in place.

Can you implement a setting on how much the main thumbnail view scrolls with one tick of the scroll wheel? Because the setting I'm using on my mouse now which works well for most other programs scrolls the thumbnails view way too much. One tick scrolls about 2 pages right now. Ideally I'd like one page at most.

I just updated to Win10 from Win7 and it wasn't a problem there.

An issue I have now is with the PTR sync. It recently got stuck for some reason. So I decided to reset it, which worked. But it's now so very slow at re-processing the updates It's only working at a bunch of dozen rows per second but especially when it seems to have done one update (say, 880/1621) it just gets stuck at the end for minutes to half an hour - often it seemed like it just wouldn't continue.

What's going on there? The database should be stored on a relatively fast HDD on XFS. I recall various Java databases (h2, hqsqldb, cassandra) and other databases (rocksdb and so on) were able to insert k-v or full tuples in the order of quite a few thousands to tens of thousands per second with no weird hangups.

>>10814

I was also having some issues with Hydrus' RAM usage. Maybe it was also mainly during saving, though I also couldn't explain why it was using x GB RAM (can you tell why yours is using 12GB RAM?).

It doesn't seem to keep all the full sized images and videos loaded in RAM, nor should it have to, right?

VERSION 333 - SOON TO BE WITH THEE

>>10813

Sure, this is a good idea.

>>10814

That seems pretty huge! I assume this will blow up your client.db size as well, if up to twenty copies (last_session and exit_session) of your main session build up. If the db size becomes a problem for you, I can add an option to change the number of rolling backups kept.

This answer is going to suck, but the best way to reduce memory usage is to reduce the size of your session. I do not recommend having more than 50k thumbnails/urls total in your session, so if you have 800k or something, that is obviously going to add up when it comes time to serialise it all and save it to db.

I explored a 'what's in memory' debug system a while ago, but didn't have luck getting a stable solution. I am going to try again in python3 in the new year. If I can see what is actually eating that memory, we'll be able to figure out how to reduce it.

>>10815

>>10816

Due to some technical bullshit, I can't do this by just increasing the number. But I would like to do something here. I am afraid it did not score highly on the 'big things' poll, so we may have to settle for a dirty solution rather than a beautiful one.

>>10818

I have not used it myself, and I don't think I wrote the shimmie login, so I can't comment cleverly. If it has logged in truly correctly, it should show what your regular browser login shows. Maybe try going manage logins and then hit 'reset login', to force it to try again.

If it is still broken then, let me know and I'll dig a little deeper. It might be r34 is returning some specific error and the script isn't picking up on it, thinking everything went ok.

You might like to try help->debug->network actions->fetch an url and put in a url you expect to have access to. That should let you save the html, so you can scan it in an editor or load it in your browser and see if it looks right or if it says 'not logged in m8'. You can pastebin the file to me, but be careful if it has private info like your username in it.

Let me know what you find out.

>>10824

>>10839

Thanks. I think the nice answer here is to improve the ui code so it doesn't stutter and hence interpret some clicks as tab drags.

But maybe in the meantime I can just have an option to disable them, or yeah, pin them. I like the idea of pinning most, I think. Pin against close and pin against move.

If it helps, and you haven't seen, you can right-click those page tabs to get a bunch of different move/collect options. This might be more reliable than DnD for you.

>>10825

Try options->thumbnails->rate per scroll tick. This is a little experimental. Typically, for every physical scroll, the OS delivers 3 scroll events to hydrus, so a default client tends to scroll three thumb heights per physical scroll. Two pages at once seems high. How high are your pages, approx, in thumb heights?

>>10834

Check your options->speed and memory to juggle the size of the caches in the client. It usually reserves a few hundred MB for things like image and thumb caches. Most of the size of giganto clients is in huge sessions that have hundreds of thousands of import objects or thumbnails. I recommend clients not have more than 50k thumbnails/urls total in their main session.

For your PTR sync, the main bottleneck on HDD clients is random access read latency. You can improve this a bit by increasing the db disk cache under than same options panel. Since your reset tables may be new and un-analyzed, you might like to try database->maintenance->analyze->force as well. Failing that, here's some help to help get me more info:

https://hydrusnetwork.github.io/hydrus/help/reducing_lag.html

The best solution is to move your db to an SSD though, where processing time typically increases to 20-70k rows/s. Putting the db on SSD and files on HDD works great:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

>>10854

the number you are looking at is 40k before the client starts to have lag spikes, with everything split into sub 1000 image tabs, its going to be alot easier to just go through it now, but it will take some time.

as for db size, its gone from 28gb to 37gb this version. granted this is more an issue when backing up then it is an issue on the nvme

>>10855

for me that's in the would be nice category

"Add an optional system to record why files are being deleted"

Is in the critical one, though its not doing a whole hell of alot better. I have a deadline where this goes from would be really nice to have to shit needs to be there to use the program at all and that's 721gb, so possibly another year and some change before the 3tb drive runs out of space.

Like I said a while ago, user notes remembers on delete, if user notes would just be linked to the downloaded files I could at the very least set up some macros for common things and just add them to the files notes, I would see them on re import. any system you make here doesn't need to be a forever system, it just needs to be a system that won't lose its data when the forever system becomes a thing.

>>10874

Here's hoping that they don't move to fucking twitter or some other shit and go to an actual porn hosting site like hentai foundry. will definitely miss short stack approved though.

The tag import copy from defaults does not copy blacklists.

>>10874

SHUT THE FUCK OUT KID DO YOU KNOW HOW MANY DOWNLOADS I STILL HAVE

At least I'll have less filter-related dupes after this.

>>10876

They're already moving to Twitter like the Japs. Some to Newgrounds too, of all places, because they gave a timely reminder on Twitter. Shit's fucked, all I can say is: at least they're not on DA!

I'd wish for Pixiv but they have almost the same policy, just shittier enforcement.

At this of all times, I'd put together something and try to get advertisers to pay me bitcoin to avoid payment processors altogether, but I don't know how to get artists to actually use it or how to get exposure. Do you think ad revenue sharing would work enough to get traction?

>>10860

>It usually reserves a few hundred MB for things like image and thumb caches.

I'd expect that. Not sure how it gets up to multiple GB of memory used over time

> I recommend clients not have more than 50k thumbnails/urls total in their main session.

The URLs are a bit surprising… do subscriptions count somehow?

> the main bottleneck on HDD clients is random access read latency

It's not merely very slow even compared to my experience value for various HDD-hosted databases. It gets stuck for hours. Maybe indefinitely now, nothing happened over 6 hours.

> Since your reset tables may be new and un-analyzed, you might like to try database->maintenance->analyze->force

No result, unfortunately.

> The best solution is to move your db to an SSD though

I'm considering it. Not terribly easy to do in this machine, but I guess it's an option if I pay enough to replace an existing SSD with a much bigger one.

OTOH again, the HDD storage isn't that slow and it can have a bunch of hundred MB of RAM to do it's job if it must- it worked on other databases with dozens to hundreds of millions of entries before. Never had them lock up weirdly like hydrus' GUI suggests its DB does; they'd generally drop down to a bunch of hundred rows per second on more complex queries that edit data inside tuples.

>>10874

This has to be the one of the most retarded decisions I came across that a company could have made.

>>10883

Crap meant to reply to >>10856

>>10877

Thank you for this report. This is fixed for tomorrow.

>>10874

Apparently, someone on Danbooru found a way to download raw images from Tumblr that was recently removed. I don't think this can be integrated to Hydrus, especially not before December 17 but it may be a good idea to download all the stuff manually before Tumblr nukes all the artworks, so I decided to let people know. Here's the method:

>I think it's still possible to download raw sources from Tumblr, just very inconvenient. I don't think it's intended so it might be fixed, but with the upcoming ban on adult content I figured it's now or never. If you export a blog you run (Blog Settings -> Export) then you get seemingly full size images even from reblogs; I've received 20MB PNGs and JPGs from this export. I last tested this on Nov 24 as all my blogs are 10+GB and I'm on mobile currently. If there are any Tumblr artists who don't post to a proper art sharing site like Pixiv this may be useful.

>>10876

Thanks. Yeah, if your db is that huge, and grew that much, I guess your session is about 500MB. I recommend you cull as many download queues as you can and put off the processing tabs with 'x to process'-type local tags so you can find those files again later.

>>10876

>>10878

>>10880

>>10901

I'm still laughing/in disbelief about tumblr's decision. I thought after the uproar they would swiftly reverse the decision, but I guess they don't want to subsidize free porn bandwidth any more. The only conclusion I can make is Verizon just don't have a real business plan for it. No idea why they took it on in the Yahoo purchase. What a way to kill a brand.

Is their a way to search through Notes or see which images have Notes written on them?

>>10859

Thanks, 0.8 works better, it scrolls a full page per tick now. I have my scroll length set to 5. One page is about 5½ thumbs high.

I think this might also be related to the fact that I have Windows 10 set to 125% scale, Hydrus doesn't seem to handle that quite correctly (for example when resizing the width of the tagging panel to the left of the thumbnail view).

>>10917

Not yet, but I'd like to present that information better in several ways. Adding search would also be useful.

>>10933

I hope to check out high-dpi stuff in the new year. The new wx is supposed to be able to handle it better, so I'll have a look and see if I can figure out what flag I need to set or whatever. I have a 4k monitor now, so I can test this better.