/hydrus/ - Hydrus Network

>layout improvements for small-screen users

Damn, just today I was thinking about this.

thanks.

>>10187

Thanks, I might try running from source. How does this impact the performance?

Oh, it might not be such a big problem after all. It's the import page that automatically opened to show new downloaded images. I must have kept the tab open.

When I open a regular search page for the artist it doesn't show the deleted entries anymore. I guess the import page has a fixed list of entries it displays?

>>10187

I've had this issue ever since the big UI update/rewrite. Running from source does not help, but it doesn't seem to hurt anything either. I'm running Linux Mint 18.2, which uses Ubuntu 16.04 for its package base.

On an unrelated note, ever since I updated from v304 to v324, my console logs an error on startup. It says, "Could not import lz4." AFAICT, everything works, but I don't know what lz4 does so I don't know if it's just because lz4 code is never called during my normal use of Hydrus.

>>10189

I get that error too. But lz4 is just some obscure type of compression that's not used in any file formats I know, and it doesn't seem to cause problems, so I didn't bother reporting it.

>>10188

Should be about the same, as the frozen exe is just spinning up a python instance and running the same code, afaik. Boot should be a bit faster since it doesn't have to mess around with that bootstrap environment shit.

That's interesting that that page kept the files through the trashed->full delete transition. They should be removed automatically.. That suggests the underlying file context of those pages is 'everything' rather than 'local'. Or maybe they are loading from db subsequently in a non-filtered way. I'll check this, thank you for reporting it!

>>10189

>>10190

Thanks. If you figure out any new/unusual AssertionErrors getting put to your log (not the GTK-critical stuff, which doesn't give me enough info, but a proper python/wx error trace), please let me know. There are still some instabilities in the linux code, but I don't always get a good traceback in my environment.

lz4 isn't a big deal. I've been unable to package it properly for a few months now due to a PyInstaller update. I burned some hours on it and couldn't figure it out, so I will revisit after the python3 update.

>>10205

Okay, I will have a look at running the source then.

>>10137

Starting from scratch and not messing everything up is probably simply harder than just fixing what you don't like in Hydrus.

>>10129

> I just want to know what I'm in for.

Programming? Most people still see nearly endless possibilities to still get better after multiple Ph.D. and 40 years of experience.

Programming teams involving hundreds or thousands of people with 20+ years of experience each do NOT produce perfect code. Nor do companies that put BILLIONS of dollars into the software of their products produce anything nearly flawless.

And so on. That's what you're actually in for. You'll hack something together if you *actually* maintain the motivation for a few thousands of hours (more for you because you don't seem to know how to program and probably don't know the maths and algorithm theory and everything else either), but you can't expect it to be good easily.

They actually did this https://gitgud.io/Dizmal/borehole

>>10197

Not me. Why would I come up with a COOL name like "Pyrus" then abandon it?

>>10244

Because you are a fag who can't actually code or write a proper specification.

>>10128

This is the guy from 8/tech/ telling you to show code or GTFO.

> UTF-8 boooo

If you do download images from Pixiv you will see why you are flawed.

> Moar tags

Already on its way to the public

> YouTube and Twitter videos

On the to-do list already

> Symlinks and secondary DB

There are winfags here so it is not for everyone… On the to-do list though.

> No client-server difference

You will regret this once someone implements a 3rd-party client/browser/viewer to work with the upcoming API

> the hideous GTK3 theme

WxPython for Winfag compatibility. Move it to Qt if you have balls.

> No "media viewer", rely on VLC/MPV

On the to-do list

> Allow users to create custom execs for each filetype

Whatever

> You could get a whole lot more complicated in the long-term by introducing a whole scripting language

Read the list of parsing scripts https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts and if you think you can write a language to replace this, take a look under the hood

>>10175

> UTF-8 boooo

I'll forgive you for misinterpreting my rant, but that was about characters being converted to their lower-case forms, not something needless like converting unicode to hex.

Considered the rest though, maybe I'll wait and see how good Hydrus gets.

>>10194

"Talk is cheap, Show me some code" Linus Torvalds

"Release Early, Release Often, Listen to your customers" Eric S Raymond

"No problem is too big it can't be run away from" Linus Torvalds

"Prototype then polish, Get it working before you optimize it" Eric S Raymond

"Any program is only as good as it is useful" Linus Torvalds

"With enough eyes, All bugs are shallow" Eric S Raymond

"Those that can, do, Those that can't, complain" Linus Torvalds

Any chance you'll address my complaints?:

>>10160

>>10161

I can keep patching it myself, but I feel like a relative update interval for high-velocity threads ranging from 5 minutes to 10 seconds is something reasonable (especially because it's not a problem for some imageboards github.com/4chan/4chan-API).

Nice. That certainly sounds interesting.

BTW, not sure if you saw it already but Pixiv did recently change the profile pages. Doesn't seem to completely break the downloader, but it seems to result in some failures. Apparently on the first 4 or so downloads it tries.

>>10171

Thanks, answered in main release thread.

>>10174

Thank you for reporting this–clearly the Nips are at it again. I don't have time to look into it for today's release, but I will check back on Saturday.

>>9579

>>9580

>>9581

>>9585

>>9586

>>9589

Thank you for this information. The new shortcuts system I am slowly migrating over to does not support double-shortcuts like ZZ, but I hope to eventually move pretty much all events over to it, so remapping arrow keys for movement should be doable in the future. I am drowning in work atm, but as the downloader overhaul comes to a close I should be able to revisit some of these longer term rewrite/cleanup jobs. Let me know how it works for you as I roll it out.

>>9587

I would like to rewrite session loading to make is less resource intensive and do delayed loading like you suggest here.

But yeah also moving to multi- pages should massively reduce the memory burden for downloaders that would otherwise have like 3000 thumbnails attached. Now the db just has to load the file import status list, which is basically just a URL and a couple of numbers per item, rather than all the tags and so on that go with it.

I have some more ideas here to relieve gigantico-clients, but it will have to wait for the next pass on the downloader overhaul.

>>9579

>>9580

>>9581

>>9586

>>9589

Let me get this straight. You people LIKE vim? Hell, I use it every so often, but only if I need to make a super-minor change and don't feel like waiting for a graphical text editor to load. Or, more likely, it can't load because I FUBARed my system and need to fix it.

Hell, I've always thought that all of those people talking about and comparing text-only editors were either old farts who worked on mainframes in the 1980s or people just joking around.

I'm not arguing against improvements, mind you. I'm not even arguing about priorities. I'm just confused as all hell.

>>9587

Low-resource mode is a great idea. With the old download system, I've long wished to be able to somehow page out only part of the memory Hydrus was using (yes I know paging doesn't work like that, but let me dream goddammit). At one point I actually bought more RAM so Hydrus worked better, especially when also doing something with my computer that's not Hydrus.

>>10150

Shit, I thought this was the new thread.

>>10150

>I'm just confused as all hell.

Why? There is a lot of people who uses vim as their primary text editor, people really really like it, there is even a game to learn Vim, the community is kinda like Emacs (but I think Emacs users are more dedicated to their text editor), there is a lot of programs that use vim or emacs shortcuts, one example is the Terminal, by the default they use the Emacs shortcuts but you can set it to use Vi shortcuts, I prefer to only use the keyboard so most of my programs use the vi keys (I use them even in my web browser) , I try to do the same with hydrus but it ins't completely possible at least yet.

>>10067

>>10066

Thanks for this update. My guess is opencv is sperging out due to some .so not loading right. I will write a 'image load report mode' in the next week or two that will specify which library is being used as images and gifs load and when failures are occurring.

Are you running from source or a built package?

Anyway, please run this report mode when it comes in and let me know what you see.

>>10068

Yeah, I'd like to write a client api at some point that will let people write scripts to tell the client to do simple stuff like it. It'll just be an http server you can talk to, with some simple authentication.

This will obviously be a big job. It'll be up for vote on the next big poll.

>>10071

>>10069

>>10072

>>10076

Thanks. Although your threads are spiking there, it doesn't look horrific or out of place. My guess is this is ui-bound, particularly since looking at the busy watcher page is making things noticeably more laggy. I will review this code and see if I can optimise and slow the refresh rate based on the number of watchers being tracked.

Although the client uses different threads to do its stuff, python unfortunately cannot use multiple processor cores simultaneously, so it never really is 'truly' multi-threaded. It can use multiple when it dips down to the C++ level (typically when it is doing low-level stuff like heavy math/file processing), but otherwise, it'll typically max out at 12.5% or 25% CPU. My threading code sucks as well (it does a lot of async shit synchronously, just waiting on results), which is a big cause of a lot of ui hang.

>>10087

One of the things you responded to was a different issue with a subscription.

On the side of reducing load. If possible can you keep the top level things, as I like seeing XXX/XXXX to see how much work is left.

Sometime this week im going to dump the entirety of thread backlogs into a saved session, I have a feeling the watchers are causing quite a bit of ram use even if they aren't visible, it will at least give me a point of comparison once I dump the somewhere around 8000-10000 threads watchers down to around 800 active.

>>10085

I'm running from source, specifically the AUR package from Arch Linux. Thanks for looking into it, and I'll run in it report mode when the time comes.

>>10016

Thank you for this report. Yeah, some of the new sizing system is borked due to some min sizes. I have a plan to write some of my own sizers to more dynamically size this stuff while still permitting min heights for the controls that need it. At the moment, I am using some stock sizers, and they don't calculate combinations of static and dynamic heights in a conservative way.

>>10013

>>10018

>>10019

I'd say associating 'pixiv id:123456' with 'creator:ching chong' is probably a job best solved by a future iteration of the tag siblings system. When tags have more metadata and you can right-click on them and get an automatically formatted wiki entry on them with known synonyms and all that a bit better than the current system, and if users could choose how to display all that, I think then you'd want to start putting in the effort.

You could do it with parents now, which would state both tags, but since the tags are true synonyms, the direction is probably siblings.

I don't know though. The problem of 'artist "x" is called "xdraws" on this other booru' is a persistent issue, not yet neatly solved in hydrus.

>>10019

1 - No. Other than bmps (which are automatically converted to png on import) hydrus should keep all files exactly the same, byte for byte.

2 - Yeah, should do these days. At some point, this info should be cached and searchable. Accessed will change any time you look at it, obviously.

3 - When I add that cache, I'll probably add it as a new kind of metadata, rather than explicit tags, which don't store specific timestamps efficiently, and search for it with a new system predicate, like 'system:time imported', but 'system:file creation/modified time'. If you want to search this info, just keep importing as normal–since it is preserved through import, no information is being lost, and when I make the cache, I'll retroactively fill it with existing file data.

>>10023

Thank you for this report. I am still moving all my old shortcut code to the new system. Mouse shortcuts are not well supported at all yet–the code is just not yet in to handle it. I hope to have more time to put into this as the downloader overhaul comes to a close.

>>10029

Thanks. That will be awesome.

>>10000

*but

fug

You sir, clearly have no idea what a hydra is.

fishfags fuck off

>>9916

Yeah, I've had a lot of trouble getting different kinds of floating window to work on non-Windows, Linux especially. By default, it is embedded for a Linux/OS X client, but anyone who switches over from a Win install keeps the old setting.

BTW: if your Linux flavour has more problems (particularly instability problems) taking the 'run from source' pill is actually proving a decent solution these days. If my build runs fine, then keep using it, but if you get a lot of crashes, check out this document:

https://hydrusnetwork.github.io/hydrus/help/running_from_source.html

>>9934

You could make this public so people can write better reports on new features, you know?

>>9938

I'm afraid that's my point–I know I am bad at dealing with collaborative systems. I find it just becomes talking talking talking and stresses me out to the point of abandoning the whole platform and the users along with it. This was most recently true of Github Issues, which I ended up turning off because I couldn't keep up with it enthusiastically.

Like many other Anons, I'm a complete fucking sperg, so figuring out what workflow lets me put the keyboard work in week to week without imploding in drama has been an important part of the whole deal for me. Although the current to-do is a mess, I can't see a more productive system for me atm. I appreciate your point, but for me the benefits are outweighed by the inevitable personal drawbacks. I'm open to other solutions, if you can think of them.

Some users on the discord put together platform where they could discuss new features and so on–which I had no problem with, although I did have this same convo with them–but I don't know whether it actually succeeded beyond the first two fun weeks. If you are interested in this stuff, you might like to chase down whatever happened to it.

>>9891

Hey dev,

The Derpibooru integration works pretty well, but seems to be subject to the site's default filter which hides anything not kid-friendly.

You should request no filtering by passing `&filter_id=56027` in with each call to the search endpoints (assuming showing everything is a reasonable default).

The filter_id parameter is mentioned briefly on the API docs page [0], and the ID is from the Everything filter on the filters page [1].

[0]: https://derpibooru.org/pages/api

[1]: https://derpibooru.org/filters

>>10010

👌

demo.illustration2vec.net seems pretty dead, is there a mirror around?

Is there any way to change the output filnename of the downloaded file & accompaning txt?

For example could i make it grab the post number as the filename based on the site? Like, from Sankaku it would be "S ####", Danbooru "D ###", Yandere "Y ###", or is it just not possible?

>>9703

Sure it's possible, but you'd have to edit the script. I did some minor changes myself but I'm not too familiar with the code/language so I couldn't do it for you without some effort.

>>9780

Do you perhaps know how to at least make it download the original filename? My idea of grabbing the post number was probably overkill, with danbooru and yande.re it'd just be fine to have the original name, but i have no idea how do edit the script to tell it to stop the renaming function

I mean, i think i've even found the line in the script with "var filename", but i don't know what to do with it

Tag parser doesn't get meta tags currently, any plans for this?