/hydrus/ - Hydrus Network

I like the idea of CBZ support because I've got quite a few of those, but it's more important to have hashes of the archive contents, then the archive itself, and I can't think of how to resolve that with the hash is the file identity paradigm.

I recognize that this will make the storage requirement of manga marginally larger, but I would be willing to have that for some order.

As such I think it would be best to just have a way to unpack these files and improve the way hydrus manages and displays collections.

-public url repo (aka PURR) - imagine searching the hydrus tag db then downloading all the urls associated with the resulting hashes

-another round of api because api is great

-PCIDR: Public ipfs content id repo

ipfs content ids seperate from url repository.

-Also I would make the PURR optional to pull from, because I can imagine that not scaling well.

-API

-duplicate pairs

-manage subscriptions

-add url without publishing a page

-get GUG/urlclass results

-Subscription url class

Generate subscription entry from url, good for use with hydrus companion

watch uma musume

From top to bottom, this is my main wishlist:

- Add multiple local file services (which will enable true nsfw/sfw partition)

- Improve the client's local booru (this likely now means a backend migration to the Client API)

- Add an incremental number tagging dialog for thumbnails (for adding page:n etc… to a sequence of files)

- Add more commands to the undo system

On the other hand, I'm still quite a few versions behind. I tend to keep a large number of pages open because my tagging barely keeps up with my downloading, but having a large number of pages open on upgrade means that I often lose my session on upgrade and all my organized-but-not-tagged-yet files vanish into the database. So I'm not in any real rush for any of those.

Sure, a mass auto-tagger would be helpful with my tagging problems, but I don't think it necessarily needs to be built in. It looks like the Client API should provide most or all of what's needed for external tools to do that, and I've already got my own little personal-use mass-tag-search tool I wrote myself that should integrate nicely with it when I upgrade to a version that has the API.

I've seen it suggested a few times on Discord now:

How about namespace aliases and/or renaming?

>>12152

Going to have to ask for an expansion of labeling why things are deleted

the current where it got deleted from helps a lot. in fact it could facilitate a jerry riggable system where if we have different places to send files, currently we have

Inbox

Trash

Archive

If the new where it got deleted from knows that it was in inbox, or it was in archive when it was deleted, then having other places to deposit files, such as

Waste of Hdd

Generic

Bad Meme

could be user made, and while its more steps to deleting the files, it would be tagged in the detailed status as 'deleted from 'waste of space'' and effectively work as an explanation of why things are deleted.

If this is possible, Ill vote for it over just deleted file tagging as more users have a use for more places to put files then they do for tagging reasons for removal, I do understand my problem is a minority but if its possible to solve though a encompassing issue, then ill go that way instead.

I think something like "file sibling" will be great.

for exmaple:

you file A and B.

you delete file B in the duplicate filter in favor of file A because A is better.

when you import file B from a booru, it will be marked as deleted. I think it should point to A instead and give it the tags. and same with the ptr.

>>12152

1. Text, Comic Books, Documents, html/xml/json and code support.

2. Scraping documents or webpages support (see list in bottom)

3. Older video, lossy audio, lossless audio and other arbitrary file support

    https://github.com/mikf/gallery-dl (Python, Art Gallery and Booru)
    https://github.com/Bionus/imgbrd-grabber (C++, Booru)
    https://github.com/Xonshiz/comic-dl (Python, Comic/Manga)
    https://github.com/yuru-yuri/manga-dl (Python, Manga)
    https://github.com/manga-download/hakuneko (JS, Manga)
    https://github.com/Hamuko/cum (Python, Manga)
    https://github.com/NguyenDanPhuong/MangaRipper (C#, Manga)
    https://github.com/JimmXinu/FanFicFare (Python, Novel + FanFic)
    https://github.com/kanasimi/work_crawler (JS, Comic/Manga + Novel)
    https://github.com/riderkick/FMD (JS, Comic/Manga)
    https://github.com/rg3/youtube-dl (Python, Videos) (yes I know)
    https://github.com/adolfosilva/libgen.py (Python, LibGen)
    https://github.com/NadalVRoMa/PyLibGen (Python, LibGen)
    https://github.com/bibcure/bibcure (Python, LibGen/SciHub)

So, I just got done with the first 1000 images on the duplicate processing

Shit was slow to start due to how the program handles zooming in and out, and seeing that progress was VERY slow, I got something called magnifier 2.4

http://www.iconico.com/magnifier/

I have a second 13 inch 1080p screen now,

So with the magnifier being the entire second screen, I put the cursor on the eye, zoom in 10x and flip

Shit significantly increased productivity when looking for dups

im wondering if there is something that could be done in program for much the same effect, or even utilizing the fact the program has the source files to look over to not only zoom in, but possibly even increase resolution till zooming is needed

The "yiff.party attachments" simple downloader is broken, the links it finds are 404s.

>>12168

pretty much doing the same thing but with dsr and [win] + [+] there is a build in solution in win that does it

>>12152

From what I can tell the client UI doesn't hang enough to prioritize that right now

Neural net auto tagging is something I'd like in the future but not before batch/sustained mass lookup

Custom thumbnail ordering might be useful for some but I see little reason to prioritize it now

My priority list would likely be

1) improve local booru and go ahead with migrating to client api

2) batch/mass tag lookup and application, maybe with an option to negate application of tags before committing to either local or remote tag repo

3) db storage and filter workflow to open up alternate file support and maybe squeeze out extra performance

4) split windows/tabs for a/b trash application on manual file import

In the manage tags window, can we get the option to NOT remove tags we type in manually if they already exist? Typically when I type it I want it added, if I want to remove tags I double click them with the mouse. The way it is now is just annoying.

I didn't follow hydrus closely. since the python 2 to 3 move.

Are pages and filenames metadata yet? as in:(on pixiv, this filename and page if part of a serie, on nijie, this filename and page, in this tank, ect) Is it even doable?

Do we have a separate editable metadata box?

Can the tag be weighed like what we see in vndb (very useful for comic/image set tags vs illustration tags?

Do we have a separation between spelling error hard siblings (always replace) and alias soft sibling (never replace but appear as).

can we alias a namespace easily?

And most importantly, do we have a way to organize the subscription per circle/author and not per site.

If metada are added, it should be possible to sort by specific elements of the metadata, the ike of pixiv work:#

Thanks for bearing with me.

>>12173

this is workflow I use often, I don't take my hands off the keyboard when tagging so I need to be able to delete tags by typing them. If dev changes this please do it with a setting.

>>12152

My wishlist from top importance to least:

-Improve duplicate db storage and filter workflow (need this first before alternate files support)

I think the duplicate filter UI could use some work. One thing is that the top grey panel covers a lot of the images. Transparency and the ability to move the panel would both help.

Another thing that could help is using symbols and color coding to make it easier and faster to recognize the details of the image.

Maybe an icon with arrows could indicate resolution/dimension of the image. If the icon is green, it mean the current image of the comparison is higher res. If it's red, it's lower res. A green calendar could mean more recent, and a green T could mean more tags. Grey could mean it's a tie for any of the aspects, and black could mean zero (as in no tags). There could be PNG or JPG icons to show file type too.

-Reduce crashes and ui jitter and hanging by improving ui-db async code / Just catch up on small work for a couple of months

Some good old fixup work would be great too though.

-Improve the client's local booru (this likely now means a backend migration to the Client API)

I had some thoughts about basing the booru more around the idea of shared pages than the current state of sharing batches of files.

-Add popular/favourite tag cloud controls for better 'browsing' search

The more browsing features, the better!

-Add file modified/creation timestamp searching

This could be really handy for going back into previous years of my collection where the tagging isn't so thorough compared to today.

One of the biggest areas that need improvement is the duplicates section: it will be so convenient to be able to check for all duplicates inside an opened page

>>12175

There should absolutely keep being a keyboard friendly way to delete tags,

but IMO a better solution than toggle would be prefixing a tag with minus to delete it.

either that or a modifier like alt+enter

>>12180

since dev has made or with a modifier key too I feel like a modifier like that is a good idea.

>>12169

oops this was supposed to go in the main release thread…

>>12175

Sure, that's why I suggested an option for it. There's that cog button right there just waiting for it. A modifier key is a good idea too though.

Since people are talking about shortcuts, I think I'd be great to go another step further and update the shortcut system to accommodate an entirely keyboard-driven setup. Currently there are some things you have to use the mouse for, not to mention window focus glitches that require you to refocus with the mouse.

>>12188

Hydrus in Emacs when?

A U D I O

U

D

I

O

>>12174

The Pixiv and nijie downloaders work but its still too messy to use in Hydrus. You're biggest concern will be trying to translate all those Japanese tags if you don't know Japanese. From 1 artist alone, it took me 1-2 weeks of changing all the Japanese tags into english using siblings as I don't know any other better way to do this.

>>12191

Vim or bust.

Anything from >>11761

>>12154

You might wanna check >>12166 which has a broader scope.

hi! could you please consider adding pagination (booru-style) to files dialog?

>>12154

Yeah, a potential route is for me to provide both actual cbz and virtual, either in a clever or dumb way, so you can access both the archive and the pages as separate files. For a first step though, I would only start with adding actual .zip/.rar inspection and presentation, so it would always be a single file in hydrus. What this job really is is adding single-file-but-multi-page presentation support to the media viewer, some kind of a secondary previous/next when looking at a cbz object so you can page through it as well as move to the previous/next actual media file.

There would also be ancillary stuff like cleverer cbz/cbr thumbnail generation from first page and so on.

>>12155

Thank you. I will add public url repo to the list. I want to avoid heavy work on the API for a little bit and see how fitting it into regular weekly work goes, so I will skip it this time.

I am changing "Add file modified/creation timestamp searching" to "Add file modified/creation timestamp searching and sorting".

>>12156

IPFS multihashes sort of fit into the idea of a URL Repository, so perhaps a second version of a URL Repo would support other identifiers. It could also fit into a completely different Hash Repository, for public sharing of md5 and sha1 hashes as well, which might be neat for some future booru lookup operations where looking up the md5 of something you don't have may be useful.

Would you like me to add at IPFS Repo to the list, or maybe just a full iteration on current IPFS support first? I still want to add that no-blocks upload system, so maybe that is better done first.

I would run a PUR I think, but it would be as optional as the PTR.

As I said just above >>12211, I will keep API off this cycle. I am happy with the first version I have just done and want it to get a bit of use and extended feedback before I go heavy back to it. It'd be nice if I can just do little work here and there without having to wait a whole big job iteration to push it forward.

>>12160

That's covered by "Improve tag siblings/parents and tag 'censorship'". I'd love to have user-custom namespace siblings, including from something to nothing, like if you don't like 'clothing:bikini', you can have that appear as 'bikini'.

>>12161

Yeah, multiple local file services is probably your best bet for this. I'd add the particular local file domain's name to the deletion reason.

But tbh, now I have that deleted reason table, I may also sneak in a quite option for you that allows you to set up some custom deletion reasons and then add a new button to the delete confirmation box. It'll let you select that 'bad meme' reason and pass that instead of the current generic reason. Please play with the existing new system for another week and then let me know what sort of workflow you would like.

>>12162

Yeah, I would like this. It will require some db prep work before I can do it, so you want "Improve duplicate db storage and filter workflow (need this first before alternate files support)" for now. File alternates will have some sort of file family relationships support with it.

I vote for collections (so we can read/archive mangas etc. properly) and audio in the media viewer.

PS: Just something to consider: blacklist files based on pHash (similarity).

>>12168

I have this as "Improve display of very large/zoomed files in the media viewer" for now. I'll move to a tile-based rendering system, so I am only storing what's on screen in memory. Atm, I make a giganto bmp which is pretty shit for several reasons.

>>12169

>>12187

No worries, should be fixed in 348.

>>12174

No, I am afraid most of that stuff is not in yet. Some of those issues would be worked on in the items in the big job list.

>>12173

Sure, that is small and I can add it in the next few weeks. I'll make a new 'add only' check item in the cog button of manage tags dialog.

I am adding "Add animated thumbnails for videos (animating on mouseover)" to the list.

>>12181

>>12180

>>12188

>>12191

>>12198

Yeah, you probably want "Improve shortcut customisation, including mouse shortcuts". There's still a lot to do here, but the ideal is to allow customisation for all actions. I'd love to speed up tag management for keyboard-only, and it would start there. So much shortcut processing is still hardcoded.

Run with command line (all argv or xargs like), context menu for thumbnail grid.

I vote to call the Public URL repository

PUR for consistency, or

PURe because it's memorable

>>12201

Can you explain this more? What's the 'files dialog', and how would you like it to page? Would you like the thumbnails from a regular search to be split into pages?

>>12217

For sub-paged collections in the media viewer, I think you want cbz/cbr support, which tackles this basic problem of adding two tiers of pages in the media viewer. I can tack collection paging onto the media viewer as part of that.

Blacklisting by similarity is something I would love to do when the duplicate filter is more mature. If I can be 99.98% sure in code that one file is the same as another, I can auto-dupe clear on several rules for easy situations (like exact pixel dupes) and only leave complicated/fuzzy dupe processing to human eyes. And as soon as there is auto dupe processing, I can do it on import and fail the import with an appropriate message.

>>12224

Thank you, I am adding "Allow multiple custom 'open externally'-style file launch commands for files" to the list

>>12161

I would go with a total 1 byte ranking number, which you can optionally assign labels to. Lower numbers have higher trash priority.

>>12226

thank you for reply

by "files dialog" i meant "my files" page

yes, I guess you're right. the only option i see when i have many images selected by query but i don't want to load all of them is to add system:limit to search, but there's no system:offset. it would be awesome to have the opportunity, for example, to set number of thumbnails on page in hydrus options and scroll through this pages even without system:limit. for sure not all thumbnails/tags should be loaded, but only ones that are on page

from gui view, i guess arrows with [curr. page]/[tot.num of pages] will be enough

>>12225

If that is the case PTR should be called PeTR

>>12233

>>12233

>Why? There is already a good software for those things. HappyPanda/X for example.

Indeed, the most zip/cbr support hydrus needs is being able to make thumbnails for those.

>>12233

I kind of agree with this, some things just shouldn't all mix together. Doujins just add a whole another layer of tagging that just doesn't mix well with single images, videos, txt files, etc. All this stuff is really starting to clash with hydrus. Hydrus needs to be split into a different programs if you really want all that extra stuff or running multiple instances of Hydrus(which I'm currently doing for videos but mostly just testing it out).

I suggested this once but a separate version of Hydrus that's meant solely for doujins would be a pretty nice idea as some people might not want to use a webclients like happypandX or LANraragi. Managing doujins is a whole lot easier than single images so I doubt you'd need to do much work to manage a program like that.

Siblings handle some cases where you want to replace namespaces, but this doesn't work well for single images or long tags (which you have to retype).

I'd like an option to change the namespace for selected tags in one or more on the fly, e.g. change an unnamespaced tag to an actual name space, or replace one for another.

For example, I'm manually changing many creator: tags to blog:, tumblr: instagram: or whatever because the account owner is not the actual creator of the work. I can do this with some SQL-fu, but it woul be much easier to say, right-click a tag in the Manage Tags screen, select "change namespace", then type in the new namespace or select it from a dropdown.

It would be nice to have some kind of search-and-replace interface for tags globally when doing this kind of maintenance.

>>12235

>Managing doujins is a whole lot easier

I might be slightly biased towards this(I'm the dev of one of the webclients and it's not HPX), but doujin providers are much more annoying to scope for metadata than boorus. Alongside this, you also can run into encoding issues with the filenames inside of the archive.

Hydrus bypasses encoding problems by renaming files currently (and it's probably much better off thanks to that), but that wouldn't apply to files inside of a cbz.

I'd personally rather see support for more esoteric formats like xml/svg than manga, but that's just me wishing for stuff that'd fit my workflow more. With the original HP dead there's not much left in terms of desktop clients for doujins.

>>12218

lol well aware of those reasons, the bmp has eaten all my ram before stopping its attempts to render a few times.

Oh, one thing hdev

I have gone into this in length in the past with manga viewing/viewer version.

If you go down this road to actually doing a manga version before you go further would you be able to have a pause like this for input?

personally i'm of the mindset that you would have to 100% make all storage user parseable. with images, I have clumps of who gives an actual fuck what its named, so if hydrus dies tomorrow, I lose nothing, but as far as a manga reader goes, if hydrus dies, there goes my entire db/collection.

Personally, I think that it would be for the best to branch hyrus into 2 programs at that point, one for manga, one for images, which while having the same base codebase so they can both be worked on at the same time, they would function in how they archive and present things in 2 radically different ways.

>>12233

>>12234

>>12235

Hentai panda x is fucking garbage

Hydrus with its current tagging system and a public repository could tag every manga in existance and it would all come down automatically with little to no hassle.

because hydrus would take control of the hentai/manga itself, it would never run into the 'woopse, you moved a file, all the info is now gone' that every current option with tagging has,

it has nearly all the ui elements that I would need to replace acdsee as my manga reader

The only issue is its not user parseable. which would need to change, and why I propose a split into a manga version and an image version, using the same codebase, just different things get ticked for different versions, images would all be stored in hashes, manga would all be stored in group-author/manga/autor folders group-author for hentai/doujin manga for known manga and author for oneshots.

This way if hydrus fucks itself, it has saved everything in a non destructive method and likely saved it in a way that would put anything we currently do to shame

every few months I get so pissed at acdsee that I go out looking to see if ANYTHING can replace it, and long story short, nothing can, but hydrus is SO fucking close to being able to do it that I would love to see it try, most cbr/comic readers fail miserably because they rely on outside programs to do something, usually acting as a browser, and this sucks because I cant one hand read, what I mean is with acdsee I get to a manga I want to read

enter, down, up, enter

i'm now reading the manga

page down read till done

enter, backpage im out

down and i'm on the next chapter

repeat above and i'm now reading the next chapter, modify for manga of choices distribution method.

The closest replacement I have found is comicrack, but the whole system is near perfect, yet a few steps away from perfect at the same time. I have to use the library in program to save any data and I can't modify files on hdd through library so quick removal of shit is not possible, I also can't do easy dup searches and have to rely on things being named correctly. so shit is very suboptimal from a archive management perspective, the way I have it set up it wont straight up loose info I add to it if a file moves, but i'm not able to even attempt to file manage anything anymore or else shit will get fucked.

I'm honestly using this just for porn as its a far better way to do it then acdsee at this point, and hentai panda x cant handle a 45000 file archive without shitting itself horribly. comic rack, while its dead, does a good enough job as an intermediary between acdsee isn't good for porn, and when hydrus adds manga to its functions. the worst that will happen is I either need to manually redo everything I do in this program, which honestly may be fairly simple, I don't plan on going overboard, due to the last time I did that with a tagging program, I lost 10k tags because I thought they were file based and moved with files, not program library based.

Have an idea for a duplicate auto mode.

going through the dup finder, I am encountering quite a few jpeg images that were converted to png at some point, the png being 3-5 times the file size for the exact same image. would there be a way to check for duplicates that are jpeg and png, then auto pair against each other to see if they are literally the exact same quality, or within a tolerance where the increased size does not make up for the extrem minor gain in quality if there is even a gain at all?

>>12243

Just make a separate database if you don't want to mix manga with images

>>12243

>>12249

Be sure to check https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/issues/70

Personal opinion: It is better for Hydrus to have a rewrite in Qt before we should have complete comic support.

>>12272

human-oriented sorting is a problem.

Also > duplicate thumbs

>>12272

https://github.com/SethMMorton/natsort there we have it, HyDev take note.

>>12272

Also if you want a fast implementation https://github.com/sourcefrog/natsort

>>12247

Yes, that's basically something like these: https://github.com/andrewekhalel/sewar or even https://github.com/lidq92/CNNIQA or https://github.com/lidq92/CNNIQAplusplus … and so on.

Having these available in Hydrus' duplicate filter (or its revised version) should help a lot.

That said, you should only expect imperfect reliability in fully automatic mode. You won't ALWAYS get very accurate scoring or even just be able to identify the "better" image automatically. These also generally can misidentify variant images as "the same but worse", and other mistakes like that.

>>12152

Like the other anon implicitly did, I also propose the ability to run some more of these image quality / image similarity metrics as a possible feature. Would be nice to have more options to populate the filter and assisted automatic scoring from within the duplicate filter.

At he same time, it would probably be a good idea to make the duplicate filter more modal. E.g. "this is above the certainty threshold you set up, so here's how the weighted scoring of the algorithms you picked would solve this and the corresponding scores - confirm?". Probably also needs some thought put into a colored multi-selection GUI thing that makes it quick to see the scoring & resolution to manually deviate and fix mistakes.

I don't know if this is a feature already or not but a sibling thing that could have multiple dependencies so that it could go like

>if [arist] + [ocname] then swap to artist:[artist] + character:[ocname](artist)

would be very convenient, expecially for scrapping tags from places like furaffinity where people don't use underscoring properly

On the subject of character name.

Would there be a way, far in the future to link 2 tag together.

Instead of:

character:character_(series)

use:

[[character]]x[[series]]

Where x is a dynamic taglink action.

And while we're at it, instead of namespaces, multiple list of tagtype a tag can be presented as for that specific media.

Tear me up, it's brainstorming more than a definitive solution. I don't even know if it is possible.

>>12277

Never trust Neural Network systems verbatim. Always find an expert system that can work well first.

>>12282

>>12277

true, it wouldnt be perfect, but right now I have to parse between two images and when the quality is close, what could take 1-5 seconds takes 10-20 seconds.

if we had dup tiers, like the current one is a blunt and stupid shit looks close, that is perfect, sadly I don't have any examples on had as I deleted them, but there was a scalie thread on /trash/ where some jackass came in, made the shittiest 'corrupt' images that were 7-8 times the good images file size, in a thumbnail looked passable most of the time, but full size, it was unrecognizable. if a dup filter was more accurate, these duplicates would get overlooked, what I would like is this

current duptetector as a base line

from this a more stringent dup detector to check the base lines work. this should filter out alternative images from normal dups

and from there a far more stringent one that would do something like jpeg to png comparisons, and if they are close enough to trigger here, in nearly all cases the jpeg was at some point converted to png, so in this case, scrapping the png would be done as it would save space without needing to go though with a fine comb by hand,

with that final level, imagine that the two images are just shown, right click is keep png, left click is keep jpeg, and due to how close they are you almost never click png. at least this is how i imagine it, dups going through 2 dup filters and then a third png to jpeg filter would turn the 10-20+ second checks into 1-5 second confirmations.

>>12285

I will add more information to the Optimization thread on the list of expert system vs NN repos

>>12286

Personally I preferred to KISS and just suggested a few python frameworks that might be easy to hack into Hydrus - but sure.

- parser revisioning

- remote fetching

- version tracking

>>12152

I'm not sure how big this is but here is a suggestion: implement an icon and search command for files which has any notes attached to them.

A probably small thing that would help me a lot would be an "delete both" button in the duplicate filter - i know that i can press del but i usually dont have a hand at the keyboard while filtering and then i also still need to press delete both anyway…

>>12232

PeTR

Public

Emacs???

Tag

Repository

- Cookie management from API

- Tag statistics from API so if you search for a_totally_sfw_tag, and it produces a lot of creator:cname , you know you should sub to cname

>>12298

Public Integrative Tag Repo

Public Incorporative Tag Repo

Public Interdependent Tag Repo

Public Ingrained Tag Repo

(PITR)

>>12166

You might wanna get https://github.com/deanmalmgren/textract (this is some good stuff for text-like documents)

>>12230

Thanks for clarifying. Unfortunately, this is not trivial to do for hydrus. Any paged system needs sorting, and hydrus supports many clever kinds of sort, so in order to implement this, I would need to load 'media' metadata for every file in a search result before I could fetch the first page of results. This would not save much time from the current system, where most of a search delay is in fetching that same media metadata. The proper solution, and I imagine how the boorus probably do it, is by having a sort cache (and they have page caches as well, and generally simpler searches to cache), to cross-reference search results against to figure out page slices. This is more complicated than I want to make hydrus search code at the moment. I am happy with being able to display and manage thousands of results at once in the main gui, and I also don't want to further complicate the viewer with paged management and load code. As you say, I encourage users to add 'system:limit=x' if they want less laggy searches.

I would be interested in your further thoughts if you have certain scenarios where search is very slow. If there are particular instances where the client runs very slow for you, I'd love to help it run faster.

>>12213

IPFS repos would be important, also an advanced API that can trade IPFS hashes and images would be sweet

>>12234

>>12233

>>12235

>>12243

>>12245

>>12248

Yeah, I am mixed on cbz. I like the idea in the sense of waving a magic wand and having great support, but I can't do that and I know I fall to feature creep too easily. If this is voted on, I would try to make very simple support and see how that goes, and then iterate on it in future if it proves popular. I can't out-compete the programs already out there, but I can do some simple stuff, and ancilliary code like navigating multi-page single-file media will have uses for things like file alternates.

I really want all future big jobs to be small improvements and experiments, ideally 6-8 weeks and pref no more 12, so I don't get bogged down like the downloader engine overhaul. I am open to experiments that fail and don't want to get emotionally attached or fall into sunken cost fallacy.

>>12236

Thanks. Yeah, I would like easier sibling workflow, including from the right-click menus, as part of a tag sibling/parent improvement. I would push in this direction with "Improve tag siblings/parents and tag 'censorship'".

>>12240

Yeah, your dialog mock-up is exactly the sort of thing I was thinking of. I'll have a new options panel somewhere that turns on the advanced mode of this dialog and let you set up some favourite reasons and custom entry. Now that I have the 'set a reason' infrastructure in place, this will not be super difficult to add. I expect to have it in in the next few weeks.

>>12247

>>12277

Yeah, my first push here will be to set up a system that permits auto-decisions in a sensible and generic way, along with user ability to control what is permitted, and then in future hang new auto-decision systems on it. The 'this is a png copy of a jpg' seems like a nice simple way to start, and I know I can do very quick detection of that by just hashing image pixels. Then maybe explore some 'this jpg is definitely lower quality than this one of same resolution' stuff. The way to slice through dupe mountain will be through automatic systems to reduce the human drudgework, but I am similarly leery >>12282 >>12283 of anything too clever/vapourware to start with. Most of all I want to get the infrastructure and maintenance processing code in, and then we can test all kinds of different comparison systems for our exact purposes.

>>12272

>>12273

>>12274

>>12275

Thanks. I have 'human' number sort capability in hydrus already, although I am sure there are still places to apply it. All numbered tags should sort like this atm. I am pretty confident I can get directory listings and file access of rars and zips with the python libraries I already have. Any first version of a cbz viewer would be simple and just read through the internal pages one by one, no bookmarks or per-page metadata or anything. Just something that lets you penetrate the 'list of numbered jpg' zips already in your db in the media viewer (and rename to .cbz or whatever, so you can 'open externally' to your preferred comic reader).

>>12277

For new image recognition techniques, yeah, I designed the search system to make this possible. Much like >>12319 , the main push of duplicate detection 1.0 was to build a search system that could handle many search systems and hang one simple 'looks like' system on it. I can fairly easily add new techniques to support rotation or colour similarity or whatever on it now. This would not be my urge at the moment, as this simple system we already have the biggest problem is there are way too many to go through, so the processing workflow is now the weakest link, but once we have that more under control I can work on this.

>>12278

Yeah, that's a tricky one. I know exactly what you are talking about, and I would love to have a nice system for it, but the actual guts of how 'if … then' tag relations would work are way more complicated than I am confident I can currently support. For siblings and parents, my first priority is to improve the data store behind the whole system first. Once that isn't on fire behind the scenes, I'll consider carefully adding this sort of power. I am sure it could go very wrong if not thought about, so it'll be baby steps until we have some real world experience.

>>12281

Both of these thoughts are on my mind. Adding tag siblings revealed to me a big set of pain in the ass problems related to tag definitions I had not considered before.

For both of these, I think the ultimate far future way to solve them is to have a tag definition structure "Add tag metadata (private sort order, presentation options, tag description/wiki support)", where clever metadata can be applied to tags.

So you could say:

character:shimakaze (kantai collection)

And the tag definition, which would essentially be a cleverer iteration of the current siblings and parent system, would say "this has 'series:kantai collection' parent" and also perhaps "this can be displayed to the user as 'character:shimakaze'" without destroying the unique tag identifier through merging with some 'character:shimakaze (my oc series, donut steel)', basically being aware of the (kantai collection) after the main tag. I am very much on the side of letting users display tags how they want, as there are many different spergy desires here, and having a system that recognises info about tags lets us do mass management rather than the current per-tag mess and endless firefight.

Same for namespaces. I am experimenting with 'clothing:' namespace on the PTR, but I know some users hate that. It would be ideally better if the tag 'bikini' had the 'property' of "clothing" rather than an explicit namespace to argue over, and then a user could say 'when a tag has "clothing" property, display it as namespace'.

As it is, I expect my next step here will be more in line with little patches. Namespace sibling control (like saying 'display all creator: tags as artist: please' or 'display all clothing: as unnamespaced') seems an easy-ish next step.

"Tags were a mistake." - t. hydrus_dev

>>12288

Thank you, I am adding 'Add version tracking to downloader system objects and explore remote fetching of updates' to the list.

>>12296

Thanks, this is actually a small thing. I assume you want no duplicate action applied, just a basic 'get rid of these two shits' and move on to the next decision? I'll see if I can add a button to quickly do this for 349 or 350.

>>12299

Thank you, I have api cookie management in my current to-do. I won't work on client api as a big job in this cycle just so it has some time to breathe.

Can you explain the 'tag statistics' idea a bit more? Could this be something to apply to the program more generally, rather than just the API, like something to click that says "show me what artists I like and do not sub to?"

Subs need a db-level data overhaul before I can do clever inspection about them btw. But this is something else to integrate into the client api as well–managing subs.

>>12293

Thank you. There are multiple jobs for improving notes that I have not been able to get to. I am adding "Expand file notes system" to the list to cover a general push in this direction. This would include multiple notes and likely connecting the notes system to the downloader as well.

>>12325

Yes, thank you very much!

>>12316

sadly the only program that you would be up against is comic rack and acdsee in terms of full featured program and in terms of comic rack, nearly every feature they have, you also have, and in terms of acdsee you either use crash prone versions from 10+ years ago (version 8/first pro to version 9) or you use the recent versions which are pure bloat for unicode support, there is very little middle ground with acdsee due to so many of the in between versions fucking with features.

>>12319

I honestly never want an automatic sort, as much as it would be good for going though my cluster fuck, I am still getting images that are full featured images going against 23 byte black boxes. running into this makes a pure auto system unacceptable to me, I would accept an auto system that looks at two images, and then has me spot check

lets say it finds a better worse pair,

it presents me the better and it presents me the wrose

green border is better, red border is worse

I scroll though like normal, but left click confirms green, right click confirms red.

Confirmed green acts like better worse does currently,

confirmed red takes it out of the auto figure it out area and into a manual pick.

Are there any plans at all to (at least have the option to) keep metadata of individual tag mappings, like a file's tag history or keeping track of which process added this tag to this file?

Like was it typed manually, was it imported while scraping, etc.

I assume this would bloat the DB by a big factor but at least for my personal use I think that might be worth it down the line

Poll >>12358 !