/hydrus/ - Hydrus Network

Hydrus+IPFS that supports concurrency

https://github.com/sakharovaan/hydrus-unleashed

https://github.com/rr-/szurubooru

IPFS+TOR/I2P is in the works

https://github.com/OpenBazaar/go-onion-transport

https://github.com/cretz/tor-dht-poc

https://github.com/RTradeLtd/Temporal

https://github.com/RTradeLtd/go-ipfs-plugin-i2p-gateway

https://github.com/RTradeLtd/ipfs-i2p-bridge

https://github.com/RTradeLtd/go-ipfs-plugin-i2p-bootstrap

>>13355

>Hydrus is not currently great at handling paged media.

This is a simple ™ fix. Treat known multi-page media types as a special class which can be opened with an external program.

Create a mimetype/filetype database which links to an external program.

It's the difference between web2.0 and 3.0 media handling.

>>13358

As a note, cbr and cbz are technically already supported in hydrus, since it supports rar and zip–they just won't be recognised as comics (yet). With a little options-setting, you can tell hydrus to open zips and rars with a comic reader rather than, say your default zip program, and if the external program is ok catching an open-file request for a rar/zip, you might be able to cobble something together until I can figure out proper 'this is a cbr' detection and retroactively fix these files' filetypes and extensions.

If anyone gives this a go, let me know how you get on.

>>13369

>they just won't be recognised as comics (yet).

This whole fiasco may be enough to finally migrate.

I can organize cbr/cbz/cb7 using tags.

I will test a small sample set later this week.

>>13369

>>13371

Please, the archives should have no compression, otherwise IPFS's chunk deduplication would not work.

>>13381

My rule for hydrus is to not change any bytes when I import. The files you add to hydrus have exactly the same content as the files stored and potentially later exported. The only exception is bmp files, which I auto-convert to png on import.

If I end up ultimately adding 'make a cbz out of these twenty-three thumbs, in this order' tech after I have cbz, I'll be interested to hear on the best sorts of ways to pack these archives, along with any useful json metadata to bundle in and so on, but until then, if there is something helpful for IPFS, it needs to be sorted before the import to hydrus–I won't touch the files for now.

>>13385

Some idea: How can we bolt-on a H@H-like system on a publicly available Hydrus server such that the content can survive the purge?

>>13381

Super wrong, you should always compress the archives as much as possible. Unless you're using the rabin chunker, it won't make the situation any better anyway. But even if you do use it, it isn't going to do you any good because each image will (should) only exist in a single archive anyway.

>>13396

>compressing an archive of images

You'll only save a few bytes, and waste time/energy compressing it. Optimizing the images themselves (remove metadata, lossless recompression, saving grayscale images with a grayscale pallet instead of RGB) would save more space.

>>13355

Hydrus needs split into a different program in order to use it for comics and doujins. I, and a lot of other anons, literally can not use the same tags we use for single images like we use for comics and doujins and vise versa otherwise, we'd need to run a separate instance of Hydrus. The doujin tagging schema is a whole lot different than dealing with single images.

>>13411

>a comic would be sorted by character / team

That's good if you have a few files you personally keep track of, but that system would break itself with loads of files and tags.

What if you have multiple with the same tags? What if the tags change? What if you add a new file? That would require a lot duct tape fixes and would slow down the system considerably.

You import a new file, it goes to the default folder, now you start adding tags, the folder structure and file location keeps changing as you add tags, then there's another file so it becomes character/team(2). Now to actually access the file the best way would be reverse index, as in you have a fuckhueg text file with every file hash that leads to the file name then the program fetches the file, for every single file you need to access, for every single file the program has to check. And the filenames themselves are prone to change if you have a lot of files, so the reverse index file is going to be rewritten all the time.

Instead of that, folder/hash is a much more simple and faster system.

Besides, the tags are saved in a generic database format. It wouldn't take much to make a simple script to extract information from that and automatically rename files if hydrus were to fuck up.

>>13411

>>13412

>>13413

I believe that we need to start splitting Hydrus into sub-modules that allows people to join in on the software testing process (even if dev can't handle co-development he needs to make his work easily editable).

Here are some of my proposed names for the sub-modules (using south-only constellations)

The downloader/subs engine and external download plugin adapters - Telescopium (symbolizes file discovery)

The local tag repository manager + tag visualization adapters - Argonavis (symbolizes browsing of the files)

The image viewer + media player and external program calls - Sculptor (symbolizes the viewing of the files)

The perceptual fingerprinting + quality comparison tools - Triangulum(Australe) (symbolizes local feature networks)

The IPFS uploader and external P2P file sharing adapters - Phoenix (symbolize the rebirth of sharing)

The PTR management engine and other admin tools - Crux (symbolizes the need for moderation)

The API and Web/Mobile User Interface Engines (not official) - Octans (symbolizes 8chan and 8-booru like platforms)

The Docker sets and other system configuration file sets (not official) - Reticulum (symbolizes network management)

The Disk encryption and other security management tools (not official) - Centaurus (symbolizes security over everything)

Only constellations left: Chamaeleon, Dorado (dolphin), Musca (fly), Apus/Pavo/Tucana/Grus (birds), Eridanus (river), Indus (Amerindians)

>>13390

I am afraid I am not familiar with that system, so I can't speak too cleverly. It is basically dynamic hosting of Sad Panda content to spread out bandwidth? Unfortunately, the hydrus file repository needs some hydrus-specific access stuff to pull files, and it isn't simple to set up a web browser to see it.

However the newer Client API could do this, I think, if a wrapper/interface was written.

>>13411

Thank you for all this information.

I have wrestled with the idea of title/volume/chapter/page info for a long time. It is in the back of my head to somehow split tags in hydrus into two domains–one for search data like 'character:samus aran' and one for longer one-off descriptions like 'title:samus goes to mars'. It feels to me like descriptive data would benefit from a separate data structure than search data. My thoughts on this are incomplete.

Although I played with volume/chapter/page sorting and similar from the start of hydrus, supporting chapters and pleasant narrative viewing has never been great. For the moment, I encourage people to stick with named .cbr files or whatever and use a program like ComicRack. It just does the job better, and for these sorts of nicely named series of files, it is fine to manage them with filenames. Although having 48,900 may be stretching it!

I don't think I have time to make hydrus a really good comic reader in any short timescale, but I expect some basic cbr/cbz support will come in the next year or so, and we'll see how the upgrades that come with the media viewer for that go. Having dealt with the awkward hassle of page tags, I am not a huge fan of splitting comics up into individual pages any more, so I'd like to see how chapter/volume files goes. I'd like to generally push in the direction of something like ComicRack, with a couple of bells and whistles like bookmarks and so on.

>>13424

My thoughts are simple, I would love hydrus to handle/manage my comics but pic related on how hydrus handles images, is just not acceptable.

now, take comic rack as an example, you have essentially 3 different windows/tabs, the library, the folders, and the pages,

Library is a catch all 'I tagged shit I rated shit' amalgamation to get everything.

folders, at least for hydrus, would be the hydrus imported comics, because I don't know about anyone else, but some days I go though my folder and just look though it and notice hey, I have aiki, I liked that, let's re read, If this was just looking at hydrus imported files, that's fine too.

pages would be where a focused manga gets opened and you can see inside.

personally I have files from exhentai that all come in some form of zip, I get files from nhentai that all come in a folder, and I have files that come from other sources that also get made into various formats, the ability to tell hydrus that this zip is a chapter, this zip is a volume, or this folder is a chapter/volume then move those folders to a place thats user parseable would be needed.

now the main reason I need it to be user parseable is, see the aiki above, along with my other post, if hydrus shits the bed, gg, you are never getting user parseable files back, that 48k+ files is just the porn section of my manga,

now, I honestly don't see 'title:samus goes to mars' as a tag I could ever use. because that would also mean that every single doujin I have would also have titles, and gg on ever finding what I want again without user parsability.

Now, hydrus doesn't need to store the files as rars, it could default to folders, and tone itself down I have fiels labled along the lines of

'are you fucking kidding me how long does this shit need to be for a file name there is no fucking way any sane non retard named their shit this.rar'

the inside is

'are you fucking kidding me how long does this shit need to be for a file name there is no fucking way any sane non retard named their shit this'

the folder and the files are

'are you fucking kidding me how long does this shit need to be for a file name there is no fucking way any sane non retard named their shit this - 1.jpg'

but with hydrus it could be toned down to

'are you fucking kidding me how long does this shit need to be for a file name there is no fucking way any sane non retard named their shit this'

'volume 1 page 1.jpg'

and all be inside one big folder for the series, for manga this would especially be better then well… I think my naruto folder had about 20 different styleings of how people named shit to the point going from one chapter to another is a fucking chore in and of itself, hydrus could cheal that up with

'naruto' the folder name

'volume 1 chapter 1.rar'

or

'v001c0001.jpg'

and slowly add to it that way, as its not only user parseable it also presents itself in the least pain in my fucking balls way so the most programs could use it. could even have it be

'v001c0001 - group name.jpg'

While that would be more of a pain in the ass with many groups doing the same book, user parsability and a way to quickly remove ones that you don't want would also be fairly high.

I just don't believe in tags for comics or manga, there are moments like what chapter is franken frans zombie arc where tags would come in handy, but for the most part, genres are what I would filter normal manga by, now tags on porn… now we are talking about something useful because me being in the mood for X one day doesnt mean I will be in the mood for it the next, and looking at the names alone doesn't tell me their content unless I explicitly remember the content.

This is why I think the need to make a split in hydrus from the image version and manga version would need to happen because I don't want the images getting mixed in with manga/hentai when I want to look for images, and I don't want a 'naruto' search being harder then it needs to be when I want to read a manga.

>>13424

>>13435

I think what he wants is just a "collections" feature. it's an ethereal item that is just a collection of other items, and can be represented in sql with a closure table. tv shows/books/manga are really the only scenario where hierarchical data makes much sense, although genres are still a tag so..

but besides that, these are all pretty much UI discussions not data structure ones. you can model everything relationally, hierarchically (if you're insane), with a graph, etc. you can even represent a graph database inside a relational sql one, but that stuff doesn't matter to the end user. tags for everything makes sense from a cataloguing perspective, you just have to make the UI represent the data in a more user friendly way. there's probably no point separating "descriptive data" just because it feels intuitive.

the only thing that really needs to be done to solve this is just to have separate "views". a manga "view" would just hide anything that isn't series/chapter/page like you can already do now, but the big difference is that you can present it differently to the user. collections can be used to prevent database bloat, i.e. you don't need to have genre:scifi on every single page only the parent node, but again from the user's perspective it makes no difference as long as it can scale performance-wise.