[ / / / / / / / / / / / / / ] [ dir / random / 93 / biohzrd / hkacade / hkpnd / tct / utd / uy / yebalnia ]

/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Catalog  Archive

Name
Email
Subject
REC
STOP
Comment *
File
Password (Randomized for file and post deletion; you may also set your own.)
Archive
* = required field[▶Show post options & limits]
Confused? See the FAQ.
Embed
(replaces files and can be used instead)
Options

Allowed file types:jpg, jpeg, gif, png, webp,webm, mp4, mov, swf, pdf
Max filesize is16 MB.
Max image dimensions are15000 x15000.
You may upload5 per post.


This board will be deleted next Wednesday. I am moving to a General on 8chan.moe /t/. This board is archived at 8chan.moe /hydrus/!

File: 4d6f470068e6acc⋯.jpg (338.2 KB,1925x1080,385:216,1517368870.jpg)

3afb9c No.12692 [Open thread]

To quit spamming the Q&A thread (sorry) I agree this should be made into its own thread.

I'd like this to be a place to ask questions on "how should I tag this" for those that don't use the PTR and think you may be able to be more intelligent with how you tag. Also a place to discuss how you tag things and, most important, WHY that way. For example, what sort of issue were you looking to solve or what did it make easier by doing it that way?

T

o start off the discussion - I'm going to write a small essay on my thoughts on tags/tagging. It is my belief that tagging should be, above all else, two things

1) Simple - There are few scenarios where you must think if something should be tagged or not. Edge cases and oddities should be minimized at every opportunity. The number of tags should be kept small enough to maintain the entire list in memory. This is important for consistency. Speaking of which, the second item is…

2) Consistent - Tags should be well defined and used consistently. If a tag is often forgotten to be applied then it is a bad tag and should be either removed from the system or renamed/redefined as to be applied more consistently. In order to keep consistent tags it is also a requirement that all tags are able to be kept in memory to be applied properly. If you have so many tags that you are incapable of remembering them all then you will likely be tagging inconsistently. It is okay to add new tags to cover areas.

A tag that is used infrequently because the number of items in the gallery are too few is acceptable but is discouraged if the term is rarely searched for (either as an inclusive or exclusive term). Remember: tags are meant to help you search for images. This is important when it comes to naming tags.

I use a collection of 288 unnamespaced tags and 11 namespaces. The tags are mostly descriptors of clothes, physical traits, and objects. Namespaces are used for more meta information like character: artist: series: or whether a piece of art is SFW or pornographic.

Some of the more heaPost too long. Click here to view the full text.

11 posts and 4 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5982e6 No.13219

>>13156

Wanna pitch in on how things can be done, judging by this thread? Of course the more variations the merrier

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7ebc5e No.13272

File: f4b95417dc53be6⋯.png (9.93 KB,405x344,405:344,8392002.png)

Boy I'm going to dread the day most of the top boorus start using male and female slave tags. That's going to ruin everything for me.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7ebc5e No.13284

>>13156

>>13219

Not the other guy but what about a collaborative parent and sibling database? I mentioned this once to the dev awhile back when he first added the option to import and export parent and siblings.The idea is something where everyone can contribute to and add to their client database. For parent tags, my biggest issue is trying to connect each and every character to a series or a sub series to a parent series. If there was a parent database that already did this then this could save a lot of time for some people.

A sibling database though might be a bit tricky to pull off as not everyone is going to agree on a certain term used for a tag and certain people can be pretty autistic about their tags. Over the years of using Hydrus, I've noticed how some booru and sites create useless tags that clog up my client. Some would have misspellings, other times I would have tags like "buttjob", "assjob", "butt job", "ass job", "hotdogging" or tags like character:[first name] [last name] and character:[last name] [first name] when they all mean the same thing. With a collaborative sibling database, not only could we eliminate stuff like this by merging everything that means the same but, also pick and choose which term you like best and have hydrus remember that from now on even when you download a new database. This would also be great for sites with different language tags like Pixiv.

Something like this though would also mean you'd need a site other then github and 8chan to pull this off as you'd also need a place to decide and discus a tagging schema that everyone can look through and follow.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5982e6 No.13287

>>13284

> Not the other guy but what about a collaborative parent and sibling database? I mentioned this once to the dev awhile back when he first added the option to import and export parent and siblings.

> The idea is something where everyone can contribute to and add to their client database. For parent tags, my biggest issue is trying to connect each and every character to a series or a sub series to a parent series. If there was a parent database that already did this then this could save a lot of time for some people.

> A sibling database though might be a bit tricky to pull off as not everyone is going to agree on a certain term used for a tag and certain people can be pretty autistic about their tags. Over the years of using Hydrus, I've noticed how some booru and sites create useless tags that clog up my client

> With a collaborative sibling database, not only could we eliminate stuff like this by merging everything that means the same but, also pick and choose which term you like best and have hydrus remember that from now on even when you download a new database. This would also be great for sites with different language tags like Pixiv

I would say that the first step of solving that issue is to have a wiki and a forum/chat dedicated to sorting those issues out. We urgently need a platform to debate such issues.

Here is a basic idea: tag relationship diagrams like [core word] [derivatives] [misspellings] [disambiguation] and [{foreign language}] for siblings, and [subsets] [supersets] for parent/child.

From these we create a page for each tag, from that we can start and sketch out a taxonomy that makes everything clear, like how Wikipedia manage their pages.

If I am allowed to pick a repo, I would choose https://github.com/Requarks/wiki for Javascript-based or maybe a Python-based wiki if the whole of Javascript is a no-go (but it definitely needs page crosslinks, chats/forums, moderation.permissions, branching/forking, archives, pPost too long. Click here to view the full text.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5982e6 No.13288

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: f2bae29e9c77713⋯.gif (102.71 KB,220x166,110:83,f2bae29e9c77713a2456968a02….gif)

f5a323 No.13218 [Open thread]

I had a great week. As well as fixing bugs and catching up on small jobs, I put some time into the new file maintenance manager and significantly reworked and improved the search timings of the tag autocomplete system. Tag results now come in as fast as they can–it feels completely different.

The release should be as normal tomorrow.

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.


YouTube embed. Click thumbnail to play.

1559ea No.13160 [Open thread]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v359/Hydrus.Network.359.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v359/Hydrus.Network.359.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v359/Hydrus.Network.359.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v359/Hydrus.Network.359.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v359.tar.gz

I had an excellent and very full week.

file maintenance

The new file maintenance system now has some nice UI under database->maintenance->review scheduled file maintenance. It has two pages. It is a little advanced, but if you have some hydrus experience, please feel free to have a play around with it.

The first page reviews what is currently scheduled to run. It shows job counts for each type and lets you manually start work on that job type or cancel them.

The second lets you schedule new jobs. It uses the standard file search interface, so you can queue up aPost too long. Click here to view the full text.

7 posts omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1a143f No.13181

>>13175

just looked at the old version and saw the reply

for multiple similar images at once, the only real use I can think of it is I have a full comic/set and I want to remove the images, I have images that hit an 'unappealing' tag, and I want to get every one of them at once, or in my case, I want that tails comic wiped from wasting space. at least for me, haveing them sort is kind of a moot point.

however if you want them to sort, I think adding a queue system to duplicate filter and having files open in a duplicate filter window would work. this way you search one files duplicates, it posts results and goes onto the next files results and duplicates. this would allow you to also have some access to duplicate tools.

as for the other one, I don't know how simple it is, but the database knows that the files were thrown away, so a 'remove deleted from view' would probably do the job, unless you want to expand the 'remove from view' options to be a bit more complex and have their own sub menu.

It could also be handled from a highlight all function, this would add another step to removing deleted from view, but it would also allow something like 'highlight all 'jpeg'' so I could move them to a different window. it may not have a whole lot of use case for search results, but in the case of publishing a subscription, I think it would have a fair amount of use.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

523e60 No.13187

I use hydrus on an external drive and the other day my drive came unplugged accidentally while using hydrus and the program closed as you'd expect it would. Though when I reconnect the drive and try to reopen hydrus I get to "initializing managers tag siblings" then hydrus crashes. Any advice? I have a backup but its a couple weeks old and I'd prefer not to use it. I still have my db folder and client file and everything. Here's the command I use to launch hydrus just for reference. I'm on osx btw.

open -a /Applications/Hydrus\ Network.app –args -d=/Volumes/Drive/hydurs/db

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1559ea No.13188

>>13178

I am glad you like the sound of it. The program does not phone home anywhere, but if you are thinking about using the program and are truly skeptical, I recommend you block it with your firewall to start with. It shouldn't try to make any connection unless you tell it to through a booru downloader etc…

If you aren't a user yet, your enthusiasm may dim a bit when you stumble over some of the unprofessional design or buggy code, though! Everything is breaking all the time.

I am a typical Anon and am trying to make the sort of software I want to use. Privacy and user control is very important to me, so it is baked into the program. I love the imageboard community and enjoy giving back to it through my work. If you do use the program, let me know how you overall find it after a few weeks, once you are comfortable with it. Feedback from new users is always useful.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1559ea No.13189

>>13179

Great, thanks for letting me know. I'm not anywhere near ready to start work on this, but next step for the whole downloader system will be some sort of subscription + version checking system for the various scripts, like you'd see in Nexus Mod Manager, so script authors can update people more easily when sites change like this.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1559ea No.13190

>>13181

Thanks, I will keep thinking about this.

>>13187

Hmm, that is odd. When it crashes, does it throw up any error window at all, or just close itself? Does the close look 'clean', or does it just die in one frame?

My first guess is the disconnection caught the db while a transaction was flushing to disk and so there is some bad data in there, although usually that stuff is recovered from.

If you go to /Volumes/Drive/hydurs/db and check your latest client.log file and scroll to the bottom, is there any traceback error there for what happens after the 'booting db' phrase? A db error generally shouldn't cause a crash, so it is also odd that you aren't getting an UI popup of some sort saying 'hey, the db had a problem on boot'. If it is erroring silently and quitting the program neatly, there could be some error in there, but if it is a full-on actual crash, there won't be any kind of tidy-up statements.

If there is no error, let's check your db files. Here's a help file that is a slight pain in the neck to get to as an OS X user:

https://github.com/hydrusnetwork/hydrus/blob/master/db/help%20my%20db%20is%20broke.txt

It will walk you through some integrity checks to make sure your db is all ok. I think there is an OS X sqlite3 executable in your App, if you know how to navigate that, or you can get the tools here:

https://sqlite.org/2019/sqlite-tools-osx-x86-3290000.zip

Let me know what you find. I am somewhat confident we can recover this db, but we might need to reset somethinPost too long. Click here to view the full text.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



YouTube embed. Click thumbnail to play.

5f3432 No.13106 [Open thread]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v358/Hydrus.Network.358.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v358/Hydrus.Network.358.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v358/Hydrus.Network.358.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v358/Hydrus.Network.358.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v358.tar.gz

I had a great week doing duplicates work and fixing bugs.

duplicates

I split this big duplicates storage overhaul into three jobs, and this week marks the third and final job done. Like alternates and duplicates information, potential pairs are now stored in a unified and more efficient way.

On the front end, you may notice your potential pairs queue shorten again on week. It will also shrink faster as you process in the duplicate filter, which will present more 'useful' duplicate pairs first and apply your decisions more intelligently at the db level.

All the code is simplerPost too long. Click here to view the full text.

10 posts and 2 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

46e534 No.13149

Thank you based dev.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

11bf7e No.13152

Is there any way to search for specific duplicate types?

I like to replace files with the best version available, but I fucked up and left a lot of "this is worse" versions around, and I'm not sure if I'm retarded or new versions prevented generic ">0 this is worse" searches.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

84fe6d No.13175

>>13144

Searching for multiple 'similar files' from right-click is probably doable. Sorting them the other end is probably a hassle, but we'll leave that for later. I'm supposed to be extending system:hash to take multiples soon, so I can probably do them together.

>>13145

That's a good idea. I would say hit right-click->select->(something) where 'something' would be like 'remote' files or similar and then hit remove in a second action, but I can't remember if there is a good option to select those files. I'll make sure that select menu is comprehensive and see if I can do a quick submenu that that does remove. Maybe if no files are selected, the remove menu instead branches to a similar select-type submenu. I often do 'select->archived … remove', so it would be nice to do that kind of stuff in one action.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

84fe6d No.13176

File: 488be98eb47f43a⋯.png (4.34 MB,1819x1786,1819:1786,488be98eb47f43a77a355bdd00….png)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
Post last edited at

84fe6d No.13177

>>13152

Not precisely. I don't have the help written yet to explain this nicely, but duplicate files are now stored in a valueless group that has a single best 'king' file.

In the coming weeks, I will add something like 'system:is_king'. I think if you combine something like [ 'system:num_duplicates: duplicates>0', 'system:is_not_king' ], that will let you search these non-king (i.e. worse) files while excluding the good ones, and you can then deleted them en masse.

Now I have this simpler duplicates tech, I expect it to slowly appear in more locations. A nice plan is to have the main media viewer say 'hey, this file has a worse quality dupe', a little like how danbooru shows this info in a file page, and have a quick way to flick to that without interrupting the main 'list' of files the media viewer is looking at. See if the new search predicates help you do what you want here, and let me know what else would work for you.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: c166441ade92a5e⋯.jpg (86.55 KB,640x545,128:109,c166441ade92a5e23d139088ca….jpg)

27d0d5 No.13153 [Open thread]

I had an excellent and very full week. I have extended the new file maintenance system and written some UI to manage it, added some undo/reset functions for the new duplicate system, fixed a variety of bugs, and figured out an advanced-users-only extention to the IPFS service to permit the experimental 'nocopy' pin mode,

The release may be late tomorrow, as there is more than usual to test.

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

90567e No.13154

>>13153

> and figured out an advanced-users-only extention to the IPFS service to permit the experimental 'nocopy' pin mode

Please dumb it down to the masses such that IPFS can be used by any pleb.

HACK THE PLANET!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

626233 No.13157

is it possible to delete whole filename: from PTR?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e5c95e No.13158

>>13154

It'd be nice if we can figure out an easy-go mode for IPFS. We'll see how this works for the advanced users, and I expect I'll figure a read-only easy mode downloader for multihashes using http gateways, and we'll iterate off that.

>>13157

Not easily yet, but once I am done with the current duplicates work, I'll be adding serverside and clientside tag filters to tag repositories so both ends can prohibit types of tags automatically.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: 442eb99a34c2ff7⋯.jpg (120.39 KB,379x834,379:834,442eb99a34c2ff7c026c477316….jpg)

c54963 No.8396 [Open thread]

ITT post/discuss/jerk off to/improve scripts to use with hydrus

One of my scripts was broken as shit and I just fixed, but I figured it was better to graduate from the Q&A thread and make it easier to find.

14 posts and 3 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0f0a54 No.12602

File: 95ef85ccc17bb8d⋯.jpg (58.11 KB,804x600,67:50,95ef85ccc17bb8da219b1d4081….jpg)

Here's a quick python one for md5 search.

import sys 


if(len(sys.argv) < 2):
print("script.py file.txt")
sys.exit(0)


fileName = sys.argv[1]
file = open(fileName)

gel = open(fileName+"-gelbooru.txt", 'w+')
e621 = open(fileName+"-e621.txt", 'w+')
r34xxx = open(fileName+"-rule34xxx.txt", 'w+')
dan = open(fileName+"-danbooru.txt", 'w+')
tbib = open(fileName+"-tbib.txt", 'w+')
xb = open(fileName+"-xbooru.txt", 'w+')
safe = open(fileName+"-safebooru.txt", 'w+')

for line in file:
gel.write("https://gelbooru.com/index.php?page=post&s=list&tags=md5%3a"+line)
e621.write("https://e621.net/post/index/1/md5:"+line)
dan.write("https://danbooru.donmai.us/posts?page=1&tags=md5:"+line)
r34xxx.write("https://rule34.xxx/index.php?page=post&s=list&tags=md5%3a"+line)
tbib.write("https://tbib.org/index.php?page=post&s=list&tags=md5%3a"+line)
xb.write("https://xbooru.com/index.php?page=post&s=list&tags=md5%3a"+line)
safe.write("https://safebooru.org/index.php?page=post&s=list&tags=md5%3a"+line)

Just open the files you want to search in hydrus, select all, right click, select -> copy -> hashes -> md5, wait forPost too long. Click here to view the full text.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0f0a54 No.12604

>>12602

I always forget to close shit. Depending on your computer that script might fuck some files up, here's the fix.

import sys 


if(len(sys.argv) < 2):
print("script.py file.txt")
sys.exit(0)


fileName = sys.argv[1]
file = open(fileName)

gel = open(fileName+"-gelbooru.txt", 'w+')
e621 = open(fileName+"-e621.txt", 'w+')
r34xxx = open(fileName+"-rule34xxx.txt", 'w+')
dan = open(fileName+"-danbooru.txt", 'w+')
tbib = open(fileName+"-tbib.txt", 'w+')
xb = open(fileName+"-xbooru.txt", 'w+')
safe = open(fileName+"-safebooru.txt", 'w+')

for line in file:
gel.write("https://gelbooru.com/index.php?page=post&s=list&tags=md5%3a"+line)
e621.write("https://e621.net/post/index/1/md5:"+line)
dan.write("https://danbooru.donmai.us/posts?page=1&tags=md5:"+line)
r34xxx.write("https://rule34.xxx/index.php?page=post&s=list&tags=md5%3a"+line)
tbib.write("https://tbib.org/index.php?page=post&s=list&tags=md5%3a"+line)
xb.write("https://xbooru.com/index.php?page=post&s=list&tags=md5%3a"+line)
safe.write("https://safebooru.org/index.ph
Post too long. Click here to view the full text.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c3e71a No.12631

>>12604

Thanks, this is pretty useful. Sankaku has an md5 lookup as well:


sankaku = open(fileName+"-sankaku.txt", 'w+')
sankaku.write("https://chan.sankakucomplex.com/?tags=md5%3A"+line)
sankaku.close()

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f8c10 No.12982

>>12604

>>12631

>>12602

I not able to get the md5 unless I open each image in the hydrus media viewer first, in which case there is no easy way to select them all and get it for all of them.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4c0673 No.13155

>>8396

Can someone bundle a torrent client with Hydrus+IPFS?

Just found this https://unix.stackexchange.com/questions/44247/how-to-copy-directories-with-preserving-hardlinks

(People in >>>/animu/81926 recommended Hard Links rather than soft links, so yeah)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



YouTube embed. Click thumbnail to play.

1fadd4 No.13035 [Open thread]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v357/Hydrus.Network.357.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v357/Hydrus.Network.357.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v357/Hydrus.Network.357.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v357/Hydrus.Network.357.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v357.tar.gz

I had a great week fixing bugs and cleaning up other small jobs.

shortcut and hover window fixes

The recent menubar open/close detection did not work well on OS X and some versions of Linux. It misfiring broke some shortcut processing and hover window display. The good news is that as I went into that code more closely this week, I discovered several other menu, shortcut, and hover window problems that were causing long-term issues. All clients should have less janky hover window placement and behaviour, but particularly on non-Windows.

Specifically, the OS X shortcuts-in-media-viewer issue is fixed. Also, on all platforms, hover windoPost too long. Click here to view the full text.

9 posts omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

df10ac No.13100

>>13035

I think pixiv updated their site again and now the downloader stopped working.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9dd115 No.13101

>>13094

Yeah modified is fine, thanks.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8be780 No.13102

e621 did an oopsie with Cloudflare!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

52812c No.13103

>>13100

Should be a fix coming in 358 later today! They switched up their API, and a user helpfully provided a new parser. Let me know if it still gives you trouble.

>>13102

Can you say a bit more about the error you got here? Was it the gallery page step, the file page, or the files themselves? I just did a quick test here on my dev machine and it worked ok here. I have seen it kick in site-wide for just an hour or so, which I guess are DDoS protection or similar, but I have also recently had several reports like this, where it works ok for one user but not another. I presume it is an automatic CF-side rule kicking in, based on IP or some other profile. Any chance you are on a VPN? If so, if you switch server, does that fix it?

I have a job to better detect the error page returned and giving a better 'CF problem' error and maybe 'ignored' status or something that can maybe try again several days later once I eventually have that download retry-later tech in. My guess is this problem will only increase in future.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8be780 No.13112

>>13103

Yeah CF got hit cause of muhChina, so it definitely is an issue for the meanwhile.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: 8ac27dafade696d⋯.gif (435.05 KB,593x539,593:539,8ac27dafade696d403d724ac76….gif)

c96e83 No.13098 [Open thread]

I had a great week working on the duplicates overhaul and fixing bugs. Potential duplicate pairs are not stored in a sensible way, with the duplicate filter now serving up more useful comparisons and clearing out the pending potential queue faster. Also, the issue with the media viewer not capturing mouse scrolls while not focused is fixed, as is the default pixiv parser, file repository file upload, and several more specific issues.

The release should be as normal tomorrow.

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.


YouTube embed. Click thumbnail to play.

55d74e No.12978 [Open thread]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v356/Hydrus.Network.356.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v356/Hydrus.Network.356.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v356/Hydrus.Network.356.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v356/Hydrus.Network.356.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v356.tar.gz

I had an excellent and busy week catching up after E3. Duplicate better/worse/same relationships are now stored more logically and efficiently, and a variety of bugs are fixed. Maintenance is also easier to control.

duplicates

The second phase of the big duplicates rewrite is now done! Rather than the old system of trying to store better/worse relationships for every possible member of a duplicate group, which is very complicated, requires many human decisions, and is not very useful in practise, this new system instead lumps all duplicate files into a single group with a single 'king' representing the best file of that group.

The grouPost too long. Click here to view the full text.

19 posts and 4 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

13dd46 No.13041

>>13025

>>13023

>>13022

You can import cookies, better help available on the discord

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9825f6 No.13079

>>13013

Thanks. Yes, there will be a fix to seeing same alts again. Might be in this week as I do the main potential data, or in next couple of weeks if it proves complicated.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9825f6 No.13080

>>13019

>>13020

>>13021

Yeah, I'm going to try to fudge something like this. I'm not totally sure on all the edge cases yet, but I will do something. Maybe matching the smallest x or y to be the same virtual zoom height, with some clever exception for portrait vs landscape pairs. and offsetting drag delta by the (0,0) point on both files. I haven't gone into this exact code, so I can't speak cleverly yet, but it is in my big job todo. Thank you for your feedback.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9825f6 No.13081

>>13022

Yeah, hit network->downloaders->manage logins. There should be a gelb entry there by default. Add some credentials, set is 'active', and do a 'do login now' test. (it is worth doing a manual download as well there and then to test it worked for real–this stuff can be unreliable due to all sorts of factors). After then, if your login expires (I think gelb login lasts a year?), hydrus will try to login again automatically without you having to do anything. More info here, at the bottom:

https://hydrusnetwork.github.io/hydrus/help/getting_started_downloading.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9825f6 No.13082

>>13026

>>13028

If you have help->advanced mode on, then clicking 'custom action' in the dupe filter should let you select 'alternate' as the dupe type, which then gives you the dupe merge options and a choice to delete A, B, or both. If you do it from the thumbnails, you should get the dupe merge options too.

I agree that having some sort of 'favourites' custom merge actions will be useful for alternate processing. I think I will do it when I get around to making a proper alternate processing workflow. I am trying to shy away from significant alternate processing in this primarily duplicates data structure overhaul, since it is a black hole of work that I could get lost in for months.

I took away the 'set as not related' from thumbnails because in data terms, this duplicate type is very inefficient if not applied to files that already have potential status. I have left it to the dupe filter and 'show some random pairs' quick workflow. I am still thinking about bringing it back for thumbnails, but at the moment probably won't. It really is a 'set as false positive' rather than 'dismiss any potential info' as it has been previously intepreted. I'm still thinking about it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: c047512639dafcf⋯.gif (365.06 KB,300x300,1:1,c047512639dafcf562b8e5fc24….gif)

9e71b7 No.13030 [Open thread]

I had a great week fixing bugs and catching up on other small jobs. Some OS X and Linux UI issues (including shortcuts and bad hover windows in the media viewer) are fixed or improved, some downloader-page flicker is reduced, the client api is moved forward a little, and the popup system now has a neat collapse/expand button to hide subscriptions and so on while they work.

The release should be as normal tomorrow.

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d7ac54 No.13049

I'd like you to pay more attention to github

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



YouTube embed. Click thumbnail to play.

9c60aa No.12861 [Open thread]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v355/Hydrus.Network.355.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v355/Hydrus.Network.355.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v355/Hydrus.Network.355.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v355/Hydrus.Network.355.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v355.tar.gz

I had a good week catching up on small jobs.

duplicates

I added a 'trash file' button to the duplicate filter hover window, tucked up in a new row with the cog button so they fit on thinner windows. If both potential files are worth deleting, hitting this is an easy way to dismiss them immediately. I also fixed some over-aggressive potential pairs removal from last week's alternates update. Users with many alternates may have lost some valid intra-alternate-group potential pairs, which I expect to add back in once this whole overhaul is complete.

The duplicate action buttons (and several others in the media viewer) are now aware of thePost too long. Click here to view the full text.

31 posts and 15 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

caed90 No.12971

>>12943

Unfortunately, having downloaders know about deletions after the fact is tricky. Completely downloader items only know file hash and whether it was new/already in/deleted at time of import. They aren't plugged into the content update pipeline and only cause additional file metadata to be loaded in the thumbgrid when thumbs are called to be displayed.

Are those 154k/139k counts are related to this duplicate search domain question? And the ~150k count seems low compared to 4M? Assuming you have done your full search at 0 distance or whatever, that sounds correct–it is only searching the files that have potential matches found in the duplicates system. Having 1/15 to 1/10 duplicate files is typical, and since you have 2.6M eligible files, it sounds correct to me.

There's no good way to do a complete wipe. The old system had some hacked-together commands, but it never worked well. This new system, once it is complete in a few weeks, will have logically consistent 'dissolve' commands that will be able to nuke everything. You can do this if you like, but I suspect your duplicates are fine.

Let me know if I misunderstood. And please bear in mind I am actively overhauling this system right now. Potentials will move over in a couple of weeks, going from pairs to groups, which may fundamentally change how this queue is presented in ui anyway.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

caed90 No.12972

>>12945

Not yet. You can hit the 'skip' button, which I think is up arrow shortcut by default or you can hit the double-right arrow icon button, but the same pair is likely to come up in a future filter.

Best solution for now to put it off indefinitely is to exclude it in the duplicate file search domain. You could try adding a 'don't dupe filter' local tag or something and exclude that, or through a system tag if that makes sense.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bb23e3 No.12976

>>12967

Thank you.

I appreciate the frequent updates.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a1cd23 No.12977

File: 63a8739feb2c1fe⋯.png (8.12 KB,1426x41,1426:41,client_2019-06-19_17-06-20.png)

File: 733b85c262d41e8⋯.png (22.95 KB,1074x1993,1074:1993,client_2019-06-19_17-06-36.png)

>>12968

The reason that It did this was an accident. I wanted to click around it, above and below to see what the image was because this particular on had a 'prefer alternate' delete reason, so when I did that I got the second image.

this is fine and all, but the issue is this is also what shows up when hydrus has a bad image/database conflict.

in the case that something like this happens, because inside the import status you are able to highlight files to view, would it be possible to have a 'file deleted' hydrus icon? some text to signify its not an error on the thumbnail

>>12969

With razor, unless they changed it, you shouldn't need a cloud profile to use it, but razer software is absolute fucking garbage to the point it makes logitech's look good. I would honestly look into point of service keyboards instead, as some of these can have a few hundred keys but cost 16$, there are also programmable keyboards, or what I would likely use as my second option, a teensy

https://www.pjrc.com/teensy/

and crash course on learning that.

>>12970

that would at the very least signify the input worked.

>>12971

no, those are 294291 files that are in the db, more or less every single file that is not either a png or a jpeg, currently there are 4,010,123 files in my db, that would mean that there should be 3,715,732 files in the eligible files area, this would mean there are 1,086,053 files that are not currently seen by the duplicate process.

at the very least here's hoping that tPost too long. Click here to view the full text.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

80f670 No.13001

>>12977

Thanks for clarifying. Yeah, I'll stick the trash icon on 'remote' files that were known to once be in the client. I think thumbnails know this already, so I can just draw the icon on.

My thing with these game controllers is having the side joystick. Feels so good to WASD with just my thumb, I can't see myself going back to a regular keyboard for vidya. I may bite the bullet of a Razer. I did some searching a while ago, and I think you need to 'initialise' your Razer controller with the cloud account, then you set your shortcuts, and can then set it to 'offline' mode. You can't edit the shortcuts completely locally and it is seen as a default device until you do it the first time. When my G13 finally packs up, I'll just have to bite the bullet. I'm ok with having a cloud bullshit driver if it is only ten minutes before I firewall it, even if I find it atomically stupid.

I understand your count issue better. I am not sure about the million files. Normally, a handful of files will be discarded if they have no useful search data–such as completely black/single-colour images–but that does not account for so many. As I do more of the current duplicates work, I will be integrating the 'phash regen' that that maintenance stuff refers to to the new file maintenance manager. If your client has somehow forgotten or missed that big chunk of files, it will be easier at that point to requeue them for generation. I'll have some UI for it under the database menu. At that point, I recommend you do something like 'regen missing search data for: system:filetype=png/jpg'.

Adding skip is a bit tricky just because it adds complexity in storage, workflow, and UI. I haven't reworked the potential data storage yet so I can't yet say cleverly if this would be ok or an utter pain in the neck. I will keep it in mind, but I suspect I will not add it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: c4d088c9e8e6b48⋯.gif (121.36 KB,320x240,4:3,c4d088c9e8e6b482aaa0e025bb….gif)

81774d No.12951 [Open thread]

I had an excellent and busy week catching up after E3. As hoped, I completed the second phase of the duplicates overhaul–all better/worse/same quality duplicate information is now stored more logically and efficiently. Duplicates processing is faster and duplicate groups are easier to manage. I also fixed a heap of bugs, including the clipboard watcher popup spam and then recent OS X shortcuts issue, and improved some maintenance code. The exit splash screen now has a 'stop doing maintenance' button!

The release should be as normal tomorrow, maybe a little late as there is a bit more to test.

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

05a5b9 No.12955

File: 3d10edfaea17875⋯.jpg (1.13 MB,1416x1412,354:353,3d10edfaea178757f0d024ff24….jpg)

>>12951

Based and redpilled.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: f16dc8fe807ba8f⋯.jpg (1.46 MB,1289x1821,1289:1821,68390508_p0.jpg)

09abfd No.12677 [Open thread]

Hydrus dev, I'm pretty sure I invented the proof to the big bang. If I get popular, I'll shout you out.

That's not why I made this thread, though. I'm so anxious I feel like vomiting. I never said this out loud anywhere, but besides porn I have nothing else in my life besides Hydrus. I use Hydrus extremely casually, anyway. But Hydrus helps sort porn, so… I was gonna shout it out.

I was gonna shout out my two favorite artists, too. I didn't have any plans to shout out anything else… So I imagine it'd be a huge influx of traffic.

I can't handle this at all. I never asked for this at all. No one will believe me until it happens anyway.

I have an appointment to show it to someone next week. It's fucked. I wish I could just be in protective custody and sleep for 20 hours and it'll be better then maybe.

8 posts and 1 image reply omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5a17d No.12791

>>12719

The majority of people view the weak with disgust, that's the hard truth. It's kind of cruel that society pretends otherwise.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

299d4b No.12842

>>12678

>>12677

>>12719

Sorry, I got caught up in the duplicates work last week and haven't checked the catalog for new threads in a bit. Assuming you aren't giggle-posting, it sounds like you are having a truly hard time. I sympathise deeply, but I am afraid I do not believe that happiness or self-esteem comes from getting love from others. I love you as a fellow Anon, but I am skeptical that repeating that sentiment would validate you.

The only one who can make your brain work right is you. As long as you believe you need affirmation from others to feel correct, you will be open to further abuse.

I strongly recommend you start getting exercise, if you do not already. Go for short walks every day. If you can scrape together a few bucks, get these books:

https://en.wikipedia.org/wiki/The_Feeling_Good_Handbook

https://en.wikipedia.org/wiki/Meditations

They are on Amazon and also in the library. If they ask, just say you are getting Feeling Good for a friend.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

363578 No.12851

File: 7d72a28138cfb76⋯.png (1003 B,113x60,113:60,ClipboardImage.png)

wow very cool

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4d547c No.12860

File: c9e6a1d4848b3e6⋯.jpg (49.67 KB,392x588,2:3,cover.jpg)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

27a3de No.12912

File: 24fdbb89515d447⋯.jpg (72.41 KB,1200x910,120:91,Ck6tWhXUoAAUHNI.jpg)

>>12678

Stop lying about being the dev. Stop lying to yourself to achieve social results you don't actually want. Stop lying in saying that you didn't mean to say anything. Stop lying because if someone gave you what you asked for, you'd push it away because you didn't trust it.

Don't lie. Ever.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



YouTube embed. Click thumbnail to play.

273809 No.12755 [Open thread]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v354/Hydrus.Network.354.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v354/Hydrus.Network.354.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v354/Hydrus.Network.354.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v354/Hydrus.Network.354.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v354.tar.gz

I had a great week. The first duplicates storage update is done, and I got some neat misc fixes in as well.

false positives and alternates

The first version of the duplicates system did not store 'false positive' and 'alternates' relationships very efficiently. Furthermore, it was not until we really used it in real scenarios that we found the way we wanted to logically apply these states was also not being served well. This changes this week!

So, 'false positive' (up until recently called 'not dupes') and 'alternates' (which are 'these files are related but not duplicates', and sitting in a holding pattern for a future big job that wiPost too long. Click here to view the full text.

32 posts and 6 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

273809 No.12838

>>12831

Sure, let's give it a go. 64GB/week. You can see it in your review services if you refresh account right now. If it goes funky for some odd bandwidth-cycling reason, I can always reset it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

463ddf No.12841

Ok hdev, still going though video files for removal, and this is the 8th fucking video that is either the same or damn near the same as the others.

So this got me thinking on if I had the ability, how would I check duplicates for video. while i'm not coming up with good answers, I do have ideas.

first, could take an image every second, or so, and use them to fuzzy match, and if something fuzzy matches it goes a bit more process intensive to see if it is a real match. This would allow for both exact duplicates, and clips from larger files to be found. given what the dup finder can do with literal garbage, I think this could work if its even an option in the first place. depending on how big files are it would pull, it could bloat things out, but ultimately I think it would be an overall good.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

273809 No.12846

>>12841

Yeah, I do plan to add videos to the duplicate system, and I originally designed it to eventually support them. The recent file maintenance system was a step forward in prepping for the CPU work we'll need to do to retroactively crunch the data on this in a reasonable way.

I plan to do something like what you propose. The duplicates system currently works on comparing still images' shapes with each other, and it allows for multiple still image 'phashes' per file, so my task is selecting a good number of useful frames from videos that will match with others.

If it is reasonably possible, I would like do something more clever than just picking one frame per x time units or frames. This would line up right if our two vids were very exact conversions or resizes, but some of the codec changes drop a frame at the start or do 29.97fps vs 30fps bullshit that would desync our comparison. My original duplicates system did add vids by using the first frame, but so many have a black/flat colour first frame that it lead to a billion false positive dupes. Vids are no longer included, and I also drop anything that looks too much like a flat colour image from the system entirely. If I could instead find the x most 'interesting' frames of a video, then 2-second gif clips of 20-second webms would have a higher chance of being matched, and 30/60fps conversions would too.

I don't know, though. That is probably beyond me to do well, or maybe I can hack something that is good enough. I could do something like generating a phash for every frame in the vid and then have them compete with each other to remove similar-looking frames/phashes until the 20 most unique were left. It might pick up a bunch of false positives again with, say, a black screen with a bit of white text on (like an 'intro' title card) though.

Still, I am almost ready to do this now, and dupe work is proceeding, including more efficient storage of potential dupes, so maybe the answer here is to get a simple system in and then iterate on it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

463ddf No.12850

>>12846

when I say literal garbage, I mean the image is drastically different, to the point I cant even see how it thought they were dups, but those are from the asshole who specifically fucked with dup detection when creating trash images, the dup detector is able to have enough wiggle room that even if the images aren't lining up perfectly, it may spit out something useable, and I used 1 second just because no mater what I watch 1 second isnt enough time for a 100% scene change, it should pick up some duplicates from that.

on the title card, you could make a generic here is a black frame with text, and have a few variations of it, this could be used as a compare to X image and trace it if it does, so it would automatically know that everything with it will bee seen as a duplicate.

If you are able to, try to get in contact with the people from what anime is this, and see how they did theirs, it may give some ideas.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

473f2e No.12899

>>12846

Could always check how Video Comparer works. It's the best video dupe finder software I've used.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: 4b005254620b5bf⋯.webm (375.91 KB,480x320,3:2,4b005254620b5bf1b99582f85….webm)

607069 No.12858 [Open thread]

I had a good week doing small work. The error-popup issue with collected thumbnails is fixed, and there are new shortcuts to better allow keyboard-only navigation of the manage tags dialog, an experimental jpeg quality estimator for the duplicate filter, a fun clipboard watcher for easy URL import, and some new options for file viewing statistics.

The release should be as normal tomorrow.

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.


Delete Post [ ]
[]
[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]
| Catalog | Nerve Center | Random
[ / / / / / / / / / / / / / ] [ dir / random / 93 / biohzrd / hkacade / hkpnd / tct / utd / uy / yebalnia ]