[ / / / / / / / / / / / / / ] [ dir / random / 93 / biohzrd / hkacade / hkpnd / tct / utd / uy / yebalnia ]

/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Catalog  Archive

Name
Email
Subject
REC
STOP
Comment *
File
Password (Randomized for file and post deletion; you may also set your own.)
Archive
* = required field[▶Show post options & limits]
Confused? See the FAQ.
Embed
(replaces files and can be used instead)
Options

Allowed file types:jpg, jpeg, gif, png, webp,webm, mp4, mov, swf, pdf
Max filesize is16 MB.
Max image dimensions are15000 x15000.
You may upload5 per post.


This board will be deleted next Wednesday. I am moving to a General on 8chan.moe /t/. This board is archived at 8chan.moe /hydrus/!

YouTube embed. Click thumbnail to play.

e175d6 No.12669 [Open thread]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v353/Hydrus.Network.353.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v353/Hydrus.Network.353.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v353/Hydrus.Network.353.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v353/Hydrus.Network.353.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v353.tar.gz

I had a great week. I finished the basics of the new file maintenance system I wanted, cleaned up the duplicate filter a little more, and fixed a bunch of bugs.

file maintenance system

There are a number of large file re-checking jobs the client wants to do, both now and in the future. Going back to figure out more accurate video durations and image rotations, discovering webms that were formerly incorrectly detected as mkvs, eventually integrating videos into the duplicate checking system, all of these will require a combined whack of maintenance CPU that I don't want to hit all at once. I have previously sketched out some disparate systems for these jobs, but none were really doing the trick, Post too long. Click here to view the full text.

10 posts and 3 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e175d6 No.12713

>>12683

Ok, thanks. I'll give 'move left/right one' and 'move to first/last' a go, which gives you four options, and we'll see how that works out.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

313ccb No.12720

>>12710

Contrary to your opinion, and in agreement with previous anon, you are indeed a saint.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e7d82c No.12747

>>12713

Move left, move right, move first, move last

on the video tab I have

stupidly large remove

convert down

not hydrus

view again and decide

with

check before remove

music video

in for good measure

the first set is a wide range of reasons for a quick cull, the second set a second pass, with move left and right, I could easily set it up so its in between 2 tabs and move it to a second for more… hell, even a worst case scenario I could have multiple page of pages and filter that way.

not as set and forget as proper defined hotkeys, but should greatly increase the rate of sorting because I don't have to move them with the mouse anymore. definitely will be a god send once i'm out of video land.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e7d82c No.12753

Ill post this again in the new thread too,

I decided that the converted down images will get will get a rating of

Converted Down/Keep Large with 2 stars and not selectable as an option

it allows files to either be converted down and hidden from a search, or specifically keep the large file and hide it, also removed all files from the archive, everything needs a filter pass. these images that were converted down were interesting enough for me to keep/convert down but not really sure if I will long term storage them or not.

when I en mass rated them to converted down I got that error

InterfaceError

Error binding parameter 0 - probably unsupported type.

Traceback (most recent call last):

File "include\HydrusDB.py", line 561, in _ProcessJob

result = self._Write( action, *args, **kwargs )

File "include\ClientDB.py", line 12905, in _Write

elif action == 'save_options': self._SaveOptions( *args, **kwargs )

File "include\ClientDB.py", line 10112, in _SaveOptions

self._c.execute( 'UPDATE options SET options = ?;', ( options, ) )

sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type.

It doesn't seem to have made anything not work, everything's rated, so not sure what it is.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e175d6 No.12847

>>12753

Thank you for this report. This is an odd error–the problem here is that the options object is not being serialised into the db correctly. This is the kind of error I see when someone has like a 2GB subscription, and SQLite falls over trying to make a buffer big enough for it. The options object there is a small thing, automatically saved at various points. Perhaps it was being nullified in some way, or some invalid data was being added to it. I can't think immediately why it would be affected by a rating set event.

Can you say more about this rating conversion? You had a 2-star rating service as set up as under manage services, and then when you did a giant ctrl+a->f4->set rating->ok, it took a moment to write that and then popped up this error right at that time?

Had you set any other options recently, either in file->options or via one of the 'cog' menu buttons? Anything related to default sort based on that new(?) rating service?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



YouTube embed. Click thumbnail to play.

ad7f64 No.12651 [Open thread]

ITT we post about our duplicates, tagging, and so on - our stats, goals, landmarks, and so on.

Never give up anons, that tidy catalog of content is waiting for you!

10 posts and 1 image reply omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ad7f64 No.12724

>>12717

Ah yes, I also enjoy having a lot of useful files myself and having other people do the work, indeed.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

932e95 No.12758

anyone else cherry pick images then tag them individually as you store them in hydrus?

seems like if you mass download images you havent seen yet, you might have a bunch of images you dont care about

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f88ef0 No.12760

>>12758

I do both.

>you might have a bunch of images you dont care about

That's what the archive/delete filter is for.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a5146b No.12765

I have 1TB of inbox to go through while 800GB are now properly archived, as in, I have personally checked them and stuff.

I've stopped working on dupes till the current dupe rework is done, since alternate not copying tags nukes my workflow, sitting at 80k dupes.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ad7f64 No.12819

OP here, down under 60k dupes finally.

>>12758

I save those to Downloads with the most important tags in their title for later import.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: ebcd0b5b00cd90d⋯.gif (1.02 MB,1280x720,16:9,ebcd0b5b00cd90d8f8fb165169….gif)

8080a4 No.12749 [Open thread]

I had a great week. The new 'false positive' and 'alternates' duplicates db storage design and migration worked out well, although it did take most of my time. Other than that, I did a variety of little new options and bug fixes. The annoying bug where a handful of thumbnails sometimes stop fading in seems to be finally fixed!

The release should be as normal tomorrow.

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

beb060 No.12751

I like how op starts off with how well he's been doing

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b91329 No.12752

>>12751

I love hydrus man and hope he is doing well in all aspects of his life.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8080a4 No.12754

>>12752

>>12751

I am overall healthy and in a stable money situation. I hope you are as well. I get blackpilled sometimes, but then I remember my true troubles are all internal and that I have a lot to be grateful for compared to my ancestors. I still want to keep pushing on Hydrus every week for the foreseeable future. I love imageboards and all other Anons, including you–the cheeky fun we get up to keeps my soul going.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec748c No.12762

>>12754

Based and Bloomerpilled

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: 48f7d79317b8899⋯.gif (381.04 KB,350x188,175:94,48f7d79317b88998c4e3dc1218….gif)

88a410 No.12665 [Open thread]

I had a great week. I finished the first version of the new file maintenance system I had planned, cleaned up the duplicate filter workflow a little more, and fixed a bunch of bugs.

The release should be as normal tomorrow.

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.


YouTube embed. Click thumbnail to play.

6c1b1a No.12583 [Open thread]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v352/Hydrus.Network.352.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v352/Hydrus.Network.352.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v352/Hydrus.Network.352.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v352/Hydrus.Network.352.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v352.tar.gz

I had a good week. .ico files are now supported, 'collect by' status is remembered in gui sessions, and I fixed a bunch of bugs.

duplicate overhaul plans

I started the duplicate overhaul work this week with some planning and experimentation with existing data. My original thought here had been to exactly replicate existing functionality just with a more efficient database schema, but having gone through the various edge-case insertion and merge operations, I believe the current system is overcomplicated for what we are actually using it for.

Most of all, the current system tries to form a chain of 'better/worse' comparisons so all dupePost too long. Click here to view the full text.

19 posts omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6c1b1a No.12637

>>12630

I assume you have already done archive/delete filter on these files, so they are all in the archive? Otherwise I would suggest using inbox and archive to track files you have processed or not. You can send files back to the inbox for another round of archive/delete processing with Shift+F7, btw.

I have a couple of like/dislike ratings on my client that I use as pseudo inboxes. One is called 'read later' that I apply to anything I am archive/delete filtering that is like a big thread screencap or a 15 min video on some bullshit that I don't want to deal with in a quick filter. I enjoy like/dislike ratings since I can just click to quick apply and have some shortcuts set up for this as well. Maybe you could make a new like/dislike rating called 'videos to process' or something with some shortcuts to like/nullify to help you keep track of all this?

Mapping things to tabs is tricky. I'll be doing some Client API tab-access work soon, so I'll be thinking more closely about this in a couple weeks. The tricky part is mapping the action to the tab–'send file right one tab' is fairly easy to manage, and has a simple yes/no error state, but 'send file to page named "sexy elves"' is just that step more difficult to track and action and fail with (e.g. there could be two pages with that name). I like the idea of setting a current 'destination' tab in the current session that you could then action. I'll think about that. I've thought maybe having a shortcut or action for 'suck up these thumbs into the mouse' and then 'spit those thumbs back in here', like in cut and paste terms.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

224b41 No.12648

>>12636

Oh, well that's good to hear. I don't have any problem with the PTR, I just thought you did since people talk a lot about tags in it and I remember there was a lot of complaining a year or two ago about wrong tags. I just use it as an aid to my own tags to sort stuff, so I'm happy if you're happy.

I do want to try running my own PTR and file server with Hydrus if I ever get the money and time, so if I do I'll give feedback.

As for that workflow idea, I'll give it a try with some files once I finish with my duplicates.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

95a331 No.12654

>>12633

I suggested 30 minutes as the obviously safe option, where no one could possibly watch a preview for that long, but the majority of AFK sessions would be longer than that. I wouldn't expect to see much reduction in sessions pruned by going with 10 minutes, but who the hell is legitimately watching the preview for 12 minutes and wants that time recorded?

Honestly, I'm not convinced this needs to be a setting. 10 minutes is way more than almost anyone will ever watch in a thumb. 60 minutes would cut out the vast majority of AFK sessions. It's such a broad range to work in that 30 minutes would do the right thing in 95%+ of cases (and I'm being cautious with my numbers), so why add more clutter to the settings for something almost no one would need to change?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

496743 No.12656

File: 1ecb47d56ca76c4⋯.jpg (8.4 KB,815x24,815:24,media viewer.jpg)

>>12633

>>12654

Could any potential solution for this be extended to the media viewer? This seems like the same problem of getting distracted during, say, an archive/delete session, and leaving one image open (but minimized) for a few hours.

I've had this, but also a similar problem with the media viewer (pic related). I basically had a bunch of 'alternate/duplicate' files, and I was flipping between all of them hundreds of times trying to decide which to keep, tags etc.

Is it possible to detect if the client doesn't have focus, and stop tracking immediately? I feel like that would solve 90% of issues right off the bat. Otherwise, tying it in with the client's inbuilt 'idle' state could also work well (and is already user configurable in options -> maintenance and processing).

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6c1b1a No.12845

>>12656

>>12654

Thanks lads, only catching up now. This isn't in yet, but I'll keep this in mind as I do this system.

I'll do min/max for media viewer as well. Options are easy to add, and I know someone will want to say 'no minimum time' or whatever, so I'll throw them in for anyone who is interested. 5s/10m sounds like an ok default min/max.

I think I can catch a focus lost event, although some of that stuff is a little unreliable. I'll play around with it and we'll try iterating on this a bit.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: 5305f71010f2bb7⋯.png (905 B,184x184,1:1,1455677.png)

bcbe46 No.9960 [Open thread]

It looks like the Bibanon wants to use Hydrus as a server, and they really want to slap a webUI on top of it… but Hydrus Dev don't have it on priority list.

They have though about using Szurubooru for the WebUI but rr- don't want to update it. See: https://github.com/rr-/szurubooru and rr-@sakuya.pl

Maybe it is time to draft up a sample API set and WebUI for them.

From antonizoon 2 Sep 2018

Hey there, it's been a very long while, but it's good to see that you've
kept working on Hydrus ever since.

One of the issues with hydrus for me in the end, which is why we never
ended up using it, is because of the lack of a webui. As such I ended up
utilizing Szurubooru, as can be seen here: https://eikonos.bibanon.org

However, what I can say is that the dev team at the Library of Congress
was very interested in adding tag support to their image tag system
which is below. It's currently utilized with the aim of crowdsourcing
transcriptions of images, but it lacks tagging or search like we would
have in a booru. Luckily, we already have the hydrus protocol and
szurubooru right? They aim to launch in October.

I find this to be a perfect chance to integrate Hydrus with a webui as
well as obtain some useful features from their existing system as well.
What do you think?

https://github.com/LibraryOfCongress/concordia

Possible CSS frameworks

https://tutorialzine.com/2018/05/10-lightweight-css-frameworks-you-should-know-about

https://dzone.com/articles/top-10-lightweight-css-frameworks-for-building-fas

Post too long. Click here to view the full text.
____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bcbe46 No.9962

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

793d60 No.9963

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

128250 No.10008

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

103a87 No.10216

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

dd3a7e No.12655

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: 3f8a8d22364b1de⋯.jpg (111.13 KB,400x400,1:1,1394958218797.jpg)

8ea9e7 No.9327 [Open thread][Last50 Posts]

Here you can ask questions so that the board is not clogged with small threads.

>>6021 has reached its bump limit, so I made a new thread.

746 posts and 130 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d8e00b No.12629

>>12610

Namespaces are for meta information like the artist, name of the character, series the character is from, whether the image is lewd or not, etc. Tags are for descriptions of what the image contains. Short hair is a description of a character in the picture. With very few exceptions a namespace should only exist once per image. A notable exception is when a picture shows a character who exists in multiple series (eg. Rage of Bahamut / Granblue Fantasy or if you're someone who tags seasons as separate series) or characters from multiple series (eg. crossovers).

You're doing it wrong because you end up with garbage like the following on a single image featuring two characters.

>hair: long hair

>hair: short hair

>>12626

You're doing it better in terms of naming, but you're still doing it wrong.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

47a743 No.12632

>>12629

sounds like a very subjective description of namespaces to me.

not sure what the difference to you is between having two namespaced tags "hair:short","hair:long" and having two regular tags "short hair","long hair". Either way you have two tags.

In fact, using namespaces liberally for tags that are important to you (e.g. if you like to add lots of hair descriptors) is useful because the tags will be grouped together when displayed. When checking which such tags are already on a file, it is much easier to find them if they are grouped together.

Maybe you have almost entirely files which contain only one character? I know I don't

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5d49b6 No.12638

>>12593

>>12594

Yeah, try tag siblings:

https://hydrusnetwork.github.io/hydrus/help/advanced_siblings.html

The system is very imperfect, but it generally works. More time will be put into this this year, including better clientside preference management for tags synced over the PTR.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5d49b6 No.12639

>>12596

There is no solution for this yet. I expect a future version of the tag siblings system to allow some sort of regex-based replacement (think a global rule for 'replace all "\s" with "_"') that will allow this.

Since desiring underscores to either go away or be mandated is common, I am likely to hardcode a solution to manage this earlier than a generalised regex solution, but the tag siblings system just isn't clever enough to handle it yet.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5d49b6 No.12640

>>12629

>>12632

I have yet to come down to a good description for what namespaces are myself. Here is my current first draft:

Namespaces are good when:

They are a higher category (evangelion is a series)

They are useful to highlight (characters are important)

They are useful to search for (creators are often searched for, and 'creator:*anything*' has value)

They are useful to group together (multiple creators is useful to know, when listed next to each other)

I've been approving 'clothing:' siblings on the PTR for a little while now and I overall like it. I'm mixed on some others though. My ideal solution here is to extend tag siblings to allow clientside preferences and then for you to say "If a group of siblings includes one with 'hair:' namespace, prefer that". I have found that it is easy for users to objectively agree that 'hair:long hair' and 'long hair' have the same semantic meaning, but the big subjective disagreement is over which is better.

An actually bad namespace, imo, is one that breaks the first rule above. 'male:erection' is an artifact of how some gallery sites do various female/male focus on tags, but I would rather 'male erection' (which could nicely have parents 'erection' and 'gender:male'). 'erection' is not a 'male'. Also 'erection:male'.

'general:coffee cup' and 'object:pencil' are technically correct but overspecific for most users, but again I think the specificity of namespaces is highly subjective, so the true answer is to let users define what namespaces tags could have and let them then customise what they prefer to show.

If you were going for 'hairlength:short', I'd probably try to aim for hairlength:short hair for similar reasons, or go to the higher category of 'hair:short hair'. 'shoPost too long. Click here to view the full text.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
Post last edited at



File: 967135a2ac38b54⋯.gif (1.21 MB,450x247,450:247,967135a2ac38b54d01456507da….gif)

cf1c43 No.12577 [Open thread]

I had a good week. I added .ico file import support, got 'collect by' settings saving through gui sessions, and fixed a whole bunch of bugs (including the annoying media-window-hover-frame-over-manage-tags issue).

The release should be as normal tomorrow.

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.


YouTube embed. Click thumbnail to play.

768770 No.12507 [Open thread]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v351/Hydrus.Network.351.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v351/Hydrus.Network.351.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v351/Hydrus.Network.351.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v351/Hydrus.Network.351.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v351.tar.gz

I had an ok week. I sped up several systems and added a new processing panel to the duplicate filter.

The 'next big job' poll is finished! I will next be focusing on overhauling the duplicate filter's db structure (including sketching out support for file 'alternates') and further improving the ui-side workflow.

duplicate filter

Seeing that the duplicate filter work was popular in the poll, I was happy to put a bit more time into it this week.

Most importantly, the duplicate filter now has a new always-on-top panel to make reviewing differences and making decisions easier. EsPost too long. Click here to view the full text.

9 posts and 3 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

768770 No.12552

>>12510

Yeah, I agree on hotkeys. I'll stick them on the tooltips, I think, including for the navigation buttons for skip and go back etc… I'm not super keen on hardcoding this as I'd rather move forward on a system for buttons to understand hotkeys and present this info dynamically across the program. I'll think about what is sensible for now.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

768770 No.12553

>>12517

>>12518

Hey, the program is on Python 3 now. If you are coming from before 335, please check that release post: >>11238

I think that is probably your dll problem here. Read the post for your situation, but basically you'll be doing a 'clean' install to wipe out the old conflicting Python 2 dlls before reinstalling a new version. Let me know if you run into trouble, and make-a/feel-good-about-your backup before you do it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

768770 No.12554

>>12520

Yeah, I don't like the sort of the sort. I've found it tricky to figure out a concise way of saying some of this stuff, like age vs time imported.

Yeah, perhaps something like 'sort by dimensions: width' would help here to group them better. I'll make a job and think about this a bit.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

768770 No.12555

>>12525

Thank you for this report! Sorry, I fucked up the new focus logic when adding the always-on-top ability. I'll 99.7% have this fixed for 352.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

01e129 No.12564

>>12554

personally, just sorting them between file facts and database entries would be enough.

database would be things like tags, rateings, so on so forth that is really only applicable in program

file facts would be things like pixel amount, file size, hight, width so on.

this would at least clearly separate the two.

>>12551

now that you say putting it on another monitor, a floating window would be perfect.

Currently I have a 4k monitor, I have the files open as the right half of the monitor, a magnifying glass as the second monitor, and if this stuff was floating, I could easily possession it wherever the hell I please and not care at all. not sure how easy that would be but that would be a solution that works as a fits most use cases.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: 1897cd131f5569a⋯.webm (2.52 MB,480x320,3:2,1897cd131f5569a99a530c5f0….webm)

3ccbc4 No.12358 [Open thread][Last50 Posts]

Here is the poll for the next big job:

https://www.poll-maker.com/poll2331269x9ae447d5-67

You can vote for multiple items. I expect to start work on the top-voted item at roughly the time of the v351 release post, in two weeks.

Please feel free to discuss and ask about items on the poll in this thread.

50 posts and 5 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3ccbc4 No.12505

VOTE DONE

I am just putting my 351 release post together now. 'Improve duplicate db storage and filter workflow (need this first before alternate files support)' stayed on top for almost all of the past two weeks and is still top now, so that is the winner.

This item will involve overhauling the duplicate db structure to be more efficiently group-based rather than pair-based and will continue the recent ui-side workflow improvements. If there is time, I might even make a skeleton for optional auto-decision making on things like 'if one file is jpg, the other png, and they have the exact same pixels, set the jpg as better'. I will keep file 'alternate' support in mind throughout.

Thank you for voting everyone. I appreciate your input and the surrounding discussions over what to do with hydrus.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e11c7b No.12506

>>12505

I have 2 suggestions for the duplicate auto filter

1a) there was a thing a number of years back that would take a png and a jpeg 95% and show you a negative that would be the difference in quality, I know this can also be done with music to show what's lost in compression. would there be a way to pair files off in something like this

1b) Would it be possible to pair files regardless of size or ratio differences off in this? this honestly may completely fuck the results but I have run into cases where the file size is half the normal one in ratio and its fairly obvious whats better, and having an auto duplicate sorter to at least give it a once over for a possible determination

1c) If you do something like this, I would assume it has to be done at or around run time as in no database of files, would it be possible to show the negative so it may be easier to see where my eyes should focus

1d) because I personally subscribe to a 'some loss is acceptable depending on the circumstances' way of thinking, would it be possible to have a better worse check determine one image was better but go with the lower quality one because it fell under a certain % difference?

1e) because of 4chan and a few of the people there I know another thing they did to piss space away, someone posted an image, and for the entire thread all he did was reopen and resave the same image as a jpeg over and over again, slowly corrupting it, I know that if I had an image set like this, I may have gen 1 (good) get replaced with gen 2 (1%wrose) and maybe that one is accepted, but then it goes down from gen 2 ot 3 ect to get 150 where its a fucking wreck. I know its not likely, but a fail safe in automation against that would be greatly appreciated where if it sees 3 images deceptively close in quality to each other it triggers manual review immediately.

2a) Due to the /trash/ scalie thread spam, I was made aware of a method to troll who use an auto dup filter, they have the base image, then they make a png, corrupt the fuck out of it, and that bloats the png, which in a filter Post too long. Click here to view the full text.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0c161a No.12511

>>12506

Check the big list of repos in >>12295 especially https://github.com/andrewekhalel/sewar (it will come in handy)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3ccbc4 No.12542

>>12511

>>12506

1a)I am chiefly limited in how much time I have. I'd love to do more complicated autodecision workflows and backup ui, and I think that sort of thing would help a lot, but I don't have time in this rewrite. I also read up on some jpeg quality estimation here https://www.politesi.polimi.it/bitstream/10589/132721/1/2017_04_Chen.pdf , but it is too complicated for me to implement in the time I have. I would also like to focus on the db side more in this cycle.

1b)I am moving to a simpler comparison system in this rewrite. You'll always be seeing files compared to the 'best' of a group once it is done, which should exaggerate filesize and resolution differences. I am squeamish about autodecision on filesize or resolution alone as there are plenty of stupid bloated pngs of jpegs out there, but I think that bias could be part of a larger system that takes multiple variables to auto-decide.

1c)Sorry, I just don't have time to write clever ui like this atm.

1d)Yeah, I am afraid of the edge cases here. My thrust will always be default to off and lots of user customisation. I'll prep any auto-system with rules like "if exact same pixels and one is jpeg one is png, the jpeg is better", but I'd like for you eventually to be able to write your own rules for what you want out of it.

1e)Yeah, single pixel edits are a problem here. In a future iteration of any autodecision system that took multiple rules to make decisions, I think a blanket "if one pixel different, the older file is better" could be the ticket. There's also issues with file metadata being stripped or altered by CDNs.

2a)Yeah, that's the difficult stuff. Any dupe filter can't be simple, or any simple rules should be able to gauge certainty and pass the decision up to human eyes when something smells fishy. This claPost too long. Click here to view the full text.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e11c7b No.12563

>>12542

yea all of 1 more or less required the image difference comparison to be a thing.

as for 1e its not even single pixels, its resaveing the exact same image over and over again as jpeg slowly corrupting it over the course of 100-150 images. that's where I see a slow better worse auto filter jumping from a good image and quickly ending with the most corrupted one is no fail safe was implemented.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



YouTube embed. Click thumbnail to play.

d3155b No.12458 [Open thread]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v350/Hydrus.Network.350.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v350/Hydrus.Network.350.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v350/Hydrus.Network.350.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v350/Hydrus.Network.350.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v350.tar.gz

I had an ok week. Some IRL things cut into my hydrus time, but I got some good work done. Some bugs in the new duplicate search system are fixed, and I improved advanced file delete and export handling.

The poll for the next 'big job' is up here: >>12358

duplicate filter

The search addition to the duplicate filter went fairly well, but there were a couple of significant bugs. The 'ghost pair' issue–where a queue would sometimes have a final pair that would never display and lead to high CPU until the filter was closed–is fixed, and saPost too long. Click here to view the full text.

8 posts and 4 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8fad24 No.12498

>>12458

>and deleting files without leaving a delete 'record'

Right when I was going to ask about that. Thanks.

I think a better term for this action is 'forget'.

How does deletion reason/forget/… work with "archive/delete filter"?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

93a027 No.12499

>>12481

With delete, I don't mean delete with reason, what I mean is in current hydrus, there are various hotkey presets depending on what you are doing

So when you get into a delete menu, its hotkey preset takes over, and like I said in my case it would be 1-8, pressing the number could select one of the reasons.

Like I said before, my use case for this is still quite a bit down the road because mass culling is still easier and will be for several months. no idea if this is possible in the current shortcut method but if it is, it would be something to do to fill time if its easy enough.

as for manga i'm leaning on keep the 'money shot' pages

If I read the manga, it will make me laugh/make me want to re read it, if I didn't, I still may get something out of it, if its porn, well… that one obvious.

as for conversion…

personally with that 112mb attrosity, I got it down to 1.5mb after fucking with settings in handbrake for a while

For what to do in the future… that a a bit harder

first you would need a 'lossless' conversion, as some animated gifs are bloated to hell and back, but they have very clean lines, and a conversion even as lossless may end up at a lower file size then non converted.

second you would need a near placebo preset, not quite there but still high quality

third would be a high quality compression, a good catch all for everything,

forth would be another relatively high quality preset, lower but still a catch all

and below that some fuck this presets, I have a few videos where they made a music video of sorts, but it's over 200mb and the only thing that's worth anything from it is the audio, but the visuals are a nice tPost too long. Click here to view the full text.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d3155b No.12539

>>12488

I dove into the code here as part of my research into the dupe db overhaul yesterday and discovered some problems in my original 1.0. I feel less confident about it as I did for certain edge cases where 4+ files want to figure out what worst->best order they should go in. What happens to tags and whether the 'worse' of a pair is deleted is up to the duplicate merge options (check the cog icon on the dupe filter to edit these). I believe the defaults are to delete the worse of a pair, but if for some reason yours is not set this way, I think this will help your processing. Clearing 'bad' files out of the local file domain clears out some unusual AB, BC, AC pair comparisons the current system wants to do.

My plans for the new storage system will do away with the over-autistic comparisons here and push for a simpler model with groups of dupe files with a defined 'best quality' King.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d3155b No.12540

>>12498

The archive/delete filter still uses the old delete system atm, just sending files to trash with a default reason. I would have added the new delete dialog to it, and likely will in future, but the batched way archive/delete filter stores up and commits its actions make it more complicated to integrate, so I put it off.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d3155b No.12541

>>12499

I will not hardcode number shortcuts for certain controls for now. I think macros to tab&space navigate the dialog are your best bet atm.

That's interesting about SFM, I didn't know about the image-based rendering. I had heard the software tends to crash a lot, especially as artists are often working on older laptops etc… Sounds like a nightmare. I assume there is no easy one-click open source workflow–that for instance just neatly eats up their pngs and throws them at ffmpeg–for these users, unless Handbrake can do something like that?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: ab048d7e6dac8b4⋯.gif (1.67 MB,500x377,500:377,ab048d7e6dac8b461b468968d6….gif)

d9ac2c No.12503 [Open thread]

I had an ok week. I cleaned a whole bunch of code, sped up image importing and the new duplicate operations, and wrote a new always-on-top action panel for the duplicate filter that makes for faster processing.

The release should be as normal tomorrow.

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5a8d89 No.12504

File: f14c40156e14c9f⋯.jpg (11.47 KB,183x275,183:275,images.jpg)

Dude the PTR account is almost at half of that 256GB already

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d9ac2c No.12509

>>12504

Yeah, we might be fucked. 560M mappings already. I'll probably reserve the next 'big job' after this dupe work to trying to sort it out.

A more temporary fix I may apply is a daily 10GB limit, which will stop uploaders from sending so much and throttle the incoming tags more smoothly over the month.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7fcc86 No.12513

>>12509

*sigh

If only we have a more web-friendly way of dealing with tags… Not saying social media works, but it is the first thing that came to my mind e.g. voting, crowd moderation etc. using psuedonym? Sorry dev

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d9ac2c No.12526

>>12513

No worries. Having too much is a nice problem to have, and I have many ideas on how to mitigate our problems. Clearing out the (1) and done tags like 'title:tuhentu honeuhoen tuhoe nthn', or giving users options on how much of that stuff they want to sync with, will be a likely next step. There is plenty more we can do here, and I am keen to keep on pushing. I am committed to keeping person-to-person interactions Anonymous in my code–boorus already do username-based curation much better than I have time to do–but I know I can give users more power to share and sync just what is useful to them.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: d7e685360192f2a⋯.png (20.52 KB,722x449,722:449,api-image-for-blog.png)

1dd852 No.11626 [Open thread]

ITT: We propose new features that can be solved by using the API, and recommend new API commends for it

5 posts omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1dd852 No.12269

bumping this so that we can discover more uses of API better.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1a745b No.12467


The Kinoshita Collection Danbooru { "id":3382448, "source":"https://i.pximg.net/img-original/img/2019/01/13/00/12/24/72630169_p7.jpg", "md5":"d5a650a70fca03ff391b50fc255dcb26", "rating":"s", "image_width":550, "image_height":770, "tag_string":"1girl :o ahoge bangs blue_eyes blue_hair blue_ribbon blue_skirt braid breasts collared_shirt eyebrows_visible_through_hair hair_ribbon juliet_sleeves kazutake_hazano long_sleeves looking_at_viewer parted_lips puffy_sleeves ribbon shiro_seijo_to_kuro_bokushi shirt short_hair sidelocks simple_background skirt solo underbust upper_body white_background white_shirt", "file_ext":"jpg", "has_large":false, "tag_string_general":"1girl :o ahoge bangs blue_eyes blue_hair blue_ribbon blue_skirt braid breasts collared_shirt eyebrows_visible_through_hair hair_ribbon juliet_sleeves long_sleeves looking_at_viewer parted_lips puffy_sleeves ribbon shirt short_hair sidelocks simple_background skirt solo underbust upper_body white_background white_shirt", "tag_string_character":"", "tag_string_copyright":"shiro_seijo_to_kuro_bokushi", "tag_string_artist":"kazutake_hazano", "file_url":"https://danbooru.donmai.us/data/d5a650a70fca03ff391b50fc255dcb26.jpg", "large_file_url":"https://danbooru.donmai.us/data/d5a650a70fca03ff391b50fc255dcb26.jpg", "preview_file_url":"https://raikou4.donmai.us/preview/d5/a6/d5a650a70fca03ff391b50fc255dcb26.jpg" } Moebooru (JPEG is the file of Danbooru) { "id":276945, "tags":"ama_mitsuki brown_eyes brown_hair gloves hat original panties pantyhose skirt underwear", "source":"
Post too long. Click here to view the full text.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1a745b No.12468


The Bisimplex collection Danbooru { "id":3382448, "score":1, "source":"https://i.pximg.net/img-original/img/2019/01/13/00/12/24/72630169_p7.jpg", "md5":"d5a650a70fca03ff391b50fc255dcb26", "rating":"s", "image_width":550, "image_height":770, "tag_string":"1girl :o ahoge bangs blue_eyes blue_hair blue_ribbon blue_skirt braid breasts collared_shirt eyebrows_visible_through_hair hair_ribbon juliet_sleeves kazutake_hazano long_sleeves looking_at_viewer parted_lips puffy_sleeves ribbon shiro_seijo_to_kuro_bokushi shirt short_hair sidelocks simple_background skirt solo underbust upper_body white_background white_shirt", "file_ext":"jpg", "last_noted_at":null, "parent_id":null, "has_children":false, "has_large":false, "is_favorited":false, "tag_string_general":"1girl :o ahoge bangs blue_eyes blue_hair blue_ribbon blue_skirt braid breasts collared_shirt eyebrows_visible_through_hair hair_ribbon juliet_sleeves long_sleeves looking_at_viewer parted_lips puffy_sleeves ribbon shirt short_hair sidelocks simple_background skirt solo underbust upper_body white_background white_shirt", "tag_string_character":"", "tag_string_copyright":"shiro_seijo_to_kuro_bokushi", "tag_string_artist":"kazutake_hazano", "tag_string_meta":"", "file_url":"https://danbooru.donmai.us/data/d5a650a70fca03ff391b50fc255dcb26.jpg", "large_file_url":"https://danbooru.donmai.us/data/d5a650a70fca03ff391b50fc255dcb26.jpg", "preview_file_url":"https://raikou4.donmai.us/preview/d5/a6/d5a650a70fca03ff391b50fc255dcb26.jpg" } Moebooru { "id":276945, "tags":"ama_mitsuki brown_eyes brown_hair gloves hat original panties pantyhose skirt u
Post too long. Click here to view the full text.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1a745b No.12469


The Andy iOS collection

Danbooru
created_at: Date string in the format "yyyy-MM-dd HH:mm:ss"
rating: possible values - e, s, u, q
md5: string
height: Int
width: Int
preview_url: URL
large_file_url: URL
sample_height: Int sample_width: Int
tags: STring with tags, each tag separated with spaces
id: Int source: URL, String, or nothing

Moebooru
created_at: (This should return the number of seconds since the epoch and be a number)
rating: possible values - e, s, u, q
md5: String
height: Int
width: Int
file_url: You should try to return the full URL, including the protocol and host name (the app tries to build it when they are missing)
preview_url: Same as above
sample_height: Int
sample_url: string
tags: String with tags, all separated by spaces.
id: Int
source: Where the image comes from. Can be a url, string, or missing

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

47ce0f No.12474

>>12467

>>12468

>>12469

Really? https://pastebin.com/vzQTRFaa since ghostbin is shutting down

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



3d3c95 No.12454 [Open thread]

Hey,

awesome tool, really enjoying it.

I did encounter an error that came up three times when fetching files from Sankaku (out of probably around 10k+ files imported so far, I've only just started using hydrus last week):

local variable 'photoshop' referenced before assignment… (Copy note to see full error)

Traceback (most recent call last):

File "include\ClientImportFileSeeds.py", line 1178, in WorkOnURL

self.DownloadAndImportRawFile( file_url, file_import_options, network_job_factory, network_job_presentation_context_factory, status_hook, override_bandwidth = True )

File "include\ClientImportFileSeeds.py", line 571, in DownloadAndImportRawFile

self.Import( temp_path, file_import_options )

File "include\ClientImportFileSeeds.py", line 790, in Import

( status, hash, note ) = HG.client_controller.client_files_manager.ImportFile( file_import_job )

File "include\ClientCaches.py", line 1144, in ImportFile

file_import_job.GenerateInfo()

File "include\ClientImportFileSeeds.py", line 283, in GenerateInfo

self._file_info = HydrusFileHandling.GetFileInfo( self._temp_path, mime )

File "include\HydrusFileHandling.py", line 233, in GetFileInfo

( ( width, height ), duration, num_frames ) = HydrusImageHandling.GetImageProperties( path, mime )

File "include\HydrusImageHandling.py", line 241, in GetImageProperties

( ( width, height ), num_frames ) = GetResolutionAndNumFrames( path, mime )

File "include\HydrusImageHandling.py", line 274, in GetResolutionAndNumFrames

pil_image = GeneratePILImage( path )

File "include\HydrusImageHandling.py", line 92, in GeneratePILImage

pil_image = PILImage.open( fp )

File "site-packages\PIL\Image.py", line 2690Post too long. Click here to view the full text.

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

251eea No.12457

Thank you for this report. This actually isn't my code failing here, but the image library I use, Pillow. Just guessing, I suspect these files are slightly malformed (broken) and Pillow is having trouble dealing with them.

As it happens, I plan to do some cleanup work next week to have a different library, OpenCV, do the initial metadata parsing work here and improve reliability of metadata parsing overall. I will test these URLs once that is done and see if they work. Thank you for the examples!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: 19493ce10e8cf1d⋯.gif (1.56 MB,500x360,25:18,19493ce10e8cf1d36e2cbbcc6a….gif)

639261 No.12444 [Open thread]

I had an ok week. I fixed some bugs from the recent duplicate filter changes, wrote a new advanced file delete dialog, and added new timing controls to export folders.

The release may be late tomorrow.

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.


Delete Post [ ]
[]
[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]
| Catalog | Nerve Center | Random
[ / / / / / / / / / / / / / ] [ dir / random / 93 / biohzrd / hkacade / hkpnd / tct / utd / uy / yebalnia ]