[ / / / / / / / / / ] [ dir / cute / egy / fur / kind / kpop / miku / waifuist / wooo ]

/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Catalog

Name
Email
Subject
Comment *
File
* = required field[▶ Show post options & limits]
Confused? See the FAQ.
Embed
(replaces files and can be used instead)
Options
Password (For file and post deletion.)

Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf
Max filesize is 12 MB.
Max image dimensions are 10000 x 10000.
You may upload 5 per post.

New user? Start here ---> http://hydrusnetwork.github.io/hydrus/

Current to-do list has: 714 items

Current big job: finishing off duplicate search/filtering workflow

YouTube embed. Click thumbnail to play.

e9223c No.4633

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v234/Hydrus.Network.234.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v234/Hydrus.Network.234.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v234/Hydrus.Network.234.-.OS.X.-.App.dmg

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v234/Hydrus.Network.234.-.OS.X.-.Extract.only.tar.gz

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v234/Hydrus.Network.234.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v234.tar.gz

I had a great week. I fixed a lot of bugs, moved faster dupe search forward, and streamlined how the manage tags dialog works.

🎉 🎉 Also, this week the public tag repository hit 100 million tag mappings! 🎉 🎉

managing many tags at once is easier

The manage tags dialog now groups multiple tag input into one action. So, if you select four existing tags and double-click that selection, it will only ever ask for one petition reason for all of them. If the many-tag event could mean several actions, like a pend or a petition, it will only ask what you want to do once, and then do that to all for which that action is appropriate. There is a slightly new button choice dialog that has some useful dynamic tooltips on the buttons.

The workflow here has been janked for a while, but I hope this makes things a little easier. Let me know if anything doesn't work.

I would also like to add 'common reason' helper buttons to the petition reason entry dialog, something like 'typo/malformed'/'not in media'/'already exists as namespace', to make it easier to enter petitions user-side and process petitions on janitor-side (as petitions with the exact same reason are grouped together for mass-actioning).

faster dupe search progress

When you update, your existing 'similar image' metadata will be upgraded to a better format. I thought the conversion might take a few minutes, but it only took about five seconds on my 350k file laptop, so you likely don't have to worry about it.

Otherwise, I fleshed out the framework I started last week, and I remain pleased with my progress. If everything continues to go well, I think the first prototype of the new algorithm should be ready for next week. It'll swap straight into the current 'system:similar to' predicate. Once that 'tool' of dupe search works, I can start work on some new dupe search, presentation, and filtering.

I'll try to be in the discord on saturdays

A user of hydrus maintains a discord channel here: https://discord.gg/vy8CUB4 .

I'm not typically an IM/chat person, so I've stuck to the board, tumblr, and email, but after some discussion with another user, I decided to give the discord a go, and it went well. I will now try to spend some time in it every week for anyone who would like fast feedback on anything. I will try to be in the discord on Saturday afternoons US central time, which is the beginning of my hydrus work week when I typically jump between small jobs that don't require a lot of concentration. If you want to report a bug or just say hello, please do! There are a number of experienced users who hang out there more often–if you would like to discuss hydrus with other community members in a chat setting, I'm sure they would love to have you.

I've added the above discord link to my help pages and the help->links menu inside the client.

full list

- the action choice workflow in the manage tags dialog now merges decisions for multiple tag entry events

- the button dialog that pops up on the new merge manage tags workflow has button tooltips to better describe the proposed action's tag and file combinations

- the manage tags dialog only requests one petition reason for an above merged multitag petition event

- perceptual hashes are now stored in the cache db (moved from the preferably leaner main db)

- the database now supports multiple perceptual hashes per file

- fleshed out the perceptual hash vptree generation and maintenance code

- added option for main gui title to options->gui–it even updates live

- added unicode path support when importing serialised pngs

- added unicode path support when exporting serialised pngs

- exporting a serialised png now reports success via the export button, which will temporarily relabel itself

- added discord links to help files and help->links menu

- when a GET network connection fails during the read phase due to an unexpected timeout, the request will be reattempted a couple of times, like failed connection initialisations currently are

- escape key now closes scrolledpanelframes (review services or import status frames)

- manage tags dialog and frame now closes due to escape key correctly

- fixed a size calculation bug that was not initially drawing scrollbars on manage options and any other listbook-containing scrolling panel when the screen is too small to show the whole dialog

- undo menu now works on the new menu system

- cleaned up some bad gui-thread interaction in the file import dialog

- fixed file->restart in the built release, including when the install path includes a space

- the trash service no longer records which files it has physically deleted, as this information is not used and is redundant compared to the existing local files' deleted record (existing records will be deleted on update)

- the subscription daemon will wait 90 seconds after boot before triggering–quitting the client before then will result in subs not being checked

next week

I'd like to hammer out this faster dupe search if I can. There's a bunch of parsing script stuff to do as well. And the Linux release still doesn't restart right.

The 100 million mappings news is great, but it also means client.mappings.db is getting pretty big again (about 4.2GB now). The repository update protocol is getting a bit creaky as well, so in the next six months I may have to insert a few weeks of db slimming (I think I can cut it another 30% at least) and a complete hydrus network update.

beb4fd No.4634

Awesome! Excited for the dupe improvements.

Discord is… an odd choice. Traditionally dev projects cluster around irc channels for chat. You'd be better off @ irc.freenode.net


beb4fd No.4635

>>4634

I have, in fact, already reserved the #hydrus channel on freenode, in the off chance that you would ever want to populate it.

IMO if you're going to build a chat community it should be on irc- discord is one of hundreds of proprietary 'flavor of the year' type services. It will be out of fashion next year. IRC is unchanging, forever, and thus reliable.


beb4fd No.4636

http://freenode.net/philosophy

freenode is, if you didn't know, a network dedicated to free and open source projects and communities.

The irc protocol is well known, well established, and mature.

Discord is… random proprietary voice/chat service for video games. It's not a good fit.


bc9f5a No.4637

Unfortunately, there's a pretty nasty bug in this last release.

I'm using Arch Linux, and when opening the tag manager from the media viewer, it's not possible to type anything in the box and thus it's impossible to tag anything. I can tag from the normal interface however, but it's much slower.


98d7f4 No.4638

>>4637

Strange, I'm also on Arch and am not encountering that problem at all, media viewer or no. Maybe I'm just misunderstanding where you're trying to input tags, though?

For what it's worth, I'd also like to echo the other users preferences in IRC over Discord.


bc9f5a No.4639

>>4638

Click a thumbnail -> press Enter -> mediaviewer opens -> press F3 on it > tag manager shows up, but I can't insert anything in the tag box by using the keyboard. Copy-paste works fine but only with right click, CTRL+C does not. It's refusing the keyboard input specifically.


98d7f4 No.4640

>>4639

Oh, I see now

I'm using the same manage tag dialogue, but made a custom shortcut (shift+space) a long time ago, and that still works fine. Using F3 I can't enter text either, though. Interesting

Maybe the same method will work for you too until it's patched


bc9f5a No.4643

>>4640

I tried making a new shortcut for "Manage tags" in "Options" but it doesn't work either.


98d7f4 No.4644

>>4643

Now that I've tried to reassign the key myself it stopped working on my end, too

Now we're both boned


bc9f5a No.4645

File: 7a30c0ab14bb1c5⋯.jpg (53.28 KB, 648x484, 162:121, 7a30c0ab14bb1c5ef0a0f5efbb….jpg)

>>4644

>our faces when


4d63cd No.4648

>>4633

I have a request; could there be some sort of progress bar for when the db is being backed up? Nothing specific, maybe just a x folders out of y have been copied kind of deal. I have no problems with how long the back up takes, but having an idea about when it will finish means I don't have to just sit and wait till its done.


4d63cd No.4649

File: ffe0a4c4740e11c⋯.png (221.74 KB, 1440x900, 8:5, Import Bug.png)

Same issue as >>4646 Get this for both folders and files.


12f998 No.4650

You mentioned perceptual hashes. I assume this is can be used is the parsing scripts. How do we use them?


a9ae64 No.4651

>>4649

Same. Arch Linux here.


2bf566 No.4652

>>4646

>>4649

>>4651

The fix for importing is at line 201 in the ClientGUICommon.py

change 'self._callable()' to 'self._callable( event )'

To fix the tag manager in the media viewer >>4637

At line 466 in ClientGUITopLevelWindows.py

add an else statement followed by 'event.Skip()' after 'self.Close()'

so it looks like this:


if event.KeyCode == wx.WXK_ESCAPE:

	self.Close()
else:
	event.Skip()

Hope this helps


f4a051 No.4654

>>4652

You are absolutely based, anon. Thank you, now I can import and tag. However there's another bug, "expand" in the import window cannot be clicked - it raises an exception:

TypeError

ExpandCollapse() takes exactly 1 argument (2 given)

File "/home/hydrus/include/ClientGUICommon.py", line 201, in EventButton

self._callable( event )


f4a051 No.4656

>>4654

Managed to fix it by editing ClientGUICollapsible.py at line 108.

What looks like


    def ExpandCollapse( self ):
        
        if self._expanded:
            
            self._button.SetLabelText( 'expand' )
            
            self._panel.Hide()
            
            self._expanded = False
            
        else:
            
            self._button.SetLabelText( 'collapse' )
            
            self._panel.Show()
            
            self._expanded = True

should be instead



    def ExpandCollapse( self, event ):
        
        if self._expanded:
            
            self._button.SetLabelText( 'expand' )
            
            self._panel.Hide( event )
            
            self._expanded = False
            
        else:
            
            self._button.SetLabelText( 'collapse' )
            
            self._panel.Show(  )
            
            self._expanded = True

This will at least allow to expand the button, but collapsing it still raises an exception. Hopefully by next week's release the dev will have fixed it.


2bf566 No.4657

File: e9ab0b551c4dfb1⋯.png (169.85 KB, 397x296, 397:296, EverybodyDrinks.png)

>>4656

Awesome. If you remove 'event' in 'self._panel.Hide' it should drop the exception.


e9223c No.4658

File: 68c2f3988e91fe5⋯.png (472.75 KB, 752x768, 47:48, 68c2f3988e91fe58e8cd37e0b1….png)

>>4634

>>4635

>>4636

Thanks. I've never been a chat guy, so I don't have much experience at all in this place. Another user set up the discord a while ago, and by now he's built a little community idling in there. It has worked for my purposes when I've visited, but if it breaks down or falls out of favour, I'll keep irc in mind.

I don't think I'll ever spend much time in any channel, though–I tend to enjoy talking about all sorts of things too much and find it so distracting I end up getting no work done!

>>4637

>>4638

>>4639

>>4640

>>4643

>>4644

>>4645

Argh, thank you for this report. When I added escape-to-close-manage-tags this week, I think I boshed a bit of linux event juggling that stopped key events being promoted from the dialog to its media viewer parent. I apologise for the inconvenience, but please use the thumbnail-launched manage tags for this week. I will explore this problem more on my own Ubuntu and figure out a robust fix for v235.

>>4648

Absolutely–with the new 'external db' support, I'd like add a whole bunch of gui to the existing backup/restore actions and also a new 'migration' mode to help transplanting a db to an external usb drive or whatever. Better progress indicators is a good idea for all of these operations, so I'll add it to the list.

>>4649

>>4651

>>4652

Damn, another stupid bug. Thank you for this report–I apologise again. I switched up how this stuff goes through and forgot to check the buttons. Dragging and dropping files onto that dialog still works.

>>4650

They are actually a different thing than the scripts. My perceptual hashes are internal to the db and what it uses to figure out which images look similar to each other. I moved them to a better data structure this week in prep for the faster duplicate search algorithm. How were you thinking of integrating them into the scripts?


e9223c No.4659

>>4654

>>4656

Thank you, I will fix this as well.


18fca0 No.4663

Are you going to add a way to sort any page by similarity? It would be really useful to get similar images next to each other in a list. It is quite clunky to have to "rightclick>show visually similar images" all the time.


606a80 No.4676

>>4658

I believe DerpiBooru uses perceptual hashes instead of normal file hashes.


217b56 No.4682

>>4658

I found another bug in this release, still on Arch Linux:

Clicking on "Local files" or "local tags" buttons under the search bar returns

TypeError

FileButtonHit() takes exactly 1 argument (2 given)

File "/home/nonamethanks/hydrus/include/ClientGUICommon.py", line 201, in EventButton

self._callable( event )


e9223c No.4683

File: a7fe1ed8d88468e⋯.jpg (325.93 KB, 1200x775, 48:31, a7fe1ed8d88468ec11ecb419b0….jpg)

>>4663

I'm still thinking about how to present this information on a gui level. I am currently thinking of having the db discover dupes on its own and then adding a new page type to filter and resolve those discovered pairs. My first concern will be merging tags and ratings across higher/lower quality versions of a file and discarding the lower quality version.

I would like to do quite a bit more, but I might not be able to fit it into this first version.

>>4676

Ah, that's interesting. Do you know if they talk about that anywhere in detail? My perceptual hash implementation is my own, based on DCT–afaik, while there are common techniques and some libraries, there aren't any actual standards in this space.

>>4682

I believe this is an artifact of the fix proposed in >>4652 . Did you apply that? It shouldn't come up in straight v234 or tomorrow's v235.

The same thing is happening for >>4654 . I misunderstood what was going on when I first looked, but it isn't my code making that exception.

The fix for the problem in >>4649 is to change AddFolder and AddPaths to no longer take the 'event' param, as the new button no longer passes this not paticularly useful object on (I forgot to remove it from the endpoint).

This is fixed in tomorrow's release. Please let me know if you still get any button problems in that.


18fca0 No.4689

>>4683

re: how to display dupes

If it is possible with the system you're using, sorting search pages by similarity would be useful to get similar images with minor variations grouped together, not just for finding dupes. It is pretty common for artists to create several variations of the same image, but a lot of the time Hydrus doesn't show them side by side. It would be really helpful and make for a better browsing experience.


e9223c No.4698

File: 2b065602c64bfc2⋯.jpg (279.41 KB, 1020x765, 4:3, 2b065602c64bfc258183e8863c….jpg)

>>4689

While I would like to collect or group similar images in some way in the thumbnail viewer, it unfortunately isn't possible to sort by this data, as it is highly dimensional. If I flatten it down so a < b and b < c, it won't always be true that a < c, so I can't sort them.

I envisage moving towards a 'master' system, where any group of dupes you have filtered will have a 'this is the best/original' version. I can add a checkbox somewhere to 'stack' these sorts of dupes behind their master, like collections.



[Return][Go to top][Catalog][Post a Reply]
Delete Post [ ]
[]
[ / / / / / / / / / ] [ dir / cute / egy / fur / kind / kpop / miku / waifuist / wooo ]