Version 230

Name
Email
Subject
Comment *
File	Select/drop/paste files here
* = required field	[▶ Show post options & limits] Confused? See the FAQ.

Embed	(replaces files and can be used instead)
Options	Do not bump (you can also write sage in the email field) Spoiler images (this replaces the thumbnails of your images with question marks)
Password	(For file and post deletion.)
Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf Max filesize is 12 MB. Max image dimensions are 10000 x 10000. You may upload 5 per post.

YouTube embed. Click thumbnail to play.

Version 230 hydrus_dev ## Board Owner 11/02/16 (Wed) 22:23:01 7e4f01 No.4371

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v230/Hydrus.Network.230.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v230/Hydrus.Network.230.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v230/Hydrus.Network.230.-.OS.X.-.App.dmg

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v230/Hydrus.Network.230.-.OS.X.-.Extract.only.tar.gz

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v230/Hydrus.Network.230.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v230.tar.gz

I had a great week. The first version of my parser is done and I have moved the menu id bug fix forward.

menu id

I further developed last week's new menu id system and moved the 'pending' menu over to it. Everything seems to clean up after itself as expected, and I believe this should fix the menu id bug for most users.

If you have had this bug, please try this version and let me know if you still get it.

parser v1.0

services->manage parsing scripts is no longer under construction. I believe I've created all the ui and hooked all the buttons together properly, and I've added some mini-help 'info' panels to the big dialogs as well. Created scripts will now save to the database on dialog ok and can be freely created and shared between users.

I've created a functional 'gelbooru md5' script as well–if you are interested in making some scripts, please check out how it works. Try putting 'c4d072a43f4db650c2d78840a5e8ff60' or any other gelbooru file's md5 into the test panel–you'll get some live gelbooru tags back. Putting an md5 that gelbooru doesn't have will also correctly return nothing. I didn't have time to add an iqdb script (which will show off how the 'link' child node works), but I'll make sure to do it for next week.

I would also like to finally make a 'suggested tags' control that pulls these 'file lookup' scripts for next week.

There is a lot more to do on all this, including more help, script types, content types, json parsing formulae, and a unified bit of gui to control a script in action and to describe how it is working or failing. I expect to make most of these changes as needed, so let me know what you think.

Eventually, all program parsing (including all the gallery downloaders) will be managed (and hence user-editable/fixable) through the same system.

full list

- all multiline text ctrls now support ctrl+a for select all, wew

- added dynamic menu_item tracking to new menu id system

- added submenu tracking to new menu id system

- added comprehensive menu, submenu, and menu item destruction to new menu id system.

- 'pending' menu now works on the new menu id system–is likely a major cause of the menu id bugs

- added 'info' tabs to the script/nodes/formula parsing gui panels

- added a 'veto' content type for discovering undesired redirects

- added special veto support to parsing engine

- expanded the content node panel to handle and display multiple content types

- added 'link' parsing node panel

- expanded 'link' panel test page and tied in secondary layer example url and data forwarding to children

- fixed the parsing node children control not forwarding example data up to nodes on edit

- file lookup url is now explicitly stored in the file lookup script object

- file lookup url is now passed to child parsing dialogs for test example data purposes

- expanded internal 'requests' interface a little–it now also always includes the 'user-agent' request header as the typical 'hydrus/version_number'

- wrote script GET/POST engine gubbins and tied it into the respective test page

- wrote link GET engine gubbins and tied it into the respective test page

- 'person' namespace added to default namespace colours–it is RGB ( 0, 128, 0 ), a darker version of 'character' green

next week

I'll add the 'suggested tags from file lookup' control, and I'd like to add more layout options for all the suggested tags stuff. With any luck, that should be all I need to do for it, and I can move on to the next big thing in my list: a faster dupe search algorithm.

Anonymous 11/03/16 (Thu) 00:51:29 f88595 No.4375

>>4371

>a faster dupe search algorithm

Will this include a general duplicate search, or is that much farther down the line?

Anonymous 11/03/16 (Thu) 19:15:24 b981bc No.4380

This isn't working os macOS for me. I even tried downloading the app version.

Anonymous 11/03/16 (Thu) 19:40:07 b60c23 No.4381

>>4375

As somebody who bought into hydrus to trim down my massive webm collection and get rid of files showing the same thing a million different ways, this is relevant to my interests.

Anonymous 11/03/16 (Thu) 20:00:40 877e0b No.4382

File: b68320a2b4ccf4d⋯.jpg (246.07 KB, 1915x773, 1915:773, wat.jpg)

I got this message trying to install the exe on Windows 10, using google chrome.

Safe to bypass?

Anonymous 11/04/16 (Fri) 00:33:21 ddd750 No.4384

>>4371

Can you make it so that nodes can test tags. For example, I can query a booru for the tags of an md5, and it will return an array of tags that look like this:

  <tag>
    <id type="integer">28</id>
    <name>solo</name>
    <count type="integer">435300</count>
    <type type="integer">0</type>
  </tag>

I'm only interested in the text within the name tag, but the namespace is determined by the type tag (in this case no namespace as <type> is 0, but it can be a number of different values corresponding to different namespaces). I would like to be able to test the tag and only return the name if the type is 0 (or any other value) so that I can get namespaced tags.

Anonymous 11/04/16 (Fri) 06:15:47 794881 No.4385

I'm sorry, I must be an idiot. I see how to test the new parsing script and have done so successfully but I don't see how to import the tags.

Anonymous 11/04/16 (Fri) 08:51:19 0c69cc No.4386

>>4381

Correct me if I'm wrong, but I don't think hydrus supports webm duplicate search.

Anonymous 11/04/16 (Fri) 18:29:12 0c11b0 No.4389

>>4386

It's just disabled, same with gif's. Just tag them all as "animated" + their subject and go that route instead.

hydrus_dev ## Board Owner 11/04/16 (Fri) 21:01:58 7e4f01 No.4393

File: 2105fe2d5407625⋯.jpg (3.6 MB, 2488x3189, 2488:3189, 2105fe2d540762550c8fe33242….jpg)

First priority here is to make the current system run quicker. I have a lot of other things to work on, and I have been working on this current set of 'big things' (IPFS/suggested tags/faster dupe) for way too long now, but I would like to do a bit more than just speed. The addition of some form of 'show me some dupes in my db' would be great, although I'll have to be careful not to get caught up in huge gui rewrites to present those results in a good way (and new db metadata types to handle dupe filtering decisions).

I removed animations from the existing dupe search a while ago because there were some many black/white frames. I'd like to either exclude those special cases from a future dupe search system or, better, include multiple perceptual hashes from multiple frames in an animation. Again, we'll see how it goes and how much time I have.

>>4380

I'm sorry you are having a problem. What happens when you try to run it? Nothing, or an error message? Has anything been written to your db_dir/client.log file? If you know how to open a terminal, what happens if you navigate to the extract's base directory and type:

./client

Do you get any error message printed then?

What version of OS X are you running, as well? Has anything updated recently?

>>4382

tl;dr: Looks like it.

Thank you for this report. As several people have reported this, here's my copied response:

This stuff pops up from time to time, and so far it has always been false positives. Some googling suggests runw and other pyinstaller bootloaders have been wrongly flagged before. upnpc.exe has also been a problem several times–it is the tool I use to create UPnP port forwards when you request inside the client, which is a function trojans usually do as well, so I can see why anti-virus might misunderstand what my program does.

If it is any reassurance, my windows build is 64-bit, so I presume the 32-bit bootloader is never run. I also checked the checksums, and the v230 runw.exe matches v229's byte-for-byte, and github verifies that upnpc_win32.exe hasn't changed for seven months:

https://github.com/hydrusnetwork/hydrus/tree/master/bin

The runw.exe is from Pyinstaller 3.3.dev0+d3e0cb4, which pip tells me is the latest release for my machine. I can't find a direct link to the exe to compare on my end, but if you know how to chase it down, the hashes of the exe I have are:

md5 BEA1AD92BB85AE8A626592FB165CB112

sha256 E2AF151CEE45109880BD5A2EC81528337B13CA1E479FFDE560A15355F2115DF9

I suspect the virus alert is the result of an overly paranoid anti-virus update rather than my dev machine being infected, although I obviously can't be certain. Please let me know if you discover anything new.

>>4384

Thank you, this is a great suggestion. I will add this to my list and think about how to do it. Maybe a conditional to the html parsing formula object.

>>4385

Not at all. I will hook these new scripts into the manage tags dialog through a new suggested tags control. You will click a 'fetch tags' button, and it'll send the file you opened manage tags on to the script, which will find some tags from the internet and then present them in a list for you to double-click on. I'd like to have something working for v231–let me know what you think once it is out!

Anonymous 11/04/16 (Fri) 22:24:29 9118dc No.4397

File: 66bc85df67ef9ef⋯.jpg (25.8 KB, 500x438, 250:219, 1477929203511-v.jpg)

>>4393

Just a heads up, if you're planning on doing any IPFS stuff I'd wait until the end of the year. They're currently working on fixing up a lot of the documentation to hopefully make it a bit easier for devs and a big feature is supposed to land before/in December which will allow you to add and pin files from disk without copying them into the datastore, this seems like a useful thing for hydrus so that data doesn't need to be duplicated to be shared via ipfs, you can just point to wherever hydrus stores it on disk and still share it. For someone like me I intend to just add my entire library when that feature lands.

https://github.com/ipfs/go-ipfs/milestones

That being said I was going to ask/request something related to IPFS later, but I'll add to the todo list now so I don't forget.

Is there a way to find IPFS pinned files, not just directories, something like "system:is pinned in IPFS"?

Is it possible to have shortcut keys point to share actions? I'd like to be able to select 1 or more file(s) and hit a key combo that adds the prefixed ipfs hash to my clipboard, likewise I'd like to set a separate one that will do the same for the local booru but without a prompt for the title etc., just a selection to a key combo to a link on the clipboard that I can link to someone else quickly without using the context menu.

As always, thanks for your time and effort hydrus. Be well.

Anonymous 11/05/16 (Sat) 14:07:14 877e0b No.4404

>>4393

Thanks for the info about why the malware alert thing popped up. I was able to download and install the update without issue after bypassing the alert.

hydrus_dev ## Board Owner 11/05/16 (Sat) 17:21:50 7e4f01 No.4408

File: 9bbe6cc9f701d7b⋯.gif (38.13 KB, 405x429, 135:143, 9bbe6cc9f701d7bef6c94e36e4….gif)

>>4397

Thank you for this useful information. Please keep me updated if/when anything new happens, as I expect you will notice anything breaking much faster than I will.

You can search for ipfs service membership with system:file service, which is down the bottom of the system predicates. Your ipfs service(s) will be listed with 'local files' and everything else.

The shortcut engine is a horrible mess right now. I have plans to completely overhaul it so users can set whatever shortcuts they want to basically any action they want, but complex per-service mappings like you want just aren't supported yet. Please hang in there and give me your further feedback when I can start rolling this stuff out.

Anonymous 11/05/16 (Sat) 20:46:58 9118dc No.4409

File: f0c31c93a718316⋯.jpg (55.83 KB, 450x285, 30:19, 15 bonewheel skeleton.jpg)

>>4408

I'll be sure to post about it when everything important merges and help out with info then if I can.

>system:file service

Thanks mr skeltal, that's what I was looking for.

Anonymous 11/08/16 (Tue) 14:34:06 544d73 No.4428

>>4393

Not the other OSX user, but Version 230 also does not work for me either. Currently using Sierra, 10.12.1.

Launching the app bundle version from the Finder simply has the app's icon appear on the dock, then closes right back down immediately. Launching the app from the terminal with

open Hydrus\ Network.app/

does the same with no additional output.

Navigating into the app package directory with Terminal and running

/Hydrus Network.app/Contents/MacOS/Hydrus Network

directly produces the following output:

Traceback (most recent call last):
  File "<string>", line 11, in <module>
  File "/Users/hydrus/Desktop/code/pyinstaller/PyInstaller/loader/pyi_importers.py", line 270, in load_module
  File "/Users/hydrus/Desktop/code/build/Hydrus Network/out00-PYZ.pyz/pkg_resources", line 48, in <module>
ImportError: No module named moves

Anonymous 11/08/16 (Tue) 20:51:38 b981bc No.4432

>>4428

I'm the first guy who complained about it not working. Just did what you said and got the exact same error. Also running 10.12.1.

Anonymous 11/08/16 (Tue) 23:40:12 044981 No.4434

Why does hydrus only download samples from danbooru? This has been a problem since forever yet I hardly see anyone mention it.

Gelbooru works fine, but I'd rather get my bulk downloads from danbooru since too much garbage on gelbooru

hydrus_dev ## Board Owner 11/09/16 (Wed) 22:16:59 7e4f01 No.4436

>>4428

>>4432

Thank you for this detailed update and the confirmation. I was able to reproduce the bug and I believe I have now fixed it. The Linux release had a similar problem ~v217. Please try the release I will put out later today and let me know if you still have any problems.

>>4434

My current booru parser can't handle detection of danbooru's larger/original images at the moment. The new parser scripting system I've been working on will be able to handle it easily, so the danbooru fix will be available when I next overhaul the downloader engine, which won't be long from now. Please hang in there until then.