/hydrus/ - Hydrus Network

Not sure if I should put this here or in the bug thread, but I'll put it here because it's related to this version.

mpv was rolled out to Linux in the last version but it didn't work for me in that version and still doesn't in this one. You introduced the popup after clicking about to see why it doesn't work if it says that it's not available. I tried doing what it said (installing mpv-libs) but that didn't fix the problem, so maybe putting the trace here for you can help you figure out what's going one. I use Fedora (linux distro). Also, The "libmpv.so.1" file that the error talks about is indeed in the right spot, where it says it should be, however, "libnsl.so.1" is absent.

Traceback (most recent call last):
  File "site-packages/PyInstaller/loader/pyiboot01_bootstrap.py", line 151, in __init__
  File "ctypes/__init__.py", line 348, in __init__
OSError: libnsl.so.1: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "include/ClientGUIMPV.py", line 21, in <module>
    import mpv
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "/home/hydrus/Desktop/hydrus/hydrus/venv/lib/python3.6/site-packages/PyInstaller/loader/pyimod03_importers.py", line 623, in exec_module
  File "site-packages/mpv.py", line 52, in <module>
  File "site-packages/PyInstaller/loader/pyiboot01_bootstrap.py", line 153, in __init__
PyInstallerImportError: Failed to load dynlib/dll '/home/['''my username''']/Hydrus_Network/hydrus network/libmpv.so.1'. Most probably this dynlib/dll was not found when the application was frozen.

>>13651

ok, just updated to see if it still impacts the files and it does.

>>13646

anon here who suggested database improvements + smartcrop. I've been testing some stuff. are you interested in a python hashing library coded in rust? I've got an implementation working with javascript ffi bindings to rust, I can possibly port it to python. It hashes the file, produces a smartcrop (~20kb per thumb, around 250x250 dimen. IIRC), produces a perceptual hash, etc in one pass of a file read. I can add stuff to it, on average it has 500ms-1s execution time but it's also multithreaded across all cores so (500ms-1s)xN threads if I/O isn't a bottleneck. I haven't worked out the specifics of mimetypes yet because that's a pain. I also need to add ffmpeg support for video thumbnailing.

if your average image size is 700kb, then the thumbnails should only increase the total data by ~3%, 4% for 500kb average, etc.

also, a blake3 implementation was just released. it's 6x faster than sha2, but I know you even keep md5/sha1. migration seems like a pain.

https://github.com/BLAKE3-team/BLAKE3

anyways, this should markedly improve import times while also providing beautiful thumbnails at a very small space cost.

>>13646

>>13653

also, a more efficient method than ffi bindings would probably be an ipc pool. you would spin up process and send it file paths. with ffi there's a cost to the calls, plus the threadpool needs to be spun up, so batch requests are required for ffi. or maybe a hybrid approach by sticking the ffi call into a python thread would work. I can give it a shot but it might take me some time.

>>13650

Unrelated but that's a nice song, can you tell its name?

>>13656

田中理恵　ニンギョヒメ

https://www.nicovideo.jp/watch/sm7730082

get the "aha music" plugin for chrome and play the song and click it, it will search its database for a similar song and if its not obscure or too much background noise, it usually gets the song.

https://chrome.google.com/webstore/detail/aha-music-music-identifie/dpacanjfikmhoddligfbehkpomnbgblf

ok, had 2 crashes since version 384 was installed, one crash just had me close it, the one just now just flat out closed itself. I have no real idea why but i'm assuming mpv related due to parsing webm/gif at the moment

>>13658

selected 2 gifs, right clicked, crash

reopened, did the same, no problem.

>>13657

Thanks.

>>13661

another crash when going from one video to the next and more nothing showing up in log for a crash reason.

>>13665

>what is a false positive

you don't into computer do you son

Thank you yet again, based dev.

test post, help compute

hydev here, I had some trouble posting, sorry if a bunch of dupe posts suddenly appear

>>13648

Thank you for this report. I have libnsl.so.1 on my Ubuntu machine, so for some reason PyInstaller is not including it. In some searching, I saw there was some issue with this no longer being included in some recent versions of Linux, Fedora included, so perhaps it is being assumed that you will have it so it doesn't need to be bundled. Here's some info:

https://github.com/knapsu/plex-media-player-appimage/issues/13

I know very little about Fedora, but could this just work?:

sudo dnf install libnsl

Is dnf the Fedora version of apt?

If it doesn't, I'll send you my libnsl.so.1 to put in your hydrus base directory and we'll see if it works, in which case we can try manually bundling it in the build.

>>13651

>>13650

>>13652

Thank you for this report and these examples. The old native hydrus video viewer works in x/y frames rather than timecodes, so single-frame music vids like these are in temporary limbo. I think the viewer is thinking these are single-frame gifs and is ignoring their duration metadata. I will make a job to add the seekbar in this case and make sure it works.

>>13653

>>13654

I am afraid I do not know anything about Rust, so I do not feel I can talk very confidently about this. I do not want to ask you to do some clever work that I then do not know how to integrate. Can you talk more about it?

I do not want to add more phashes to the database at the moment (the bottleneck in the dupes system atm is processing potential dupes rather than needing to find even more), and I am not interested in adding anything like blake3 unless it gets wider adoption, but I would be interested in a simple python file that could take an image in PIL or OpenCV format that would then return the cropped SmartCrop version, or the respective coordinates. I can handle the ffmpeg side of providing a jpeg/png screen grab. If that is something you think is possible and would be willing for me to fold it into the code so I can play with an experimental thumbnail mode for it, I would appreciate it.

I am overwhelmed with my bugfixing/Qt-cleanup/small jobs queues at the moment, so please forgive me if I cannot spend the correct time or attention on anything too new or clever.

>>13658

>>13661

>>13664

Thank you for this report. I am sorry you are having trouble. Can you say what OS you have, and how you run hydrus? Is it with my build, or do you run from source?

For now, I recommend you change the options-media settings to play video/audio/animations with the native viewer.

>>13665

>>13666

Thank you for this report. I had a similar question on tumblr about it, if you are comfortable visiting this link:

https://hydrus.tumblr.com/post/190847701679/just-a-heads-up-that-the-latest-version-of-hydrus

This has happened before, and I expect it will happen again. Hydrus does some funky stuff and is built with duct tape, so we get hit by false positives from time to time. It tends to go away in a couple of weeks when the next set of anti-virus definitions go out, or my code changes enough to not trigger the algorithm.

Please keep these reports coming–if we ever are hit by something real, I obviously would like to know asap.

Could a way to sort duplicates for processing be implemented? For example, if I have a bunch of "pixel-for-pixel" duplicate images, it would be nice to just get those out of the way first and quickly. Because they're dead easy

>>13672

win 7, your installer, current session is 1424860 weight, been going through /gif/ mass trendwatcher removing shit I don't want, so its a constant opening video, and due to sound being a primary reason im able to do this, im keep with mpv, even with crashes its so much faster then opening exturaly even if double click still did that.

for what its worth these were from the preview window not the media viewer, now that I say that when I get back to parsing gifs ill try media viewer only and see if it makes a difference, right now i'm half paying attention to news while parsing images.

>>13682

ok, had a crash in media viewer too, then shortly after a second crash.

part of me is thinking something didn't install correctly, so fingers crossed wednesdays version will fix it.

>>13686

Ok, now that endchan is back up, and because its less annoying to post there, i'll be moving back to posting there. however I have an interesting… problem?… and i'm wondering if there is a solution.

so right now with hydrus I have a threadwatcher that is 2100 watchers big, and this poses some issues with parsing, some threadwatchers only have 5 images, some have 150+

some have a good name that is easy to follow, some decided a new name scheme each fucking thing is acceptable.

some make it to competition and some struggle to get passed the 100 mark.

so lets take something like… vore seems to be a good one. there are 3 threads where its the first word, but then there is /vore/ which has 22, then there are different flavors of vore… ect ect.

now lets say I want to parse them all at once, im unable to do that because they are not in any way lined up for me to do so.

now, I will admit, since the delete tagging has become a thing, I have been going through watcher lists and I probably went through several hundred this week alone parsing every images. going forward what im going to ask for has a far lesser need or impact, but if its easy to implement it would be helpful currently.

is there a way to add an exception 'search' to watchers, where if I wrote 'vore' as a search term anywhere in its subject it has 'vore' it would show it and it exclusively? when I remove the term it would show everything again?

at the very least for me this would greatly help the parsing process and even if it took a bit to implement, I have a saved session that has a metric fuckton of watchers as well.

i'm hoping that by the end of the year with all the tools that came in I can be done with parsing old watchers.

>>13687

I agree. I think adding a text filter to watchers, downloaders, and subscription lists is a good idea, and I would already like that tech available for some other parts of the program. I need to do some research first and rejigger my list code to support a filtered view, and then I will try to get this done.

I am sure you know, but just in case you do not: you can shift/ctrl-select multiple rows in the watcher list and then right-click->show all importers' new files to do easy one-step en-masse display of imported files.