/hydrus/ - Hydrus Network

>>14716

Thank you for this report. I am sorry for the trouble. When you try to boot the server that way, does the terminal say:

The server is already running. Would you like to [s]top it, [r]estart it here, or e[x]it?

It seems like an old server exe was indeed running, but perhaps it did not have a terminal appear for it.

In any case, if you do get that line when launching from a terminal, if you put in 'r' or 'x' and hit enter, it should close the old instance.

When you run the server, does it not show in the terminal? Maybe this EOF is coming back immediately because there is no terminal to get input (e.g. your 'x' character) from. I will try to brush up this error handling in either case.

>>14720

Unfortunately not yet. This is complicated data, and hydrus does not have support for the translation overlays yet. I expect to expand notes to allow note parsing in a downloader, and I know some users are enthusiastic about grabbing raw translation text at the very least and put it in a file's notes, but it isn't available yet.

>>14726

Yes, it's turned on.

>>14721

>I don't have SQL experience so please let me know if what I am doing will lead to breaking things in the future. It seems to work for now.

Sorry, to actually answer this, switching journalling modes should be fine. WAL is nice because it allows some faster writes and so on, but hydrus can work in no_wal mode ok. I think your plan to create a no_wal client and then transfer it over and keep using no_wal is correct. You can also just run the sqlite3 executable four times and run:

.open client.db
PRAGMA journal_mode TRUNCATE;
.exit

For each of the client*.db files. You might need to do that to the files when they are on local storage.

The reason why I don't encourage network databases is that samba etc.. don't always offer 'real' file locks, so while SQLite might say 'hey, filesystem, lock this file, I am about to change something important', not all the systems along the network connection can guarantee what a purely local system could. Dropped connections, multiple user access, OS lock synchronisation, it is all less great. Also access latency, which can be critical in hydrus, can be crazy compared to a local drive. In TRUNCATE mode, you should have ok access as long as the network is ok, you do not want to sync with the PTR, and you only access the db with one computer at once. Otherwise, I suggest moving the db to your local machine and storing files on the network if you need the space, as described generally here:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

I tried replicating the issue you had with the error being raised as the 'memory' db is attached, but it turns out my network is fine with WAL. I will try to catch the error anyway, and see if I can detect 'WAL not supported' and recover. Do you have the full error traceback, or can you reproduce it? The actual SQLite error type and text produced by _InitDBCursor would be useful.

My 'the client is already running' code will not work with a db hosted on a network, as well. Multiple computers accessing the same shared db is dangerous, so it is up to you to enforce.

Hi, I read we're supposed to run the gallery downloader to catch up before adding to subscriptions. This is not a smooth workflow though if you add lots of queries because for each one, you have to click through several menus to transform the query into a subscription (there's no simple rightclick option for "convert to subscription"). The program also has no memory of what gallery downloads you've run aside from the gallery page, but the gallery page method feels unreliable since I've lost my sessions multiple times through misclicking load new session or even the my pc freezing and losing the open session. At this point, I have to keep track of my queries in 4 separate places: my subscriptions, whatever gallery download page I happen to have open, a "completed but didn't like" txt, and a "unrun queries" txt. I think a "convert to subscription" button and "previously run queries" could go a long way to helping people that are adding lots of separate queries. That, or a way to make the gallery downloader page permanent instead of coupled to the session which if you're like me you usually wont remember to backup the session before each time you close.

Any way to run just the client api without the rest of the gui? With multiple instances running the memory consumed really begins to climb.

I've been having problems downloading from Nijie. I thought the parsers were broken, but if I look at them they're fine. The problem seems to be that a login is required to get to the page with the download links. I used Hydrus Companion to send my cookies to Hydrus after logging in, and even manually put in the NIJIEJIEID cookie that my browser was using, but Hydrus is always asked to log in.

What am I doing wrong?

>>14740

Yeah, I do the same deal with a couple of notepad lists outside the program to keep track of pending/good queries.

Now that I have subscriptions loading fast, I believe I can add exactly what you want–some sort of UI feedback on gallery downloaders for whether a query is in a sub atm, and then a menu action to add it to an existing sub. I just need to find the time to do it.

I am not really happy with how the client works with a session other than the auto-saving 'last session' as the default. Can you think of a way to get that work better? Should I just mandate some kind of 'last session' and always save/restore on exit/boot?

>>14745

Not at the moment. A very long term plan is to increase the Client API and client db in general to be an optional network 'node', perhaps even taking some responsibility from the server, but the UI and session code is too tightly coupled to make a headless mode yet.

>>14750

I am not familiar with nijie's specifics, but one last step for complicated login situations can be the User-Agent header. Try hitting network->data->manage http headers and then set the nijie domain or global User-Agent to the same as your web browser's:

https://www.whatismybrowser.com/detect/what-is-my-user-agent

If nijie tie login cookies to User-Agent, that will hopefully fix it all.

Is it possible to export files, including the tags AND the known URLs?

Basically a way of packing up the file in a way that could be re-imported into another Hydrus database while retaining the most important details?

Any thoughts on debloating the PTR of all those single use and typo tags?

If a single use tag has been there for over a year it probably will never get another use.

>>14750

I had the same problem a few month ago.

Nothing I ever tried fixed it, it fixed itself after a little more than a month.

It may be a soft temporary ban based on your internet provider. (less ban and more auto rejection of a download request.)

I launch from the command line so I can point hydrus to a db on an external hd. Is there a way to pass a pw into the launch command. previously when I tried to start w a pw-protected external db i just get a "db is locked" error on startup. thx!

So I was syncing to the PTR and tried importing some pictures, and they were getting auto tagged with info from the PTR. Is there a job to update pics with tag info from tag repositories? Or does it just auto tag based on info present at import time?

When using Animebox.es to access Hydrus, is it possible to have it shown all known tags rather than just showing my local tags?

Currently the app only shows my tags.

What's the difference between virtual siblings/parents and the old system? I thought the old system worked the same.

>>14768

Files automatically get the tags from the PTR as long as they exist in the PTR and it's fully up-to date in the database.

>>14772

>Old system

You make a parent relationship, the tags get instantly added. A fuck up means you need to manually check every file that could've been affected by that relationship.

>Virtual siblings/parents

You make a parent relationship, the tags get added except they're not actually tags. If you have to remove/modify the relationship the tags get removed/replaced instantly. Fuck ups are easier to solve and modifying the relationship modifies the tags, instead of just adding more tags.

>>14753

Anything that would allow you to not have to manually track what tags you have or havent gallery searched would be awesome. Currently subscriptions functions that way since it can keep everything you ever ran, and you can just pause what you arent interested in without fearing of it disappearing with the session data.

If I had to guess at what feature could fulfill this, it would be supplanting the gallery searches tab altogether with subscriptions, and making the tab permanent.

Where did the option to automatically add parents and replace tags with siblings get moved to? It was in the tag management window before and it disappeared after I updated, I used that feature a ton.

I have a pretty big gallery download that is going to take a few weeks due to bandwidth limits and I'm scared of losing my progress on it. What's the most that I can do that won't cause me to lose progress, e.g. closing the client, migrating my db, updating hydrus, etc.?

>>14784

None of those things should affect any open tabs or their state, in my experience.

>>14756

Not yet. Just tags and files are mass-exportable atm. I want to write a system that uses the Client API, when it is more fleshed out, to do direct client-to-client migrations of all useful info.

>>14758

Yeah, this is an increasing concern of mine. I have exhausted most of the 'just make it work faster' tech I can, so now it is time to regulate what is actually shared. In the next two years I expect to focus on removing old unused content and allow you better control over what you get.

Before 2020 is out, there should be a filter serverside to stop certain tags being added, and I understand the lads who run the PTR will block 'filename' tags as a start.

A step after that will be allowing you to block what gets added clientside, so you can say 'get PTR, but only creator/series/character tags' and so on.

After that, I really want 'recycling', so old unused/invalid content is removed from master definitions, and the whole thing has the capability to shrink and potential for equilibrium. Even if it never stops growing, we can discard crap along the way.

>>14767

No, I don't think so. This password is my thing, right, where you hit database->set a password? The only way to solve that atm is to enter it when the entry box appears on startup.

The 'database is locked' error is an unusual one. That suggests that something else is accessing your database, so hydrus cannot get exclusive access to boot itself. Is there any chance another version of hydrus is still running in the background or similar?

This error has come up for a couple of users before due to hard drive weirdness, and it went away when they rebooted.

>>14769

Sorry, I don't know.

>>14771

Hmm, thank you, I will look at that. Some of those actions have bad feedback on the situations where they cannot apply, or need something undone first.

>>14777

There isn't an excellent method yet. You can import/export files and tags fairly nicely, but inbox/archive and import time and ratings aren't easy to move.

As here >>14790 , I want a really nice system that does everything, but we aren't there yet.

Check out right-click on a lot of thumbs->share->export->files for files, and services->tag migration for tags. That tag dialog is scary and big, so take it slow and feel free to ask questions. You'll want to export you current 'my tags' to a Hydrus Tag Archive file, and then import that file to your other client in the same dialog once all the file imports are done.

Do two file import/export rounds, one for inbox, one for archive, to preserve that information.

>>14779

It is being phased out for the virtual siblings and parents system. Unfortunately, that button caused havok at the PTR level as a bunch of erroneous tags were being added by unusual or older clients. If you ever saw a bunch of 'shadow the hedgehog' tags added to files with 'shadow', for instance, that's what caused it.

Siblings are now virtual. In 414, due in a couple of weeks, parents will also be virtual. Siblings and parents implied by rules will not 'exist' in the database as hard tags, and if you undo the sibling/parent rule, everything goes back to how it was before. You'll be able to add parents and siblings if you want, but in normal searching and display, you'll see siblings and parents as if they were really there, so there isn't a great need. Furthermore, parent logic will be universal, so all files that 'should' get a parent, now will.

A 'hard-replace' system that does permanently replace or add tags according to rules will come later on as a separate thing.

Please let me know how virtual siblings and parents works out for you as I roll all this out. It is a lot of work on my end, but I am hoping to solve several difficult technical problems with it.

>>14780

I think that is how it is supposed to work. Perhaps they changed. I think that downloader is a little old.

EDIT: Ah, when I did a little test with an example URL:

https://gelbooru.com/index.php?id=123456&page=favorites&pid=0&s=view

I now get a login screen. Maybe you can't look at user favourites other than your own or something now? Or you just have to be logged in to see any favourites?

Best solution is to set up your gelbooru login in hydrus, or use Hydrus Companion to copy your cookies across. If the only thing they changed was login access, it should work. Let me know how you get on!

>>14784

>>14788

Current limit is about 300,000-600,000 session weight, and then memory limits start breaking session save. You get errors if this happens. You can check your current estimate session weight under the 'pages' menu of the main GUI.

To keep the client snappy, try not to go above 50k session weight.

>>14796

Logging in with my Gelbooru account worked. Thank you!

>>14792

There's a good chance that's the case. The last time I got the error was when I was doing updates from like 334.

Presume the safest way to do this would be to back up my .db file (no pwd), set a pwd in app, launch from command line, and restore .db if I'm locked out.

Will attack that this week and let you know how it shakes out.

How do I go about storing my files on a NAS? Is this possible with Hydrus, or do they need to be stored locally?

Retard here.

I deleted some downloaders from Hydrus because I thought I wouldn't use them, but now I'm having trouble adding one back (danbooru). I tried adding all of the PNG downloader files back, and although it now shows as an option in the gallery download it instantly says "DONE" with no results when I try to start a download.

What do?

>>14803

read this https://hydrusnetwork.github.io/hydrus/help/database_migration.html

>>14804

make sure that your parsers are linked to the correct url class in network>downloader components>manage url class links

Is there a way to remove PTR tags from my local copy of a file other than waiting/hoping for your removal petition to be approved?

And/or remove PTR tags (just for my db) entirely?

>>14807

>read this https://hydrusnetwork.github.io/hydrus/help/database_migration.html

Thanks, good to see it works over a network as well.

holy freaking crap PTR syncing is slow beyond belief, this will take actual weeks at this rate

how do i speed it up if i've already downloaded a full installation from https://cuddlebear92.github.io/Quicksync/ ?

>>14811

nvm, I think I figured out a solution:

(?<=hydrus import folder\\)[^\\].+(?=\s-\s)

"hydrus import folder" being the folder that the files are all stored in.

Do you need to keep a watcher tab open for it to work? Does closing a watcher's tab kill the watcher, and you need to go back and make it again? If it doesn't, where do you see the list of watchers currently active, so that you can change settings and keep up to date on them, like you can with subscriptions in "network → manage subscriptions"?

Is there a way to group by alternate/duplicate group, besides just "view this file's relations"?

>>14731

Is there any way to collapse a picture and its alternates into a single collection without having to tag them both under a single title? If not, are there any plans to allow collecting by alternates, in the same way that one can collect by title/chapter/creator?

>>14813

It should take less than a week to download the entire PTR currently, so what's the bottleneck for you? It should only take a day or two at most to sync up to current from the april version of the fully updated client they have there. If you're running on a regular ol' HDD it does take forever to actually process updates.

>>14815

Closing the tab kills the watcher. Subscriptions are permanent watchers.

>>14816

Not at the moment, but it is a planned feature.

>>14813

If you have an SSD, put the database on there while doing the initial processing if you can. It will speed up massively.

Is there any way to change the number of rows it scrolls per "tick" of the scrollwheel in search pages? I'm not seeing one.

>>14822

Options > thumbnails

Is anyone else getting a weird error when trying a pixiv artist lookup?

"CloudFlare needed solving"

After which the gallery download fails.

>>14825

I came here to ask about the same thing. I guess that means it's something with Pixiv, and not something I messed up.

>>14826

specifically, when I'm on a post page and I use Hydrus Companion to download the files in that post, it says something about a cloudflare needing to be solved, then it waits about 5 seconds, then I got a 403 error and the download fails, meanwhile a bunch of html garbage is outputted to the terminal.

Sorry if off topic, looking for the name of an old board before the nuke that was about downloading/archiving shit due to the TPP. I first found this board via thread about it on there and wanted to try diggin around 'n see if I could find some old threads.

I started downloading the PTR, and decided to start processing it before it was fully downloaded. It got to a certain point and then I stopped the process since it was taking too long.

Now when I try to search for tags it just says "loading results…" in the selection tags area and hangs after a while.

I already used every option in maintenance, check & repair and regenerate. I also removed the PTR via manage services.

The PTR's tags are still there, and I can only access my files by selecting the system:inbox tag (which does load my files) and manually selecting tags from the files that show up from the system:inbox results.

Is there any way to fix this?

Is it possible by now to change from SQLite to another SQL service, preferably MySQL/MariaDB?

>>14831

https://hydrusnetwork.github.io/hydrus/help/faq.html#sqlite

https://gitgud.io/prkc/hydrus-why-sqlite/blob/master/README.md>>14831

>>14831

Why.jpg.avi.gif.xvid.exe.doc.what

Is someone else having problems recently with "pixiv tag search" in both the gallery downloader and in subscriptions? It worked for me before, but now when i try to download any pixiv tag, it appears the message "The parser found nothing in the document!" i try with different tags/search in english and in japanese but none seems to work. On the other hand individual downloads and the artist look out are working fine.

>>14834

Read up, the answer is yes. No results on either what causes it, or when it will be fixed though.

how do i remove the public tag repository

i tried it to see how well it would tag my shit and it wasnt that good

can i just delete the mappings.db and the 2 other large files and the small hashfags will get removed automatically?

or do i just copy all my shit and nuke the entire db

overall the program is pretty based

>>14840

delete the service from manage services then run a database vacuum, you can search for the update files and delete them through hydrus also.

>>14834

>>14825

>>14826

change your pixiv password and reimport your cookies.

Is there a way to "redo" my duplicate sort for all files? I've done some on several levels but i want to redo it.

This question might have been asked before, but can i do duplicate search for webm, mp4 and other movie type files?

I'm currently running the Linux version of Hydrus on Fedora. Would it be more stable or helpful for me to just run it in Wine, since the Linux builds are only made for Ubuntu? I used to do this, but I moved to the actual Linux version because I figured that would be more stable than running the Windows version through a compatibility layer, but maybe that's not the case.

Running from source isn't an option for me because I'm a low-IQ pleb and don't know how to python.

>>14843

No, video de-duping isn't a feature yet, but it's one of the most requested ones.

I've run into a problem with known urls and hydrus companion. A very useful feature of the extension is that it highlights links to known urls on webpages you visit, as green if all files with that known url are in the db currently, as yellow if some are in and some are deleted or in the trash, and as red if all are deleted or in the trash.

A problem I've encountered is that when using boorus or other similar sites to grab images using Hydrus companion, the parser for that site will grab the url of the image on the booru, but if that booru page has a "source" link, then it will also grab that one. That sounds like a smart idea, but it leads to the occasional problem where, if that file was only part of a set of files on the post page (like on pixiv for example), hydrus companion will still highlight it green even though I haven't downloaded all of the files from that post page, because I didn't download from the post page at all. It's just the booru source that was grabbed as a known url, without grabbing the other files from that post page. This gives me the wrong impression that I've already completely downloaded a post when I see the green highlight around its link, when in-fact I never downloaded directly from the post page, so I only have the one file from the booru rather than every file that's part of that post, and that's caused me to miss some files because hydrus companion makes it look like I have them all with the green highlight, when I actually don't.

Is there any way to remedy this, or is this something I just have to keep in mind, that even highlighted post pages might have files I didn't download, because the known url might not mean that I downloaded directly from the post page and got all files? Thanks, and sorry about the lengthy explaination. It's a very specific situation, but one that I've already run into multiple times, because I download from boorus a lot, and the booru source links keep getting grabbed, as if I downloaded from the source link directly.

is there any way to download stuff from boorus that use cloudflare?

>>14807

>make sure that your parsers are linked to the correct url class in network>downloader components>manage url class links

That seems to have fixed it, thanks!

Is there a way to have logical operators when searching? So I could do something like "anime OR manga".

I've been using hydrus as standalone (client+server i guess) with DB on a separate (external) drive for a long time. As I understand, it's possible to move server+data part onto another machine and connect to it with a client remotely. If so, what's the basic steps in doing so? Help was very intimidating and confusing with all the "if you know what you're doing" stuff.

Also have anyone tried running hydrus or at least server part on FreeBSD? I want to host it on my FreeNAS server either in freebsd jail (cheap) or linux vm (expensive)

is there a way to make hydrus execute an external program, or some python, (e.g. youtube-dl) for a url class?

Quick question, I've automated every part of my daily image import except that I still need to open up notepad++ and add md5: to the front of my exported md5s. Is there any way to add these to the clipboard with that portion already appended?

Thanks for the program, it's my most used daily by far.

Hey, I am sorry for not checking in correctly. I was tied up with all the sibling/parents bullshit, and then was still trying to catch up on my schedule last week. Hope to be getting back to normal once the election is past.

>>14855

Best way at the moment is to get your browser through the CF check and then:

A) Make sure your hydrus User-Agent header (network->data->manage http headers) is the same as your browser.

https://www.whatsmyua.info/

B) Use Hydrus Companion to copy your cookies over from your browser to hydrus.

https://gitgud.io/prkc/hydrus-companion

>>14861

Yes, make sure you have help->advanced mode on, and then restart the client. All your search pages should have an 'OR' button beside the 'searching immediately' bit in the tag autocomplete dropdown area. Click this, and you'll get a really cool place you can paste long strings with all sorts of operators. A user wrote this parser, it is great.

Alternately, when you type a tag, instead of hitting enter to submit it to a search page, hit shift+enter. This will put it in a 'OR chain' queue, which appears in the autocomplete results. You can add more tags to the OR chain with shift+enter on more tag searches, or just hit enter to cap the chain off and submit it to the search.

So, for anime OR manga, you would go:

anime

shift+enter

manga

enter

>>14865

This is the main help for that, if you missed the precise page:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

tl;dr: Hydrus has two parts you care about: database files and media files. DB files should be on your local machine, media can be over a network. Make a backup before you try anything and you can't go wrong!

Just make a backup (i.e. a full copy of everything while it was working), then you can fuck around. Let me know if you run into any trouble.

If you run a server, you can move all of that to your network as long as the network machine runs it. Then multiple clients in your local network can talk to it and sync tags across multiple clients. Hydrus is not great for this (the 100,000 second refresh rate is usually a bit slow for LAN situations), but it does work.

>>14866

I don't specifically remember anyone doing it right now, but if you can get the python modules running, there shouldn't be any huge problems running from source. Qt is usually the biggest pain in the neck.

https://hydrusnetwork.github.io/hydrus/help/running_from_source.html

>>14868

Not yet, but I want exactly that. I want youtube-dl for twitter vids and many of the other things it can do.

Rather than hardcode youtube-dl, I will make an 'external executables' manager to handle multiple user-created external program situation and pre-load it with youtube-dl for URLs and eventually some others like ffmpeg or imagemagick for batch file conversions.

>>14872

Hmm, I don't think so. I remember talking to another user about this. I will add an option so that when you copy a hash to clipboard, it can prepend with the booru standard 'hash_type:hash'.

>>14850

Thanks, this is an interesting report. I'll talk to the guy who writes Hydrus Companion about this. I think normally this shouldn't happen, much as how if you feed a pixiv URL into hydrus, it will check it even if it has seen it before, because it knows pixiv URLs sometimes have multiple files. Maybe HC uses a different look-up, or something weirder is going on.

>>14844

If it is buggy for you, yeah Wine is generally regarded as good with current versions afaik. Most bugs I hear about still on Linux are Window Manager specific, mostly to do with the 'floating' windows like the tag autocomplete dropdown, the popup message toaster, and the hover windows over the media viewer. All that seems to work better in a Windows environment.

Let me know what you discover, if you give Wine a go.

>>14830

If too much processing happened at once, perhaps the maintenance calls that optimise the database once it has some data in did not kick in in time. Try running database->maintenance->analyse and choose 'soft' version.

If that does not speed things up, please try doing some profiles of autocomplete lookups, as here (although sorry, this help page may be a little outdated):

https://hydrusnetwork.github.io/hydrus/help/reducing_lag.html

Even if it takes a long time, your tag lookups should finish, even if you have to wait a bit. If you are on an HDD (rather than an SSD), the PTR may just be unbearably slow to use, in which case I recommend you remove it under services->manage services.

>>14817

Not yet. Alternates aren't a developed system yet, really, more of a landing pad for things that don't match the duplicates system. I will flesh them out and surface them to the UI level so we can do sorting and collecting when I next iterate on that system. I expect that to be a popular item in the next 'big work' poll that I will put up for Christmas.

>>14850

>>14882

I talked to him. Unfortunately, this is tricky. The checks that I do in the downloader to re-hit a pixiv (or other multi-file) URL have more information available than the Client API currently provides, so this can't easily be added to HC atm. This information is likely to come to a future version of the Client API, so we will revisit it then, although we'll have to figure out a new type of green or whatever to mean 'some, but perhaps not all' in multi-file URL cases. But even then, we couldn't guarantee a full green in any case, even with more info, as we don't know if all the files on a page are in hydrus (some new ones may have been added since the check, and what files are currently in a page's html/json isn't known until the URL is downloaded and parsed in hydrus anyway).

For now, although it sucks a bit, the best solution for now is to know that green on a pixiv page or any other mini-gallery means 'some, but perhaps not all'.

>>14878

I ended up setting up a blank server (using linux vm, couldn't get python modules installed on freebsd jail, i don't know freebsd nor python/c programming very well).

I connected both my "main db" client and a blank db client from another machine and uploaded some images. It was confusing but I think I understood some things in the end after lurking some more. The 100000 second refresh rate could be ok but lack of manual "sync now" button is confusing. Not to judge but if it's an anti-spam measure it should probably be implemented server-side…

Things I ran into:

1. Images appeared as red squares with ψ logo and were missing tags on the "blank db" machine. Clicking the image would load it (or its thumbnail?) however. No idea how to load all of them.

2. For syncing tags I assume that I also need to use remote tag service, but I haven't quite grasped how multiple tag services work, and migration dialogue looks menacing and confusing, I've yet to experiment with it because I need to make another backup, just in case.

Two things that I don't quite understand are: 1) does sync get all the references, thumbnails, raw data? If I have 30GiB worth of dank memes and exquisite art to my NAS, will syncing it back to local machine end up copying over all 30GiB (including thumbnails)? and 2) does server require GUI for some features? Trying to modify a user from a client's administration menu just says "not available yet", and docs mention server minimizing to tray or something, which isn't a thing on a linux server without X11 or any sort of GUI installed and yet it works and doesn't complain.

>>14879

I'm guessing bigger problems for me are dealing with FreeBSD. I tried using pip to install all the dependencies listed but it would fail at compling some of them, i think it was cmake complaining about LibVU or something, after spending 4 hours to get only so far I just gave up and used linux vm lol. Maybe next time.

>>14844 >>14883

Works fine for me on Linux even with VM with tiling features (KWin + Krohnkite). I'm using it from source because it actually was the most stable, best-looking, and easier way. t. low-IQ javascript pleb who doesn't know how to python nor c.

Just install pip3, qt5, as well as other fedora packages listed here: http://hydrusnetwork.github.io/hydrus/help/running_from_source_linux_packages.txt . Dunno about fedora but you could probably install missing stuff with pip3. After that it should be just a matter of executing the client.pyw

>>14877

Nice. I remember people asking for this a while ago, didn't know it was implemented. Is there any more documentation on it somewhere, or is that all that there is to it?

>>14755

I did that, and along with manually updating my NIJIEJIEID cookie it seems to be working now. Thanks!

Not sure which one actually fixed it, or if both were needed.

Are the default bandwidth settings just right, or can I raise them?

I understand that they are there to prevent IP blocks, but is 2GB a day enough to get you blocked? Would setting a small hourly limit (1GB or so) but a larger daily limit (10GB?) be good enough to avoid the banhammer?

>>14892

depends on the site, but defaults are quite low. you can likely remove them all together.

is there any way to import files and add the original filename as a note? My main concern is content from places like pixiv where I find it helpful to have the image id.

I don't think I understand the whole 'session weight' thing properly. Since I had about a dozen pages open (full of thousands of images) I reorganised the lot into sessions so I only have one session open at a time. Thing is, if I clear and load sessions to switch between them, the 'session weight' only increases as if it's added onto the previous session weight rather than being cleared.

Essentially, what should I be doing to open and close sessions properly, so that every time one gets closed and a new one gets opened, the session weight resets and adjusts accordingly rather than accumulates?

>>14898

Actually it readjusted after a bit of time. Might assume it was always accurate but could take a couple of minutes to reflect the new session weight accurately.

>>14899

Sorry just got one more question. After creating a bunch of pages for subscriptions in one session, is there a way to prevent those subscriptions from creating new pages in other sessions whenever they update? At the moment this is what's happening:

>Create session (only intended to contain pages of certain subscriptions)

>Create said subscriptions in the subscription manager

>Wait for the subscriptions to create a bunch of pages in the session

>Close session

>Create a new, 2nd session

>Create subscriptions exclusively intended for the 2nd session by repeating the above steps

But then…

>If I'm in the 2nd session and subscriptions from the 1st session update, a new subscription page intended just for session 1 gets added to session 2, and vice versa

I can see that the subscription manager has a checkbox to disable new files being added to a new page, but just not sure if that would give the intended behaviour and whether that would end up applying to both sessions resulting in no new files being added to either session. Could someone please clarify how to go about it? I'll shut up after this.

>>14900

As far as i know if you save a session and then create a new one, the previous session won't be updated, only the current one. The create pages for new files option will also only update the current session.

Can someone please explain the warosu thread watcher to me? I can't get it to run

>>14904 This is me and I am an idiot:

>>14876 was the solution. Thanks and sorry for not reading more thoroughly

Is there a way to export some or all of my client settings to another client?

>>14908

No, it seems settings are stored in the database itself and you can't export it

In v414, you mentioned…

>in the manage tags dialog (and some other places), tags with parent implications now show a '(x parents)' after their label, much like the 'will display as' sibling suffix. I do not like this, but I ran out of time. I hope to add a more advanced actual listing of virtual tags with a nice 'ghostly' colour or similar in future

I just wanted to ask how high up on your list of things to do this is? I'm sure there's a million other things more important, but for me personally this is a big one

I have a vast amount of parents/siblings and do a lot of manual tagging, so seeing "(9 parents)" doesn't help me very much and it leaves me to guess what has and hasn't been tagged yet. If I want to check, I have to exit the tag editor and look at the resulting tag list before going back to editing the tags

Having "ghostly"-coloured tags for parents (and possibly siblings) would be wonderful

how do I download files with tag "my:upvotes" from derpibooru?

I send cookie via companion, but I still cant download it.

>>14919

I think you'll have to include your API key with the request.

iirc, edit your URL class and GUG to include ?key=API_KEY

https://derpibooru.org/pages/api

i cant use the backup feature because i did some database migrations. is there anything i can do to make backup possible again?

How can i enable the tags on furaffinity?

>>14946

I managed to enable them, I know that they are shit lol, now I'd like to ask if someone knows how to enable tags on Deviantart.

If I have something like a short two page comic that I want to group together, how should I go about doing that? I previously just gave them a title and page numbers, but would it be ore efficent to just tag them as alternates?

Are there any ways the Hydrus "team" could make updating a bit more foolproof?

Currently, it is not recommended upgrading past 10 versions.

I was thinking, could there not be some sort of implementation that instead of allowing a user to upgrade 10 versions apart, it does it incrementally automatically? For example, if a user is on version 10, and they upgrade to version 20, it instead doesn't try to directly upgrade to version 20, but instead upgrades to version 15, then does 20. This would be very nice, to make it a bit more foolproof.

I know it gives the warning, and says not to, however it would be nice if it could be done automatically.

_____

My other suggestion was having a RSS feed on the Hydrus website, that posted suggested updates for big fixes, or major security fixes. The releases usually have massive changelogs, and they get pushed fairly often, so I don't blame people for falling behind on updates. It would be nice to be able to subscribe to a Hydrus feed that gives suggestions on when you really should update Hydrus in the case of major security or bug fixes as these are important.

>>14959

The alternate system isn't developed enough yet to be useful for things like that. You can't sort by alternates.

So for now the best way is still using tags like title and page numbers.

>>14915

This, tagging now is so annoying.

Unfortunately, dev has a tendency to leave things 90% finished…

>>14915

This, tagging now is so annoying.

Unfortunately, dev has a tendency to leave things 95% finished…

Is there an easy way to check for and re-download broken files? Basically I want a button that regenerates all the thumbnails and if a thumbnail fails go to one of the source links and re-download a good copy of that file.

Background: I was scrolling through a page with a couple thousand files, got a couple messages about a thumbnail not generating. It was annoying to find the files in question when only given the on-disk file name and even after finding it on disk actually finding it in hydrus is still a pain because afaik you can't search by hash (?). Then even if you find a broken file in hydrus, if the file is actually messed up there seems to be no way to fix it besides manually downloading the file externally and replacing it on disk. We already check if files are valid and most have a list of sources so this seems like most of the parts are there?

Is there a way to get notifications of rejected petitions? It seems like they act as permanent client-side deletions (or sibling/parent changes) that notify the repository.

I think local deletions should be separate from petitions. If one gets rejected I'd rather stay in-sync with the PTR and resubmit or do a local deletion if necessary.

as of now, can Hydrus only support grouping image sets/manga page by setting the images all as "alternates?" I can't find any other options that to that effect.

Not sure if this deserves to be its own thread so I'll just ask here. When you make the polls about which major jobs you'll work on next, do you allow multi-choice voting?

To make it short, multi-choice voting is better for polls like this, because it lets people express approval for multiple option if they think that more than one of them would be an acceptable winner, and this leads to the winning option naturally being one that more people like overall. I checked the github repo for the major jobs, and I can already see a situation where I'd have to not vote for one of my favorite options present because it's probably going to be less popular, and I'd have to pick one that's more likely to actually win. I won't be able to show my support for both.

poal.me (for example) has an "Allow multiple choices per voter" option, if that's what you use to run the polls. I'm a little new here so I don't know if that is what you use, but it's popular on imageboards.

forgot to clear trash and ended up with no hard drive space when subs were running. Now Im getting

>sqlite3.DatabaseError: database disk image is malformed

There a way to remove the stuff that was downloaded the last 1-2 days to fix this? Cant even open hydrus to delete cus of this message.

>>15021

nm fixed by following instructions in the

>help my db is broke.txt

to sql clone the dbs

Hey fellas. I've lost the drive that my files lived on, and all that remains are the thumbnail files. Is there any quick way to re-download the files via the filenames of the thumbnail files with a gallery download? They seem to be too long to be md5s, which would be an easy job. Thanks, and congratulations on the anniversary!

>>15027

I'm pretty sure that those are the SHA256 hashes of the file they are a thumbnail of (not 100% sure though). I doubt this that would be very useful for re-downloading them. It may be possible to get the URLs for your files that were downloaded within Hydrus if your database is intact, you'll have to ask a database expert on that though.

Enjoy your empty inbox, and consider investing in backups this time :^)

>>15028

Yeah lesson learned. I had a backup, I think, but it may have been kept on the same drive as the collection was. Oops.

Gallery downloads stop downloading if I tab to another page besides gallery in v402. Is there a setting to change this or am i missing somethin?

>>15034

Nm it looks like Im getting bandwidth free in x hours which probably has something to do with it…

Also getting bandwidth free in x hours every time I come back. Doesnt make sense becaus default bandwidth rule is not exceeded for any of the network contexts.

>>15035

ok I wasted a ridiculous amount of time on this bcuz I didnt remember to set new bandwidth rules for the domain too, not just the 4-5 default contexts or whatever. Had to double click the domain name in the bandwidth manager to get the menu

A lot of downloaders suddenly broke for me. For one site the gallery downloader works but it gives an instant "Connection refused" error from urllib3 when a file is attempted to be downloaded (the URL of which opens fine in a web browser), for others even the gallery is refused.

Some don't even seem to be trying to download anything at all, not even enough to give an error on (maybe Hydrus automatically stopped trying after too many errors, though a few of these were sites I've never used).

A few downloaders still work perfectly fine, though.

If it matters, here's all that I could test.

>everything working: lolibooru, pixiv tag search

>only gallery working: sankaku

>nothing working: danbooru, gelbooru, konachan, realbooru, e621

Anyone know what could be going on?

How do I tell Hydrus which font to use? Is qt making the decision?

>>15040

You can set fonts via qt stylesheet's located in your root hydrus directory in static/qss

Is there a script to use or fix the blacklist for gelbooru when using the downloader? For some reason hydrus ignores my blacklist over on gelbooru and continues to download the stuff I don't want.

Is it possible to skip ahead or speed up videos in the media viewer? Ideally, I'd like to be able to add/change shortcuts for it, but it doesn't look like that's possible at the moment.

Is it possible to have the database itself on one drive, and the client_files themselves on another?

I fucked up my duplicates. is there any way to reset both applied duplicate pairs and potential duplicate pairs?

oh also what happens to the files that are deemed worse or equal? the help says it is kept in a duplicate file group but can it still be found in searches and is it still on my computer?

>>15053

Make sure you have your tag_blacklist cookie from gelbooru in hydrus.

>>15058

Not currently

>>15063

Yes, see https://hydrusnetwork.github.io/hydrus/help/database_migration.html

>>15064

You can reset all potential duplicates if you click the gear icon in the duplicates tab. I am not sure if there is an easier way to reset applied duplicates but you can select the files then right click and reset them under the manage>file relationships menu.

>>15065

Worse are moved to trash, equal are both kept. Files in trash do not come up in normal searches and by default are removed from the system periodically.

>>15066

>Yes, see https://hydrusnetwork.github.io/hydrus/help/database_migration.html

thanks

god fuck it's so much faster now

>>15069

> I did import the new header from here due to not being able to see explicit tags.

That may be the case, try removing the header for gelbooru and adding the fringeBenefits=yup cookie into the cookies menu for gelbooru.

I'm not sure if this is the right place to ask, but I can't watch or download from 8chan. I keep getting 404 or not found. Is there something I'm missing?

Is there any way of making downloading from twitter practical? The downloader keeps dropping the connection with 404 results when getting galleries and this makes the downloader think that it's done with the job. This gives me incomplete gallery downloads. Is there a bypass/fix to make the downloader at least fail with another status code besides DONE so I don't accidentally think it has actually downloaded everything?

is there a way to download my private pixiv bookmarks? I used the downloader and it grabbed my public bookmarks, but some of my older bookmarks are marked private

>>15071

I literally just came here to ask the same thing. If you find a workaround, please post it here.

>>14731

Is there any way to force hydrus to import one picture at a time in order? That way a set of images can be sorted by import date. Any other way to achieve the same? Now, let's say I have three sets of images that I import at one time, it might import, let's say second set's first picture before first set's last. I could add page numbers via regex, but I think that's a bit clunky way to achieve this. Do I make any sense? Thanks.

>>15070

Dumb question but which or where is the header?

>>15103

network>data>manage http headers

>>15115

are you running from source? you need python-mpv.

>>15119

in the tag manage menu there will be multiple tabs, tags will only be added to the public tag repo if you enter tags in that tab. So just don't add those tags in that tab.

>>15122

That's just gelbooru changing the site and breaking the parser, see >>15118

Hey, I am sorry, the holiday ended up killing my schedule, and I couldn't give this thread proper attention.

8kun are now saying the expect to go off the clearnet in the near future and go TOR only. I have been wanting to move to a different board for a while, I was just procrastinating, so this seems like the time to do it. Endchan will be our primary board for the time being. I am going to clear this thread of newer posts, make an archive of it to post on Endchan, and then make a sticky announcing I will delete the board next week.

>>14908

>>14914

Not yet. In future I want better options and other portability.

>>14915

>>14977

>>14978

Sorry for this. This last 10% ended up needing more additional work to get my taglist supporting the additional rows to show parents, which was more complicated than I thought.

Several jobs are waiting on taglist presentation improvements, including tree-like [+] buttons to collapse/expand when sorting by namespaces. I am not sure when I will find the time, but it is still on my list.

>>14921

I am sorry, my internal backup solution is not clever enough to deal with a migrated database. I have a job to make this more clear in the UI and help. Since you are an advanced user, please set up a backup routine with an external program that can do the job better than I can. I recommend FreeFileSync:

https://hydrusnetwork.github.io/hydrus/help/getting_started_installing.html#backing_up

>>14935

No way at the moment, it is just a patch for advanced users. Reducing CPU overhead, particularly with large downloaders in the current session, is the next primary optimisation job.

>>14946

>>14957

I think the default DA parser does not pull artist tags because they are so non-useful and I don't want new users to get them. I know nothing about this one, but maybe it gets what you want?

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/blob/master/Downloaders/deviant%20art%20page%20parsers%20-%20fetch%20tags%20-%202020-07-18.png

>>14968

Unfortunately I am the only person working on hydrus, as I have acute difficulty working with others. Since I am always short on time, and I am not always as professional as I hope to be, I cannot guarantee nice user experience for certain parts of the program.

>>15008

I think database->maintenance->files->manage->add new work->if file is missing/incorrect and has url, move file out and try to redownload is the basic route you want. This assumes the files have actually been damaged since import, though (e.g. hard drive damage), rather than them once being importable, but technically no longer allowed with stricter rules now about how broken a jpeg will be accepted into hydrus.

Try adding a 'my tags' tag to those files you saw and adding that file maintenance job just to a search for that tag. It might fix them, it might not. I had to turn up the safety limiters on truncated jpegs about 6-12 months ago because it was crashing some clients on really broken files.

>>15012

Not yet. I don't think I will be able to deliver it this round, but I want to begin the framework for this in the network improvements I am starting properly this week. The PTR jannies want to give reasons back for denied petitions too.

>>15017

Not yet. When I get to alternates improvement, you will. It came #3 in the latest big jobs poll, so I think I should get to it within 12-15 months.

>>15027

>>15028

>>15032

If it helps, if you still have this problem at all, you want the same file maintenance job that I just said in the post above. >>15128

>>15029

I am sorry for the trouble here. The blacklist should work on the tags parsed from the page. So if you tell it to stop 'guro', then any page with that tag should get the 'ignored' status with an appropriate note in a downloader using those 'tag import options'.

One user has been telling me about this not working for him, but I couldn't figure it out. Can you give a search example on danbooru and a blacklisted tag that aren't applying for you?

>>15039

I am sorry, I do not know what is going on here. Sounds like it could be an ssl issue or similar, maybe if your OS is old, although I'd guess that sort of thing would be more likely right on Jan 1st 2021 as some certificate timeout ticked over.

If you can paste the whole error, I can have a closer look. If it is moaning about ssl certificates, one way around it is to uncheck the 'verify regular https traffic' option under options->connection.

>>15071

>>15089

If you mean 8chan.moe, I believe they did some direct linking referral trickery on their spicier boards to make sure direct links from external sites don't work. There may be a solution here with some forced hydrus referral urls on the URL Classes, but I do not know the details and have not looked into it myself. I think you'll have to browse these boards manually for now.

>>15074

Any site that doesn't operate like a normal booru is tricky. Sometimes you just have to babysit them.

I assume you are downloading through nitter, right? I think the twitter API that nitter hooks into does not deliver tweets beyond about 3,000 tweets into the past, so if you are hitting that limit somehow, or if nitter has its own limit, that may explain it.

I have been getting reports recently that nitter has been giving '429 - server rources limited' or similar recently as well. This may be related, I am not sure.

>>15088

I am not sure if the existing pixiv login script works right now, it has always been temperamental. If not, I think you'll have to use Hydrus Companion or similar to log in by copying your browser cookies to hydrus. Then pixiv will think your hydrus is you and it'll have access to private stuff.

>>15095

Support here is not yet great. If you are importing from hard drive, they should import in filename order. If you are importing from a site, it will come in the order the site's parser pulls the URLs, which is usually good, sometimes reversed. The main problem is hydrus currently has integer timestamp for import time, so if two files are imported in the same second, they will sort randomly when you sort by import time. If this is what is hitting you, I apologise. At some point there will be a conversion to sub-second times, although a retroactive fix would likely be impossible.

I am hoping the eventual file alternates work will provide a nicer system than page tags. I agree they are clunky.

>>15115

>>15116

This is some developer bullshit unfortunately. You don't need mpv, but 'libmpv1'. You should be able to get it with apt.

This thread is now archived at https://archive.is/0WoQk . I will not post here any more, as the board will be deleted next Wednesday. Please move to the Endchan thread at https://endchan.org/hydrus/res/10.html , thank you!

The plan to move to Endchan only has changed!

Codexx on 8chan.moe has kindly offered to host me on /t/ over there. As I have never been a comfortable BO, this is a much better solution for me as the primary place to talk Anon about hydrus.

I will be maintaining a Hydrus Network General, the first instance is here: https://8chan.moe/t/res/2219.html

The links in the release tomorrow will reflect this. Endchan will remain as a bunker.