/hydrus/ - Hydrus Network

>>2738

That's a bit of a difficult question. In order to quickly query the ptr to figure out which of its mappings should apply to your files, we would have to store them in efficiently indexed database tables. Once that's done, we've basically reinvented the hydrus db.

The client never transmits which files you have to the server (to keep them private and minimise server CPU), so filtering can't happen before you download.

If your concern is that you don't want to see all the remote files' tag results in the manage tags dialog, check out file->options->tags->by default, search non-local tags in write-autocomplete, which will set the autocomplete dropdowns in dialogs to 'local files' by default, instead of 'all known files'.

If your concern is hard disk space, you could probably fudge something like what you want by using a hydrus tag archive snapshot of the PTR. Something like:

Get the above download, run it.

Create an HTA from its PTR service.

Import the HTA into your other client's 'local tags' service, which will only import for files in 'local files'.

Let me know if you would like help doing HTA stuff, as it is slightly advanced.

>>2747

It isn't too difficult, but HTAs are powerful, so make sure you have a backup somewhere in case it goes wrong or you don't like the result. The main dialog you want is under services->review services->your tag service->perform a service-wide operation. It's an ugly debug-tier dialog, but it can do a couple of neat things, including exporting all a service's mappings to an external HTA db file or importing back from one. I think the import action now includes a 'do you want to sync with all files or only local files?' option dialog–you only want local files.

If you don't want to wait for the PTR to sync on your existing client to create the HTA, you can use the above download on a fresh extract of the client to get it pre-processed.

>>2790

Vacuuming is pretty slow, unfortunately. Make sure you defrag your disk regularly, is the best thing you can do. You can change how often it runs under file->options->maintenance and processing, and even turn it off if you like. It trims free space and makes your db run a little faster, but it isn't essential.

Yes, keep client_updates and everything else in the db folder. If you remove anything, things are likely to break when you next boot the client.

I'm not sure if I should ask this in this thread:

I tried importing gelbooru's tags using the gelbooru2016.db kindly provided in the previous PTR thread. It seems to be working well, however, I get this entry in my menu, and I'm not sure what it means (1st pic related).

If I try to click commit, I get "No access key" error. Which I think it's expected since it's not an online connection, right? I wasn't able to find this issue on the help, so excuse me if there's an obvious explanation. Could I have configured the DB import incorrectly (2nd pic)?

Thanks in advance and for your work on hydrus

>>2941

You have set up the HTA import fine, but I think you have misunderstood where you are sending the tags. In that screenshot, you have set up a connection to a hydrus tag repository, which would be a tag service on another computer. The space for 'credentials', where one would normally put something like:

4a285629721ca442541ef2c15ea17d1f7f7578b0c3f4f5f2a05f8f0ab297786f@hydrus.no-ip.org:45871

Is left as default. When you tell the client to commit those pending tags, it goes 'ok, I'll send them to that repository at 'hostname:45871', which is not a valid address.

If you meant to send those gelbooru tags to your 'local tags' service (i.e. straight to your hard drive, rather than a repository somewhere on the internet), then delete that tag repo and set up the HTA import again on the 'local tags' service, which is also on the manage services dialog, but under the 'local' tab at the top. Since 'local tags' goes straight to your hard drive, there's no pending/commit queue–they'll apply instantly from the HTA as appropriate.

>>3094

Unfortunately, this is essentially the crux of the problem–there isn't an easy generalised way to ask boorus for the tags for a particular file.

You can try manually redownloading the files through my booru downloader by searching for their artist and setting 'get tags even if file is already in db' to true (with the files being new, they will be near the front of any search you perform), but the best way is probably just to sync with my public tag repo, as some other user will probably do that download sometime in the near future and you'll get the tags synced automatically.

And here's another!

This is version 214 and ptr update 1455, which is yesterday, July 16th. It now has 73 million mappings.

http://www.mediafire.com/download/xf8pm738a2ucffi/Hydrus_Network_Client_bare_PTR_database_v214_update_1455.7z

>>3449

I meant the one in the hydrus client. Hit F9 to bring up the new page chooser, then go download->gallery->booru.

>>3928

I've got it currently scheduled for every 10 weeks, which by my to-do means 2016-09-24 will be the next, for v224.

Here is the next!

This is version 224 and ptr update 1515, which is yesterday, September 24th. It now has 88 million mappings.

http://www.mediafire.com/download/l9aecivwowa3yd6/Hydrus_Network_Client_bare_PTR_database_v224_update_1515.7z

>>4027

Yes, but only if you are starting a new client. This is an empty database that is synced to the PTR. If you place this in a fresh extract of the client, you'll be off to a running start.

>>4388

Nothing special–I made a fresh client on my dev machine a while ago and synced it with the PTR. When I want to update this bare db, I update the client to the latest version and catch up the PTR sync and simply 7zip the new db. You are basically just continuing using the client I set up.

If you have a similar need, whether to replicate an existing client or to create a new kind of bare db for another computer or a friend, you can do the same yourself. I made the db very portable and easily migratable for exactly these sorts of things. Let me know if you need any help with something you want to do.

>>4413

I prefer to do it one at a time, as I can choose randomly from a simple system:limit=256 page that samples my larger collection. My stuff remains mostly unfiltered and untagged, so searching for artists or eras or 'is good' isn't something I can do yet. A number of my favourites are on the example read-only file repository I set up though, which you can connect to following this:

http://hydrusnetwork.github.io/hydrus/help/access_keys.html

If you want way more high res art than you can handle, there's plenty in torrents, which is where I got much of mine from. The key is to search for 'painting' instead of 'art', which will turn up a load of unrelated porn and so on. 'museum painting' and '(era/artist) painting' are usually winners.

Mg-Renders.net HTA

https://mega.nz/#!qNIHBYpB!emc8LF38Kl5p-VRSgZ7wI-7kFD83pwhY3JaQcB7W5Gs

And for a limited time, mg-renders siterip.(14GB, 6.5k files)

https://sukebei.nyaa.se/?page=view&tid=2186495

Here is the next!

This is version 234 and ptr update 1578, which is yesterday, December 6th. It now has 102 million mappings.

http://www.mediafire.com/file/ab9yn50hkifzz8s/Hydrus_Network_Client_bare_PTR_database_v234_update_1578.7z

>>4722

Hydrus Tag Archive.

They're sqlite databases containing hash tag mappings to all the files on the site.

Of course, new files get added to Gelbooru all the time, so in order to stay updated, the HTA's have to be updated too. I'll try to update them yearly. I could always add my updated tag archives to the PTR. This way everyone sync'd to the PTR would receive the updates as well, but I mostly prefer to leave that option up to the users. It also solves the case where some users may not want to use the PTR, but may only prefer a specific siterip.

>What's the difference from the old tag archives?

Just more tag mappings.

In Gelbooru's case, I added the 'source' namespace for whatever reasons.

In Rule34Hentai's case, I fixed a major issue in the old one where tags were skipped if the tag had an apostrophe in it.(thanks to a user for pointing it out) I added a tag to better differentiate rating:e content. Lastly, I combined Gelbooru's namespace with Rule34Hentai. R34 does not use namespaces, which means there aren't any character:/series:/etc tags. I used a simple script to see if any of the tag names matched ones from gelbooru, and thus used gelbooru tags if any matched. Now a good majority of tags from rule34hentai have proper namespaces.

To sum it up;

Any file from gelbooru as of 12/13/16 will be tagged if synced.

Any file from rule34hentai as of 12/12/16 will be tagged if synced.

Now the bandwidth has been cut and update files are portable and processing time is so much quicker, I am not sure whether to continue putting out these bare dbs.

I may instead put out updated versions of the PTR update files all zipped up, like this that I mentioned in v245 release post:

http://www.mediafire.com/file/1s1ee30fk27ejc9/ptr_update_files.7z

You can import these files into a fresh client via services->import update files and then add the PTR in manage services and you won't have to download them all from my server. This could save you bandwidth if you want to set up multiple clients that all sync to the PTR, although you can also recreate your own directory of update files by exporting them yourself from review services.

So I don't know. It may be that processing is now good enough to retire putting out any sort of expedited install package. If you have thoughts, I'm open.