[ / / / / / / / / / / / / / ] [ dir / random / 93 / biohzrd / hkacade / hkpnd / tct / utd / uy / yebalnia ]

/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Name
Email
Subject
REC
STOP
Comment *
File
Password (Randomized for file and post deletion; you may also set your own.)
Archive
* = required field[▶Show post options & limits]
Confused? See the FAQ.
Embed
(replaces files and can be used instead)
Options

Allowed file types:jpg, jpeg, gif, png, webp,webm, mp4, mov, swf, pdf
Max filesize is16 MB.
Max image dimensions are15000 x15000.
You may upload5 per post.


This board will be deleted next Wednesday. I am moving to a General on 8chan.moe /t/. This board is archived at 8chan.moe /hydrus/!

File: b28babaa13b7dbf⋯.png (765.02 KB,1278x720,71:40,b28babaa13b7dbfb92d271ee0a….png)

2f807c No.12641 [Last50 Posts]

Here you can ask questions so that the board is not clogged with small threads.

Old thread >>9327

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8dac8a No.12643

Is there a good place to discuss tagging standards for the PTR? There has been plenty of talk on what tags and namespaces we should use, and I think there should be more. Maybe it's worth its own thread.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

71077a No.12644

>>12638

Siblings are useful, but being able to right-click a tag in the manage tags window and select "replace tag with…" which lets you type in a tag to replace it with would be great. This would only affect the selected files that have that tag. I might still want to keep that tag on other files I haven't selected, so a sibling isn't helpful. I could manually remove the tag from the selection, but then I can only add the new tag to all of them.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

17913a No.12646

i'm working on tagging approx. 18k files right now with rating tags, i.e. rating:explicit, rating:questionable, rating:safe. i'm trying to figure out if there's an effective way to do this automatically.

I've set up obvious tags as parents for rating:explicit, so explicit is applied automatically with practically no problem. is there a way to say to the client "apply tag rating;questionable IF file does not have rating:explicit AND file has tags (tag1 OR tag2 OR tag3)"

the reason I want to do this is because if i set a parent relationship for let's say tag breasts->rating:questionable, then i'm going to end up with a load of files with both rating:explicit and rating:questionable since the parent relationship does not check for the rating:explicit tag.

also, mr dev man, i love your program. thanks a ton

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a49c88 No.12647

>>12632

>sounds like a very subjective description of namespaces to me.

Tagging is entirely subjective to begin with.

Programmatically, namespaces are used for disambiguation to prevent naming collisions. What this means for tagging is that in practice you should have namespace_A:same_character_name and namespace_B:same_character_name to disambiguate two characters. People do shit like "character:Yuno (Mirai Nikki)" and "character:Yuno (Black Clover)", cluttering the tags with two variants of Yuno instead of providing a namespace for each of them (eg: "black clover:yuno" "mirai nikki:yuno"). This is probably done to avoid cluttering namespaces with is often its own namespace (series: or copyright:). In practice, almost nobody uses namespaces solely for disambiguation. They use it for meta information like "series:" or "artist:".

>not sure what the difference to you is between having two namespaced tags "hair:short","hair:long" and having two regular tags "short hair","long hair". Either way you have two tags.

I explained the difference. Information about the image (meta) vs information that describes what the image is (descriptors). This is also how every booru I know of works and how I choose to tag. Boorus will use the tags "black panties" and "black dress" and not namespace them under "black:" like "black:panties" and "black:dress". Mixing descriptors with meta information is messy as many things can be swapped with their namespace and still make sense. For example, which is more valid: "dress:black"/"dress color:black" or "black:dress"? Why would you namespace one and not the other? Would you use both namespaces?

"hair:ponytail hair:ribbon hair:black hair:short" is clutter and even if you break each "hair:" down into more specialized namespaces like "hairstyle, hairaccessory, haircolor, hairlength" you only worsen the clutter.

Compare: "ponytail ribbon black_hair short_hair" vs "hair_style:ponytail hair_accessory:ribbon hair_color:black hairlength:short"

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

146181 No.12650

>>12647

Character: Yuno Gasai is different to Character: Yuno, the Meal-Saving Prince. Clutter is a buzzword to excuse minute and less important tags from being tagged on images. A dress being black doesn't mean it's not in the style of Dior's iconic Little Black Dress, and so a Dress:Black and Black:Dress can be applied to the same image for the same garment and mean two completely different things. If a user believes that Hairstyle:, HairLength: or HairColour: tags are too specific, they can simply not use them. Hydrus allows certain namespaces that the user deems unimportant or uninteresting to be left out when viewing a certain tag or groups of tags. It's called Censoring. Hydrus does NOT allow for tags to appear on images where there are none. Therefore, it is the duty of any user of the PTR to overtag images and allow the user to pick and choose which tags or namespaces they see fit to read or discard according to their preferences.

All of this relies on people actually using namespaces, though, which people still don't.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a49c88 No.12652

>>12650

>Character: Yuno Gasai is different to Character: Yuno, the Meal-Saving Prince.

No shit, that's the point of differentiating them. I excluded last names for the sake of example but had a feeling you'd be insufferable enough to ignore the point. So here's a more solid example of two people with the same first and family names.

Michelle Williams (All the Money in the World) and Michelle Williams (Destiny's Child)

Both would be tagged "Michelle Williams". This is what namespaces are *meant* to resolve and where the name "namespace" comes from as a programming term. You'd provide a namespace for each Michelle Williams as a way to disambiguate between two otherwise identical names.

>Clutter is a buzzword to excuse minute and less important tags from being tagged on images.

If you noticed, I didn't remove any "less important tags". I simply didn't give them redundant namespaces that provide no additional value over a plain tag and only serve to add clutter the search terminology.

>Therefore, it is the duty of any user of the PTR to overtag images and allow the user to pick and choose which tags or namespaces they see fit to read or discard according to their preferences.

I don't use the PTR and am not speaking about public choose-your-own-tags-and-censor-stupid-tags group tagging. Most people are retarded when it comes to tagging things which is why once boorus grow past a point their tag quality goes downhill fast with very poorly or even misapplied tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8dac8a No.12658

>>12652

>which is more valid: "dress:black"/"dress color:black" or "black:dress"?

If I wanted to namespace this tag (I don't), "dress:black" makes the most sense. "dress color" is unnecessarily specific, leading to more namespaces than needed.

However, if I cared for searching by/adding/removing dress colours often, then I could more easily find all dress colours in a long list of tags if there was a "dress" namespace. This applies to a list of tags on an image, as well as search results for tags having entered only part of the query (e.g. just the namespace, or just the colour).

This philosophy works well with Hydrus, and does not increase the number of tags. More namespaces are introduced, but only for what you are interested in.

The only problem is that one has to add local siblings for all tags they wish to namespace, and possibly add the reverse siblings to the PTR if they want to easily mass upload tags.

>had a feeling you'd be insufferable

Now, now

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a49c88 No.12661

>>12658

>If I wanted to namespace this tag (I don't), "dress:black" makes the most sense. "dress color" is unnecessarily specific, leading to more namespaces than needed.

Not saying you wanted to use these namespaces - but I was pointing out that black:dress and dress:black are both equally valid ways of tagging the same thing using namespaces. "dress color:black" was just used as a more verbose "dress:black", the comparison was more between "dress:black" and "black:dress". Further examples of these namespaces, you might have "dress:wedding" and "black:shoes".

>However, if I cared for searching by/adding/removing dress colours often, then I could more easily find all dress colours in a long list of tags if there was a "dress" namespace.

I don't believe this is a common enough scenario or a significant enough increase in quality of life from scanning an alphabetical listing of tags for "color_dress". It takes maybe an additional second, if even that, to see if an image is tagged "red_dress blue_dress black_dress" instead of "dress:black dress:blue dress:red" even on my images with ~70 tags. It provides that small amount of QOL with the tradeoff of either increasing the complexity of the tagging system (rules about what belongs in the "dress:" namespace) or increased verbosity or redundancy when searching.

Complexity: "hair:long black_hair ponytail ribbon (Complexity is defined as having to clarify "Which values belongs in the 'hair:' namespace?")

Redundancy/Clutter: "hair:long hair:black hair:ponytail hair:ribbon" (Reduced complexity, "All hair related values belong in the hair namespace")

Verbosity: "hairlength:long haircolor:black hairstyle:ponytail accessory:ribbon" (Removed redundancy and reduced complexity by being more verbose)

No complexity, no redundancy, no verbosity: "long_hair black_hair ponytail ribbon"

>…as well as search results for tags having entered only part of the query (e.g. just the namespace, or just the colour).

"hair:" returns "hair:long" and "hair:black" (and all the other length/color in the hair: namespace) and "*_hair" returns "long_hair" and "black_hair" (and all the other length/colors tagged as adjective_hair).

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

923d52 No.12663

>>12643

>Maybe it's worth its own thread.

Sounds good to me, it's a bit messy to mix it in with Q&A

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

69bc83 No.12666

How do you get Hydrus to download Explicit tags on rule34hentai where you need an account to see? The Hydrus login doesn't do anything and Hydrus still skips these when downloading from that site.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

69bc83 No.12667

How do do you undo an imported downloader from an png? I tried importing a downloader from an png and I don't really like how its getting tags so I want to go back to its default settings.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7fa1bd No.12668

>>12667

Is namespaces the only way to sort tags?

Is it possible to add them to sorting list to have them presented in a way that looks logical to humans?

If it was possible and if coloring a tag was moved from namespaces to these list, I don't think we would need so many namespace wich essentally multiply the tag in the database.

dog and species dog both exist.

This may be stupid, though, I'm no programmer.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a49c88 No.12671

File: ba02e151b04e6f6⋯.png (6.15 KB,436x147,436:147,1558575951.png)

>>12668

These are the tag sort options. Currently there are no ways to manually give sequencing or priority to individual tags. Also only namespaces can be colored. There is not currently a way to override "unnamespaced tags" to be colored on a tag level, to my knowledge, but it is probably something that is capable of being added.

May I ask why you feel that dog and species:dog are redundant tags? Personally I use "dog" for any actual dogs and "species:dog" for anthropomorphic characters. For example, I don't have a "catgirl" tag, I use "species:cat" while a pet cat would simply be "cat".

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bc2b78 No.12673

Has the Linux version of Hydrus gotten any more stable? How do I migrate my DB from windon't to linux?

Will it ever be as stable, or will this always be a primarily windows-focused application?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cc18da No.12674

>>12673

Im running it from source, so far only once hydrus crashed.

>How do I migrate my DB from windon't to linux?

Same as in Windows.

>Will it ever be as stable, or will this always be a primarily windows-focused application?

idk seems pretty stable to me.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2ede7d No.12676

My database sees only 21 images out of few 1000. Even though they are there. Everything else is considered orphans. Am I fucked?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c9523f No.12684

>>12674

That is nice to hear, last time I tried it on ubuntu it would crash constantly.

Maybe there was an error in my setup.

I'll try it soon!

Do you think there is any way to easily remove the more smutty compnents of the program (explicit references to hentaihaven/gelbooru etc.) becuase I want to use it in a way where it'll be relevant to cataloguing photos of family and friends? (i.e.: father, mother, brothers will see the program).

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

eed651 No.12687

How come the installer fails everytime I try to download it?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7fa1bd No.12688

>>12671

I used that example because the tag "dog" is aliased to "species:dog" in the ptr.

I too would prefer it not to be a defacto alias.

I wager you either don't use the ptr or found a way to blacklist alias locally?

I wonder if the way we use namespace is necessitated by boorus compatibility rather than being the most efficient way to sort tags.

Using a list system would however requires more user involvement though, which doesn't seems a good idea for such numbers of files.

In the same vein I wonder if affix and suffix are possible in lieu of those qualifier, by which I means linking up to tags togethers.

"character name" is affixed to "series it come from".

It's 100% not doable now but may be technically possible, perhaps.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

21e517 No.12689

I'm getting a filenotfound error even though the file exists on my system, not sure why this is happening? db diagnostics say nothing is wrong

I created the db on MacOS and have since imported to Windows, exact same db works on mac but fails on windows… did I miss something during my migration? (db was fully portable)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a49c88 No.12690

>>12688

I don't use the PTR because people tag like retards, so you're right. :)

>I wonder if the way we use namespace is necessitated by boorus compatibility rather than being the most efficient way to sort tags

As I explained somewhere, namespaces are used to differentiate two things that have the same name. It comes from programming, especially object orientated programming, where you'll have many objects that may share the same name but aren't the same thing. For example "second" and "second". "time:second" and "position:second" provides disambiguation. ( See: https://stackoverflow.com/questions/4211827/why-and-how-should-i-use-namespaces-in-c )

People don't use namespaces properly and end up with "(clarifications)" in their tags which, in my opinion, is quite ugly. Because disambiguation is rarely needed, it's boorus often use it for *meta* information like who drew the image or what the image is a drawing of rather than what it contains.

>"character name" is affixed to "series it come from".

This is already double by making use of child-parent relationships and if you don't tag like a retard. Unless I'm misunderstanding you. :)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1f9246 No.12696

I'm still kinda new to all this, but im wondering why theres a image limit on the twitter downloader? It only seems to stop at a point. Also will it ever be possible to download gifs and videos off from twitter?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8dac8a No.12700

Has anyone ever noticed booru subscriptions consistently failing to pull certain tags? I am suspecting that any tags which are replaced by a local sibling are not getting added to the image at all.

For me, this is a Sankaku subscription. I'll look into it a bit more.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

06eef5 No.12721

Where can I find other downloaders? Or are all of them already installed?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9bf8dc No.12727

>>12684

>Do you think there is any way to easily remove the more smutty compnents of the program

At the very least you can hide them from view from network -> downloaders -> manage downloader and url display. I think you may be able to entirely remove references to them through network -> downloader definitions and clicking on each submenu and deleting everything in there.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c74f94 No.12731

File: f7ebb1b57225a86⋯.png (208.86 KB,927x794,927:794,2019-05-27-111720_2560x144….png)

Hydrus has every feature I need, but the major deal breaker was that it wants to copy everything to it's own folder, when I rather it, like music libraries + players, just to monitor a list of directories.

Is this still the case? If so, can there be changes where copying "imported" files to its own directory is entirely optional?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

38f133 No.12732

>>12731

>If so, can there be changes where copying "imported" files to its own directory is entirely optional?

Yes, but that would fuck with the entire program. Hydrus depends on files being in it's own folders in order for half of its features to work.

There might be some workaround in the future but no plans yet AFAIK.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c74f94 No.12733

>>12732

The only thing I really want is some aggregate, search, catalog, tagging features. Should I be looking for another (simpler) program?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

af4ce4 No.12734

Not quite sure how to ask this, but is there a way to purge all PTR data from my db?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6f816a No.12736

>>12684

>That is nice to hear, last time I tried it on ubuntu it would crash constantly

Sorry for late reply, DO NOT USE THE PORTABLE VERSION, RUN IT FROM SOURCE.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.12737

I could swear I asked this already but maybe it didn't send as I can't find it.

Following scenario:

One PC acts as a local server, nine others use that local connection to access files on the server. Would it be possible to put the DB on the server and access it from all other PCs?

What happens if several clients try to change things?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8dac8a No.12739

>>12734

have you deleted the ptr from the services menu?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

edfbea No.12740

File: 56150d5bf7aac98⋯.jpg (133.86 KB,850x963,850:963,rainbow_vomit.jpg)

New to Hydrus, feeling like a proper retard. The public tag repo's account referenced here: https://hydrusnetwork.github.io/hydrus/help/access_keys.html seems to be maxed out.

An alternative way to get tags imported seems to be to import a tag archive, BUT it looks that functionality has been removed in the current version?

So how do I get tags? I've searched all around and I can't find a resource explaining how to get tags with the current version.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5b4ce7 No.12742

How should I tag variations/revisions of a illustration?

I found using a combination of page and chapter (what should the chapter be even called?) with collections produces the desired effect, but it seems like a terrible solution.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ad3314 No.12743

>>12740

Just going to have to wait it out. The public tag repo is unaccessable until the bandwidth limit resets at start of next month. In the meantime check out https://cuddlebear92.github.io/Quicksync/#two and use the services -> import repository update files menu to get a head start on updates.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

af4ce4 No.12744

>>12739

That was the first thing I tried, but it didn't seem to do anything. I somehow got client.mappings.db from 15GB to 3.5MB (I think I exported my local tags, deleted client.mappings.db, and re-imported my tags), but client.master.db only dropped from 4GB to 3.5GB after vacuuming

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

38f133 No.12745

File: de9f0f1a4e298a0⋯.jpg (26.61 KB,347x450,347:450,de9f0f1a4e298a01561b7f9354….jpg)

>>12740

Shit's fucked. Like >>12743 said, you can only wait.

>tfw the ever growing tags to petition only makes me more afraid to do one big fuck up like accidentally petitioning all tags to the PTR again and having to erase it all

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b459c1 No.12746

>>12740

Use >>1914 or do the individual file lookup. The problem with that is if the hash doesn't match then you'll get nothing. That's why I run everything through IQDB and use the script to collect it all for import

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9bf8dc No.12750

>>12737

>What happens if several clients try to change things?

I'm not sure which thread it was in (it was sometime within the last month or two) but I'm pretty sure Hydrus Dev said not to do this. Something about SQL not being able to handle multiple programs trying to lock/unlock the db.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12794

>>12644

Nice idea, thank you. I know what you are getting at with the current workflow not being able to do this for a multi-file selection. I think I can do this, but I can't promise it super quickly.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12795

>>12646

Great, I am glad you like it!

Best way to do that for now is with a search page that you hit F5, ctrl+a, F3, ctrl+v, enter, enter, ctrl+r on every week. A search like:

[ (semi-lewd tags), -rating:explicit, -rating:questionable ]

Then hit F5 to refresh and see all the files that apply. ctrl+a, f3 to bring up manage tags. ctrl+v, enter to add 'rating:questionable', then enter to close manage tags and ctrl+r to remove all selected files from view.

There isn't a way to do kind of 'logical' tag siblings/parents that rely on more complicated tag states and implications. I have thought about it several times in a 'wouldn't it be nice to add tag x if both tag y and z were present' way, but that is a lot of power and I can see how it could go extremely wrong if I did not wrap it behind two advanced modes. So I am a little wary of it atm and am ok that human eyes go over complicated jobs like this for now.

rating is a very subjective namespace. I agree that explicit is going to be easier to set up than questionable or safe.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12796

>>12666

I have heard the R34H login script can be finicky, just failing sometimes. If you are absolutely certain you have the credentials set up correct in manage logins dialog, and that the login script is active, try hitting the 'reset login' and 'do login now' buttons on the same dialog, just to retry.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12797

>>12667

Hit the manage dialogs under network->downloader definitions menu and delete the new objects. You are probably talking one GUG, a couple of url classes, and a couple of parsers. If they overwrote old objects, those dialogs should have 'load from default' buttons or similar. Once you are all set up, hit the manage url class links dialog and make sure the 'url classes' you want are connected to correct-sounding parsers. You can click the 'try to link…' button on that dialog to try to auto-fix the links here.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12798

>>12668

I would like future versions of the tagging system to support metadata about tags to permit custom sort orders. Future tag siblings will allow you to better control your various dog/species:dog preferences locally just to your client.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12799

>>12673

If I could wave a magic wand, I'd love for the Linux version to be more stable. I'll be putting some time into improving multiplat stability in about three or four months. Unfortunately I am a Windows guy so just have less experience and time spent on different Linux flavours and window managers.

The stability problems on Linux are mostly to do with my shit UI code interacting with stricter/different window managers that want things in slightly more proper ways. I'll keep pushing on the problem, but I can't promise excellence for all systems in quick time. That said, some Linux users experience no problems at all with source or the built release. I recommend you try it, see if your situation is good enough for you, and plan to be ok with staying on Windows or whatever.

To migrate your db, the basic rule is just to install on Linux and then copy your install_dir/db folder across. Don't do this if you are uncertain–feel free to ask me for more details if you need help. Make a backup beforehand. Background reading here:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12800

>>12676

This sounds like a serious problem. Please check the 'help my db is broke.txt' document in install_dir/db. If the files are legit still in your client_files folder, but just seem to have disappeared from your client's records, there are ways to recover. I am happy to help with all of this, but first you need to make sure your hard drive is healthy, which that document will walk you through. Feel free to send me an email if you want to work more one-on-one with this.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12801

>>12684

>>12727

Yeah, delete the stuff from downloader definitions. Bear in mind this is imageboard-centric software. Anyone who googles 'hydrus network' is going to end up here pretty quick, so I recommend you sit in the cockpit and just drive some slideshows if you don't want to be asked questions about the INTERNET HATE MACHINE.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12802

>>12687

Have you got an anti-virus that doesn't like something in it? This has happened a couple of times before. The 'upnpc' executable file in install_dir/bin are common culprits for false positive detections, and I know some scanners can see inside the archive of the installer exe.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12803

>>12689

Can you say a bit more about your migration situation? If you open up database->db migration, does everything appear to be in the right place there? What's the first two hexadecimal characters of the file's hash, and is it in the right 'fXX' subfolder for that? If the client gives you path details in those FileNotFound errors, do they line up with what you are looking at, or are they old/incorrect to what you expect?

Also, if this is a video, is the actual error about not being able to find ffmpeg, which hydrus uses to render video? Do any videos work? What is ffmpeg's apparent version under help->about?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12804

>>12696

There are two limits with twitter–one is a ~2,800 tweet limit caused by their API, and another is if the user has a series of 20 or so tweets without media content, the downloader bails early.

Twitter videos is tough. twitter deliver videos as streamed mp4s, so no easy URLs for hydrus to latch on to. I know one user is working on this problem. Apparently twitter can give you direct vid URLs from the API, but it takes an API key.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12805

>>12700

Can you check your options->tags for 'Apply all siblings to all services'? If it is checked, would unchecking it help? I expect to extend this system in future so you can control which services can apply to which others, but for now it is all or nothing.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12806

>>12721

Lots here:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/Download%20System

Check the 'all-in-ones' folder for easy-import pngs. Bear in mind the quality of these downloaders varies a lot. Test them a bit to make sure you only get the tags you want, and don't import a thousand all at once or you'll fuck up your UI with gigantic lists of shit you don't care about.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12808

>>12731

Here are my current feelings on 'external file' management:

https://hydrusnetwork.github.io/hydrus/help/faq.html#external_files

It is technically possible, and if it were a simpler problem all around I'd do it, but I just don't think I can put the maintenance time into this sort of system. If you only want easy-external access to a few thousand files, you may be able to cobble together what you want with an 'export folder' that maintains a synced folder of a search and gives the files nice filenames based on tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4 No.12809

File: 25d07823678a708⋯.jpg (125.19 KB,626x850,313:425,25d07823678a708de5fa6cd342….jpg)

>>12806

Nice.

Is there a list of all boorus that allow md5 or some other type of md5 search?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12810

>>12733

ACDSee is good. If you only want to catalogue a few thousand files and external management is critical for you, that might be a better fit.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12811

>>12734

>>12739

>>12744

Unfortunately, my current 'clear orphans' tech doesn't do master definition recycling. I plan to add this, so it will go down eventually, but we just aren't there yet.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12812

>>12737

>>12750

Yeah, there aren't great solutions yet for a single client accessed from multiple locations. I do not recommend you run a client.db from a network location, as the locks the network drive provides are usually fake and not good enough for what SQLite needs to be safe.

Running multiple clients at once from multiple network locations will cause your db damage.

The Client API is likely to open things up here. I know users who are working on phone/tablet/web browser wrappers that present you booru-like interfaces.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12813

>>12740

Yeah, sorry, PTR bandwidth is tight atm. It is now free again but will likely block again for the last week of the month. I have a plan to fix this in 2-3 months. You can just leave it to sync and it will catch up when it can.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12814

>>12809

I don't know. Most boorus share the same software, so if you know Gelbooru 0.2.0 supports it in some way, any booru that runs that should be good. Doing a search for '(booru) advanced search' or '(booru) cheatsheet' can usually get you to a nice page like this:

https://gelbooru.com/index.php?page=help&topic=cheatsheet

https://danbooru.donmai.us/wiki_pages/43049

https://e621.net/help/show/cheatsheet

https://konachan.com/help/cheatsheet

So looks like gelb 0.2.5, danb, e621, and moebooru are good with a flat 'md5:hash' search. I'd bet Gelb 0.2.0 can do it. I am not sure about Gelb 0.1.1.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9bf8dc No.12816

>>12809

Check out the last few posts from the Hydrus Scripts thread:

>>12604

>>12631

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5552df No.12822

Does gallery downloaders check for newly added posts periodically or do I have to create a new query to fetch new posts?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12827

>>12822

The downloader page just does one job once. If you were to manually create a search for the same query you put in before, it would start a new queue starting at the current new beginning. Obviously you do not want to do this yourself every week or so, so if you want to have a downloader that periodically re-checks and keeps up with a particular query for a site, check out 'subscriptions':

https://hydrusnetwork.github.io/hydrus/help/getting_started_subscriptions.html

PROTIP: Don't go fucking nuts with this. Start slow and figure out a workflow that won't nuke your available bandwidth and hard drive space.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e38a70 No.12829

File: 3afc2de946b8f08⋯.jpg (46.53 KB,420x377,420:377,Scr100.jpg)

>>12641

I suppose this happened because of a disconnect, but what do these "lg"s mean?

Some files got corrupted on download? Can I check this somehow or should I do something else to remove "lg"s?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e38a70 No.12830

>>12829

Forgot to mention, It's a danbooru/gelbooru subscription.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4 No.12832

>>12814

Nice, thanks.

>>12816

I already use all of those. I want more tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a11e90 No.12833

>>12829

Ig means ignored, so 27Ig means 27 ignored URLs.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c6c972 No.12834

>>12833

Got it, thanks.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5552df No.12835

File: c8c97e5452c9da7⋯.jpg (2.04 MB,1288x1822,644:911,c8c97e5452c9da7656de4548c9….jpg)

Is there a way to search only images with certain namespaces other than putting a wildcard like "character:*"? It's very slow to search with wildcards. Using only "character:" does not work.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12836

>>12834

Ignored tends to be either 404 or tag blacklisting (say, the file had 'scat', and that is in your blacklist). If you zoom into the individual sub query and order its big file import list by status, you can review the ignored reason in the 'notes' column.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12837

>>12835

Type in 'character:' in the autocomplete entry and wait a moment, and a new special tag should appear in the dropdown list called 'character:*anything*'. This is a special tag you cannot type in manually that uses cleverer db code to speed up the namespace search.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5552df No.12839

>>12837

Nice! Any reason why "character:" doesn't default to this?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12844

>>12839

That's a good idea, thank you. I think it is probably reasonable to say that 'text:' is not a tag people will typically want to attach to anything and search for regularly, so I'll swap in the 'text:*anything*' version whenever this is added.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

21e517 No.12852

>>12803

database -> db migration said everything was in the right place, all the files existed on my system and were in the correct subfolders. Client wouldn't cite a path in the errors, not sure why. ffmpeg is also installed locally + system wide.

I fixed the issue by importing a new file; not sure what this changes but after that I didn't get any more errors. not really sure how to reproduce this, sorry.

for posterity: what i did was move a fully portable ~45gb database from mac to windows, using resilio sync (home version)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

50e20e No.12853

Is there a way to search for files that have mutiple tags in the same namespace?

Main scenario i am thinking is searching for files that have mutiple 'series:' tags for crossover images but dont have a 'crossover' tag.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

973c0a No.12854

i have two files that i want them to appear as a group when i select sort by collection

what do i do?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.12855

Will using socks5 with your application result in the DNS lookup also being routed through the proxy? E.g. querying safebooru will be resolved through the proxy setup in the settings?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.12856

Is there a pane with tags that I can click to add to an image? I hate having to right click>manage>file's tags>type tag name in just to add tags to images in a page. It isn't much better in the media viewer either from what I can tell. It's super inconvenient.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b459c1 No.12857

How would I go about recording a file's original creation/modified time as a tag? Maybe a script to record it in a text file? Like the ones the booru tag parser makes.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.12859

>>12855

Potentially answering my own question:

Looking into the source code on page

https://github.com/hydrusnetwork/hydrus/blob/master/include/ClientNetworkingSessions.py

line 126, it appears you merely set the proxies to be the necessary value. Presuming a sufficiently up-to-date version of Python is installed with requests[socks], just making sure to use socks5h in the manage_options->connections should do it. In short, the user does have power over this.

Consequently, one can easily run hydrus over TOR. I did notice socks5 being quicker than 5a, presumably because naming resolution can be slow. In addition, since you'd be using the proxy's DNS, you can't really cache results. It'd be interesting if you could use your cached DNS results from the host machine, but for new lookups use the proxy. However, that's a separate thing all together.

tl;dr you can easily proxy over tor with hydrus.

If you could confirm BO, I'd be grateful.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.12869

Quick question, should I be seeing saved webms as zips of images in Hydrus? Sort of like how you can unzip an epub and get a bunch of HTML pages. Or, should the saved webms be *.webms and viewable in Hydrus? This has probably been answered at some point or another, so, if so, sorry about that.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

62d3c6 No.12875

This could either go in the bugs or this thread: Does the Hentai Foundry downloader support fetching tags? I can't get it to work with subscriptions, nor with download tabs.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12881

>>12852

Hmm, I am not sure what this was. I wonder perhaps if the OS X legacy db info had slashes in some path instead of backslashes, and one part of the files manager was unhappy with that? It is possible that by looking at your current paths and ok-ing the dialog put those unusual paths through a normalisation routine that cleaned them up for Windows, but it could also be something else.

If you go to install_dir/db and check the relevant 'client - 2019-xx.log' file and scroll to the error time (I think you'll be able to ctrl+f for 'missing' or similar to find it as well) and check the exact path that was logged in the error, is there anything funny about it? Maybe like 'E:\my files/ab\abcde….jpg'? If you have the whole error traceback, that may be helpful to me, but if so make sure you either email it to me directly or redact the path and file hash if you post it publicly.

I'll keep this problem in mind. If you aren't getting it any more, even on the same file, it sounds like it fixed itself. Please let me know if you start getting it again.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12882

>>12853

Not yet. I'd like to add an option namespace component to system:num_tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12883

>>12854

Collection works by namespaces right now. If the files have a shared 'title:' tag, you can collect by 'title' namespace and that'll put them together. If they have page numbers, you can sort by page to get them in the right order.

I overall do not like how paged-content is handled in the client. Page tags are clunky to work with, and grouping many sorts of comic and mini-comic is a pain. Adding cbz support and the associated hydrus database backend to deal with file groups/pools with indices is probably the silver bullet here.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12884

>>12855

>>12859

I know very little about proxies, so I cannot comment very intelligently. I just pass whatever you put in those input fields to requests, and it handles the rest. A bit of searching lead me to this:

https://stackoverflow.com/questions/12601316/how-to-make-python-requests-work-via-socks-proxy

and this:

https://github.com/urllib3/urllib3/blob/a019c5e9e8ac461111552fdf9aa97899977cccbf/urllib3/contrib/socks.py#L141-L166

So yeah, it seems like using 'socks5h://' does DNS through the proxy, but 'socks5://' does not. I'll update the UI on that panel to suggest 'socks4a' and 'socks5h' as well.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12885

>>12856

There are a few tag 'suggestion' columns you can turn on for the manage tags dialog under options->tag suggestions. Favourites/related/recent can be useful depending on your workflow and can be double-clicked (and in the 355 I just put, you can keyboard-shortcut focus to these) rather than having to type every time.

The manage tags dialog that opens from the media viewer is actually a window not a dialog (i.e. it does not prohibit interaction with the parent window while it is open), so you don't have to close and open it again to edit other media. Also, by default, hitting page up/down on an empty tag autocomplete input should send a previous/next command to the media viewer parent, so you don't even have to change focus to tag and move on.

That said, I would like to add some quicker 'remove'-style actions to the normal taglists when looking at thumbs or the media viewer. Something like right-click->remove this tag, which could spawn a little yes/no dialog that would handle the more complicated back-end for you without having to make the whole manage tags dialog.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12886

>>12857

I have a long-time job to cache this information and make it searchable and sortable. I know another user wants this very much, so it will eventually happen.

If you do not want to wait, yeah I think a script would do the job best. Do a stat call on each file and then output to a (hash).jpg.txt file that you could reimport with 'parse neighbouring .txt files' checked. You'd have to reimport these beside your files, which might make for a pain in the ass workflow though. You might be able to wangle it by reimporting from your actual client_files directory, with the .txt files embedded beside, but that is probably asking for trouble. If your collection is relatively small, maybe something like:

system:everything

export (copy) everything to a separate folder

generate your .txts

reimport that folder, setting to delete original files.

It'll crunch that import pretty quick, since it already has all those files.

The files will get your 'creation-date:2015-02-01' whatever.

But that would also be a pain to keep up with. I'll add this eventually (another thing the new file mainteance manager will be good to retroactively fill in) and handle it all automatically, so if you can wait, I'd advise that.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12887

>>12869

Are those from danbooru? They are probably ugoiras imported from Pixiv, which danbooru offers both in native zip format and a converted webm. If you would rather the webm, please hit network->downloader definitions->manage url class links and switch the 'danbooru file page' entry to 'danbooru file page parser - get webm ugoira'.

If you want to reimport the zips you already have as webm, do a search for 'system:has url:danbooru file page' and 'filetype:zip', and then ctrl+a and right-click copy danbooru file page urls, and then queue those up in a url download page with the 'file import options' set to 'do not skip downloading because of known urls'.

Native ugoira support remains a long-term hydrus goal.

If you are getting weird zips from somewhere else, please let me know. If the zip contains a list of jpegs or pngs, it is probably an ugoira.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12888

>>12875

It only does 'creator' and 'title' tags. The first HF parser I made grabbed the artist tags, but they ultimately proved too low quality to keep. Too much #i_feel_good_today #lewdtuesday and #the #party #is #in #my #pants bullshit unfortunately.

Alternately, are you not getting the creator and title tags? I just did a test here and it seemed fine, but let me know if you aren't getting anything at all.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.12893

>>12884

Yeah, I just wanted someone else besides me to try it out. I believe it works for me, and based off of function definitions and descriptions, I'm pretty sure it's acting as expected. However, it's not like I've done some absolute test like using Wireshark to see that no DNS lookups are being done locally, but, rather, are all being done through port my proxy lives on.

>>12887

The website is ATFbooru. I created my own parser/gallary downloaders ect. because I didn't see, at a glance, an existing one that suited my needs. It works perfectly for images–including gifs–however, despite the data being in .webm format when I go to the page in my browser, the download is a .zip. As you said, the zip contains a list of of images, therefore, it's probably that thing you called an "ugoiras".

I can look into the 'danbooru file page parser - get webm ugoira' downloader and add it to my own script.

However, I do want to be certain, the webm conversion wont lose quality right? They're equivalent? I ask as a comparison between differing video compressions (e.g. Xvid yields a loss in quality when compared to H264).

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12894

>>12893

That's the tricky thing with ugoiras–they are a pretty weird format, a bunch of images rendered by javascript according to variable framerate. There is no excellent lossless format to convert them to. The sites you are getting webm conversions from may be using some clever encoding options to preserve lossless and frame timings, but they may not. Webm has a lossless mode, but this is not typical–the filesizes can get pretty gigantic. The problem is aggravated by most artists putting jpegs in their Ugoiras rather than nice pngs, which can look beautiful in an ugoira when presented right, so you are already taking the compression hit from jpeg.

Ultimately, if I can write a native hydrus ugoira detector and renderer (although we'll have to figure out how to attach the JSON frame timings, which aren't guaranteed to be in the zip), then we can just keep the original files. Or figure out the right params for lossless variable frame rate conversion using ffmpeg and do that webm/whatever job 'properly' ourselves. Whole thing is a bit of a shitshow with no ideal solution at current.

If you want to copy the danbooru parser, I think it basically detects the webm embed with a higher URL priority than the normal file download link. Then, when the page provides zip on the download link and webm embed, it goes for the embed, and when there is a jpeg or whatever in both, it won't find the video embed and falls back to the non-sample-guaranteed download link.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b459c1 No.12896

>>12886

Well it's not really that tedious….if I knew how to script.

I thought once Hydrus imported the files, it gave them a new creation date matching their import time though

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.12897

>>12894

>filesizes pretty gigantic

not too important.

>sites use clever encodes ect.

>artists putting jps in their Ugoiras

This is where my lack of knowledge of pixiv comes into play, but what is it that exists on the artist's computer that is uploaded? Does the artist upload a zip of images and then pixiv determines how to render it? If so, then the zip is the "loseless" original since that's exactly what was on the artist's PC.

As a completely separate aside it's a tad sad when you notice something, but it doesn't work out. E.g. there was an attribute field which held all the tags for the image, delimited by spaces. Consequently, if you HTML parse your way into that object and get the attribute, proceed to split (via regex option in Hydrus) you can get a list of all the attributes. However, although one will have a list of tags, they wont be counted as distinct results. I guess it's where optimality and usability are traded.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12900

>>12896

Ah shit, yeah, sorry: I had forgotten how the copystat worked. It copies modified date, but a new creation date is made. I use this, if you are further interested: https://docs.python.org/3/library/shutil.html#shutil.copy2 Some permission bit stuff is copied as well.

This was not true in early versions of the program, I think maybe four or five years ago, so files imported before then will have modified dates the same as creation dates.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c No.12901

>>12897

I don't use Pixiv, so I don't know how you make an Ugoira. The uploader sets the framerate I think. I presume they have an upload form for all the images and you add ms timings beside each file, and then they create the zip and the accompanying JSON or whatever to deliver the timings to your browser. But maybe they have a program that generates that info clientside and you upload it all together.

Yeah, I'm sorry about the inability to split strings at that last stage. When I next put a good amount of time into the downloader, untangling the rats' nest of the edit parser ui will be a top priority along with fleshing out some needed parsing functionality like 'split this one string by whitespace and return multiple strings'. iirc, the current pipeline can't deal with n strings appearing from that space atm, so I can't just add a checkbox or a 'split this by xxx'. That might be wrong though, so I'll make a note to look at it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.12902

>>12885

Thanks for the detailed response. I don't understand what you meant by this because no matter what I do, I have to switch focus from the tag input window to the media viewer to go to the next image:

>Also, by default, hitting page up/down on an empty tag autocomplete input should send a previous/next command to the media viewer parent, so you don't even have to change focus to tag and move on.

Btw, would you be averse to fleshing out the hover tag pane on the left of the media viewer with more functionality? I picture a possible workflow inside the media viewer being:

1. text input gets captured automatically by a new search box added atop the hover tag pane when pane has focus.

2. The hover tag pane shows tags already on the image when the search box is blank. When search box contains text, it's populated with autocompleted tag suggestions, and any autocomplete suggested tags that are already on the image have a highlighted gold background.

3. If there's no autocomplete tag suggestions, then the tag doesn't exist and therefore the Enter key should create and append the typed tag to the image. Idk what the performance implications of this would be, so maybe namespacing can be used to cut down the tag pool so it's less laggy?

3. up/down arrows can navigate autocompleted tag suggestions without needing to use mouse input to change focus from the pane. Enter key appends tag to image, delete key removes tag from image. Shift/ctrl click to select multiple tags perhaps?

4. Left/right arrows can navigate to the next image without needing to change focus via mouse click.

5. right click>mange>file tags for more advanced tagging

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

69bc83 No.12922

>>12796

Still doesn't work. The rule34hentai downloader still ignores Explicit rated images on rule34hentai by saying it couldn't find file or post URL to download.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e63191 No.12923

Is it possible to download a pixiv tag starting from the oldest images through the gallery > pixiv tag search function or do I need to go through page by page starting from the first/oldest page in url download instead?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e63191 No.12924

>>12923

Another bug I've discovered: if I go via the individual page links, after page 10/11 it keeps giving me the same set of images no matter which page I go to afterwards. Very odd!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

deb25c No.12928

>>12897

To better explain, if I open up the media viewer and then hit F3 to open up a manage tags window on top of it, I start with the keyboard focus in the autocomplete tag input text box, so if I were to start typing a tag, results would load in the list below. If from there, instead of typing regular text, I instead press page up/down, the underlying media viewer will browse previous/next. Do you not get that?

If you have multiple tag domains, like 'local tags' and 'public tag repository', then pressing up/down arrow in the same situation–focus on the tag input text box, and no other content in that box–should switch between them. Do you get that at all?

I am open to allowing easier tag editing in the tag list on the left, but the simple view it presents makes it tricky. It presents the 'combined' tags from all tag services in the client, so if you add a new tag, should it go to 'local tags' or 'public tag repo' or something else? I suspect my next step there is to add simple right-click menu options for 'remove tag' and then refactor the tag processing code, which is currently all bundled up into the manage tags dialog, into a shared space, so that taglist can be clever enough to say either 'remove this tag from x yes/no?' to 'this tag is in two services, remove it from x or y?' and so on. Once I have that code refactored, I am open to playing with more quick workflow here.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

deb25c No.12929

>>12922

Hmm, I am not sure then. If you open network->downloader definitions->manage login scripts, go into the 'shimmie login' and run a test on the domain with your credentials, it should list the results of each page hit. You should be able to copy the result body the server returned. You could try to copy and paste those pages into some new .txt files and see if there is a 'bad login' or cloudflare-related error that the login system isn't picking up? I am not sure if I wrote the shimmie login script or someone else, so I do not feel very confident talking about it. I understand it does work for some users.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

deb25c No.12930

>>12924

>>12923

Yeah, the basic hydrus gallery downloader works on the same principle of starting at page 0/1 and walking through each 'next' subsequent page, adding results, just like you might browse. Since most boorus present images in descending upload order, you will be getting newest first. I know some users have played around with adding a booru-specific param to the url to change the sort order, but this has mixed success.

Can you say more about the same results after page 10/11 issue? Is this for pixiv, and is it in your browser or hydrus? Can you give an example URL that does this? Several boorus cap their results to 500 or whatever, even if they suggest something big like 152 pages, and after page 25 or so they'll deliver a blank page, 404, 403, 'please login to see more', or the same last page results again. Does this sound like that?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e63191 No.12932

>>12930

The pages on pixiv in the browser are fine, they continue as normal. But no matter how many times I refreshed the client, reloaded page 12 in the browser and copied it again, it gave me the previous pages' images over and over, which obviously didn't import because they're already in the db.

Here's the URL I was copying:

https://www.pixiv.net/search.php?word=%E3%81%A9%E3%81%86%E3%81%B6%E3%81%A4%E3%81%AE%E6%A3%AE&s_mode=s_tag_full&order=date&p=12

The previous URLs had exactly the same layout except the number was just lower. 11 is the last one to load properly, after that it just returns page 11 irrespective of the page number.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

69bc83 No.12933

File: 79a1eb2c07b16ec⋯.jpg (34.62 KB,832x289,832:289,3464634.jpg)

>>12929

Actually forget all that, I'm an idiot. I didn't set the login to shimmie, this probably should be a default setting for rule34hentai.net if it isn't already.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

17dfbe No.12934

File: 5d5a0908308f8ca⋯.png (9.62 KB,367x233,367:233,Capture.PNG)

Why do I keep getting this?

How do I stop getting this?

I've only got three pages open.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e63191 No.12935

>>12932

I'm dumb and I figured it out. It needed a login to complete more than 11 pages it seems.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f92c9c No.12946

Just started using Hydrus, and I'm trying to get it to autotag an existing collection, but I'm a bit confused.

>Tags are rich, cpu-intensive metadata. My repository has hundreds of millions of mappings, and your client will eventually download and index them all.

Seems way overkill that the user should have to download all that information if they only need tags for the specific files that are part of their own collection. Isn't there a more efficient method?

I'm aware also of the tag archives, but it seems that's also a broad and inefficient approach containing data for the complete file set from a booru/gallery website as well. Is there no other way except to maintain a massive database of all the tag associations of all files ever seen? If downloaders could be used to query downloading tags only for the user's existing files (it may be possible, but I am ignorant), that would be very useful and eliminate the need to waste HDD space/ processing power.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b459c1 No.12947

>>12946

Only if the hash of your file exactly matches the hash used by the booru of choice. Protip: they usually don't unless you got it straight from that booru.

Use >>4475. And read the thread. It'll take some light editing to get it working due to a recent hosting issue.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0ffbbc No.12948

>>12946

As I understand it, main reason is privacy.

Under this system, you have full access to all those tags without having to tell a tag server anything about which files you do or do not have.

And you can just sync once in a blue moon, instead of having to phone home every time you add a new file.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f92c9c No.12950

>>12948

That's a very good point I hadn't considered.

>>12947

Having a real bitch of a time messing with scripts. I got a few from that furfag dr guy. Got one of them to work for one image that I manually selected, and then everything took a shit and the scripts just hang at the fetching stage now.

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/wiki/2.-How-to-fetch-tags-in-bulk-with-md5-hash

It seems this method would be the absolute easiest but there is no longer anyway to get the md5 file hash in the options menu, and the default file names are the same as the sha56 hash. I think I'm going to just throw in the towel and try the public tag repository.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

24e08e No.12952

Not sure has this been asked before: How to minimize the active subscription windows? They are blocking the thumbnails and are very annoying.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.12953

I ctrl+f'ed the threads, so hopefully this hasn't already been asked.

I created a default sankaku channel tag subscription with a test query of "tifa lockhart" (without quotes), but I'm getting a network error.

Specifically, bad gateway 502 along with some html shows up at the top of the edit screen for the subscription. When I first performed the search, it asked me for a user agent string I believe which contained firefox and some other stuff. I did a similar subscription for gelbooru which worked fine. I'm using tor but can access the site just fine in my browser.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

458202 No.12961

>>12932

>>12935

Thank you for the update. Pixiv seem to change their access rules every few weeks, it is a constant battle.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

458202 No.12962

>>12933

Ah, thank you for this update. I think it is supposed to be default for new users–I think the shimmie login script has an example URL that does the link-up automatically on client first-boot initialisation–but I wonder if you got caught up in some combination of version updates that meant this did not apply? I apologise if so. The login system, as I am sure you now know, it pretty a scrappy v1.0, barely out of prototype.

I do remember other users telling me R34H had funny login rules anyway, so I recommend you keep an eye on this in case it starts giving you more trouble. A future version of this whole system will be able to detect if you get suddenly logged out for some reason and will try to recover and give human-friendly errors and so on.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

458202 No.12963

>>12934

I apologise, I fucked up the clipboard watcher! It is spamming those messages false-positively on certain kinds of URL that are not matched. It is fixed today, please let me know if you have any more trouble with it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

458202 No.12964

>>12946

>>12947

>>12948

>>12950

Yeah, privacy and low server CPU are the top reasons. I prefer that your personal CPU cycles happen on computers you control, so the tag repo is really just a clearinghouse of data. The PTR is getting huge now though, 580 million tag mappings, so after this duplicates work I am going to put some time into figuring out better solutions here. One top priority is adding a tag filter to the incoming stream, so you'll be able to say "I don't want filename: or title: tags" and so on.

This is very much on my mind. The PTR has been a great success, but it was always a crazy prototype and we are hitting some limits.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

458202 No.12965

>>12952

No excellent way yet. I think each sub has an option in its edit panel to not show a popup, but I generally advise against this as you often then don't notice it at all, even when it is having problems. This is on my mind, and I have a fairly high priority job to figure out an up/down arrow button or something so you can collapse and expand them. For now, best solution is to hit network->pause->subs and then just unpause them when you are happy for them to run.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

458202 No.12966

>>12953

502 is a server-side error. It could be that Sank was overloaded at that time and could not respond to your request, or it could be CloudFlare giving the 'we tried talking to the site but it timed out' similar message, or they may have blocked Tor. Is the browser Tor as well, or just hydrus?

If you run a test download and check the gallery log button (the little icon button on the gallery search section of the downloader panel), it should bring up a list with the first URL it tried to hit with more error info in the 'notes' column. Do right-click->copy notes and put it in a text viewer–that should be the html page that was returned, which should, if you scroll down a bit, have some more info. It might be like 'Tor is blocked m8' or something else. I'd be interested in knowing what you find, but be careful just pastebinning that html and linking it here, as it may have personal info.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a7d23b No.12973

I just clicked here from /tech/, shithole that it is. I have no idea what this project is. Could we have a sticky?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b459c1 No.12975

>>12973

>New user? Start here -→ http://hydrusnetwork.github.io/hydrus/

lrn2read

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.12985

>>12966

Thanks, seems to be working now.

>>12966

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.12986

What do you guys think a good max allowed bandwidth would be? The default 64mb for sankaku is too little.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

24e08e No.12992

>>12965

That'd be nice. Or make each of them a tab would be nice too.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f92c9c No.12995

I noticed some of the tags from the PTR are just flat out wrong, such as pictures in which a girl is standing and facing the viewer so that her ass isn't even visible being tagged ass, and so on. Is there a way to deem certain public tags as less important and hide them, or a way to ignore public tags and copy only a portion of them to local tags?

I suppose I could always just add a local alias and search by the alias instead of the public tag.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

054854 No.13002

>>12986

I added this after being contacted by a top person at Sank who was concerned about their bandwidth costs, which I sympathised with. After some discussion, we compromised that I would set a restrictive bandwidth limit by default. You can of course edit your client's rules to be whatever you want. If you want to do them a favour, you might like to try to find what content you can on other sites.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

054854 No.13003

>>12995

There are not excellent ways yet. The PTR has plenty of weird and incorrect stuff, and there are so many new tags every day that it is completely beyond me to moderate effectively (I am also not a natural janitor or site-manager). I will be doing substantial work on the client-repository relationship after this current duplicates work. You will have more control over what tags you get and where they go. I expect to add multiple 'local tags' services and easier ways to migrate your tags around so you can better handle all this stuff.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5f52a4 No.13014

Anyone else having problems with the deviant art downloader, first the login script stopped working, now it can't seem to download any page content to parse correctly.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e7d1d7 No.13016

File: 1480f17fa211481⋯.png (1.48 MB,1921x1050,1921:1050,Capture.PNG)

File: 1e3b8fc7807581f⋯.png (5.05 KB,512x112,32:7,imgur-2019.03.07.png)

File: 2d6a9abd4847968⋯.png (6.03 KB,512x109,512:109,reddit-2018.09.21.png)

>>11887

>>11791 here, sorry for the late reply

I tried it again on v355, still happens

I'm using the AIO reddit and imgur parsers (or the imgur one that comes with the hydrus client, not sure) from the "Hydrus Presets and Scripts" repo

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.13017

I created a gallery download page and it stopped after 19 images with a message about 502 bad gateway. I figured the booru was having issues, but no, I can access it just fine. In fact, a new gallery with the same exact query is downloading right now just fine, making the message below confusing:

expires in 50 minutes: 502: <html>

<head><title>502 Bad gateway</title></head>

<body bgcolor="white">

<center><h1>502 Bad Gateway</h1></center>

<hr><center>nginx</center>

</body>

</html>

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c247f No.13018

Is it possible to have two clients share a database? I keep my data on external drives and want to be able to work in both Windows and Linux.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4a3c97 No.13027

>>13018

I think the answer is yes, they just can't be open at the same time.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.13029

>>11122

Sankaku gives you at most 1000 results from any query, but you can circumvent that to some extent if your initial query was at most 3 tags by looking up query + order:filesize and query + order:filesize_asc.

Is there a better solution to this? Finding a custom query to get as close to 1000 images without going over and then having to do this every 1000 images is a pain. Even 2000-3000 images would be a massive improvement.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.13031

>>13029

Nm, I happened to be checking the bugs thread and noticed someone made a parser that solves this supposedly. Just out of idle curiosity, why are parsers shared as png files and not text files? Also could a parser be malicious?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5b6c03 No.13032

Databases work cross-platform, right?

I want to try out hydrus through wine because of all the linux version's issues. How do I go about making the switch? Do I simply copy and paste the linux hydrus network over the windows one like I was simply updating? Or do I restore the (linux) backup I have through windows?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

308209 No.13033

just started using this a few days ago, really liking it for sorting my giant folders of booru rips, but one thing I can't find is parent child picture relations, like danbooru has.

I want to be able to set one picture as a parent, and then another as a child picture, for alt versions and small doujins/image sets. Is there a way to do this or am i fucked?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

03b279 No.13034

>>13031

I wrote that parser. It's a png because Hydrus lets you export them as .png files for easy importing. If you're that worried just copy and paste this:

[58, "sankaku gallery page parser fixed", 2, ["sankaku gallery page parser fixed", "b29f9b48f7c6374d12e7d4cbb527f16fd3c725946dd6479c0684e5e0e1dea15f", [55, 1, [[], "example string"]], [], [26, 1, [[30, 4, ["Next page from rel link", 7, [27, 6, [[26, 1, [[62, 2, [0, "div", {}, 22, null, false, [51, 1, [3, "", null, null, "example string"]]]]]], 0, "next-page-url", [51, 1, [3, "", null, null, ""]], [55, 1, [[[1, 7]], "parsed information"]]]], 0, false, [6, 50]]], [30, 4, ["Thumbnail URLs", 7, [27, 6, [[26, 1, [[62, 2, [0, null, {"class": "content"}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]], [62, 2, [0, null, {"class": "thumb"}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]], [62, 2, [0, "a", {}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]]]], 0, "href", [51, 1, [3, "", null, null, "example string"]], [55, 1, [[], "parsed information"]]]], 0, false, [7, 50]]]]], ["https://chan.sankakucomplex.com/?next=3794306&tags=krystal"], {"url": "https://chan.sankakucomplex.com/posts/index.php?id=123456"}]]

And I suppose a parser could be designed to work in such a way it makes a shitload of network traffic to get you IP banned from a booru, honestly, mine might because sankakuchan hates people using their bandwidth yet has the most expansive collection of images imo. Also it's not too hard to make a image file into a malicious script, that used to/might still be an issue on Linux, it just requires you to execute the file which isn't really a concern with Hydrus.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6aca6e No.13036

>>13033

You can set images as alternates, and you can click an images and find all of its alts, but not much more. It is something the dev is working on currently.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

422735 No.13037

Is there any way to print a list of tags and their occurances? It'd be really helpful to weed out the typos and stupid non-tags artists insist on using in their galleries

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8dbe79 No.13040

>>13033

This is a highly requested feature, which I suspect may win the 'next big job' poll that the dev does every few months. IIRC, the duplicate system overhaul he's doing right now is basically a precursor to being able to overhaul the 'alternates' system.

For now, I'd suggest setting those images as alternates of each other.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c31583 No.13043

I imported the latest Quicksync update files (not the full db), and it's taking forever to process now, like 1 entry per 10 minutes, then cancels out after around 15 minutes. Is there something i'm doing wrong or do I have to let this run for a year?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

afd4aa No.13050

Is there a way to set multiple tags with one shortcut?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.13051

>>13034

But doesn't hydrus execute the code from the png?

Also the downloader still finishes at only 1k even though I've set it to stop after 2000 and changed the bandwidth restrictions. Is sankaku channel tag search the right downloader to choose?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

481666 No.13058

Is there a simple way to set up a blacklist for subscriptions?

I have a subscription setup for sankakucomplex with a few hundred artist queries in it, and while I do want Hydrus to download anything tagged with those artists, I don't want it download anything tagged with, for example, "holding_hands". I know could add "-holding_hands" to each query, but I don't want to have to do that for every single query.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b No.13062

>>13014

It seems ok to me here. I just did a login test and download and things came through ok. Are you getting any errors, or are the URLs just not appearing when you do a gallery search? If they have any errors, can you look at the list of file import statuses (hit the little icon button on the downloader page) and right-click->copy notes on the bad results, so we can see more of what went wrong? I wonder if they turned on CloudFlare protection or something briefly.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b No.13063

>>13016

Thank you for this follow-up. After a bit of playing around, I was able to reproduce this problem. There is a certain unusual parsing type that was kicking in here that was not sending tags to the next stage down. This is fixed for 358. Please let me know if you have any more trouble here.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b No.13064

>>13017

If you turn on help->debug->report modes->network report mode while this works, does that give you any more information? Is there perhaps a url class that is redirecting the URLs to a different URL? I've had that sort of problem before, where I was accidentally hitting the wrong URL despite the URLs in the file list looking correct.

It could also be some CloudFlare-style protection that is blocking anything that doesn't look like a browser with a BS 502 error. Usually you get more info than that brief html document though.

If you have some example URLs, I am happy to try from my end in browser and hydrus.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b No.13065

>>13018

>>13027

Yep, the db is platform neutral so you can have two installs point at the same db and they should be fine. Only rule is only one client install can be open on a db at a time. Normally this is impossible to violate since the client will detect if it is already open on the same computer and complain, but if you have a funky network setup this check can be skirted.

If you just want to have a db on a usb drive and move it back and forth between two machines or boots, Linux and Windows, you should be completely fine. I recommend you extract two portable installs and put the db proper in its own directory, something like this:

E:\hydrus_db (db and client_files folders go here)

E:\hydrus_linux (linux install extracted here)

E:\hydrus_windows (window install extracted here)

And then launch Windows/Linux with shortcuts with '-d="..\hydrus_db"' or '-d="../hydrus_db"' as appropriate. But you know your situation better. Here is some more help about all this:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

Let me know if you would like any more help.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b No.13066

>>13032

I'll just point you up to my post above: >>13065

Basically there is no trouble migrating the db around–it is completely portable and will work on any install from any location, so you are good to copy or move it around to test in other locations. As long as you can see the main db dir with your hydrus client exe or script, you should be good to go and if everything is working, it'll launch.

I still recommend making backups and so on before you try anything too fun, just in case.

The help file here talks about the different components of the hydrus install:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

I think in your case you'll be wanting to move around your original install_dir/db folder. Extract hydrus to your wine folder structure and then copy that db dir straight into the install. I am happy to help if you run into trouble.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b No.13067

>>13051

I am no cyber-hacker-3000, so I cannot promise anything iron-clad, but my serialised png system shouldn't execute any external code. The data embedded in the png is zipped up JSON, stored as grey bytes at the bottom of the png itself, which I read and load. It unzips to the same as in >>13034 's post. There's a couple of data bytes in the top-left of the png to say where the data starts and how long it is. If what is read and unzip parses as a hydrus parser object, it is good to import, otherwise it will throw an error. At no point are arbitrary lines of python code intentionally executed.

If someone has an attack vector for https://docs.python.org/3/library/zlib.html#zlib.decompress or https://docs.python.org/3/library/json.html#json.loads, they could make up a malicious png. My Windows dev machine here says the current zlib version I am on is 1.2.11, which the docs suggest is currently OK.

If you are very concerned about your privacy, I recommend you only import by copy/pasting the json text and giving it a sanity pass with your eyes just to make sure it isn't trying to api-url-redirect your gallery queries to "NSAHydrusUsers.ru.cx" or anything.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b No.13068

>>13037

Not at the moment. Many clients have way too many unique tags to show in one place in any kind of reasonable wait time, so figuring out show-all UI has been tricky. There's a 'big job' to figure out a tag cloud control which would likely have this tech.

If you only have, say, 2,000 unique tags, you might have some luck messing around with wildcard searches in the autocomplete tag input. I have some blocks to stop simple '*' queries, but someone was telling me the other day that '*:*' works and will show basically everything you have. Be warned it may take some time for such a query to return all results and may be hyper laggy.

Otherwise doing a search for a whole bunch of random files and then sorting the results' tags by incidence is a decent way to find (1) and done stupid tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b No.13069

>>13043

Try forcing an analyze under database->maintain->analyze->full. It will take a few minutes, but if your problem here is due to too much data at once, this should fix it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b No.13070

>>13050

Not yet. Future shortcut system should be able to do this, but I will likely not work on that until I have caught all the old code up to what is current.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b No.13071

>>13058

Sure. You do this through tag import options of any downloader context. You don't want to set this for every individual subscription, so I recommend you go network->downloaders->manage default tag import options and then set them either for the global default or per domain. If you haven't overridden this info in your subs, your subs will inherit from these defaults at runtime.

Under your tag import options edit panel, check the button for 'set file blacklist'. Add your scat/gore/whatever tags that you don't want, and any downloader using those tag import options that finds a page with one of those tags will skip the file download and give an 'ignored' status.

It is worth pausing your subs while you do this and testing this with a manual download page, just to make sure you have it set up how you expect.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b459c1 No.13072

What was the procedure for manually copying login cookies into the client?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.13073

>>13067

Thanks for the breakdown, and I'm not too concerned about it. It was an idle thought that arose from thinking how this png system weirdly obfuscates the downloader code, so I wondered what the security implications were.

>>13034

I looked and I didn't see any new downloaders added, so I assume this was added onto the sankaku channel tag search downloader. Is this intended for use with the gallery downloader? Maybe that's why it halts after 1k downloads.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7b9286 No.13075

>>13072

This should be explained in >>11696.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6b1ae4 No.13077

>>13072

>>13075

I have a lot going on at the moment, but in the 'to-do fairly soon' list is adding cookie access to the Client API, which will make this whole thing a lot simpler. The guy who writes the browser plugin is keen to add a button that'll just transfer login cookies to hydrus in one click.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6b1ae4 No.13078

>>13073

I don't know exactly how or what you imported in your situation here, but if you added this, you likely also need to update the 'links' to make sure it is applied to the right 'url class'. Check network->downloader definitions->manage url class links. Find the sank gallery url class and make sure it is matched up to the new parser you imported.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

fdc241 No.13083

Anyone else having issues with pixiv? I keep getting an error that "The parser found nothing in the document!" Only started happening within the past 48 hours.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9687ab No.13084

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.13085

>>13078

I imported the png version of >>13034 via network>downloaders>import downloaders. It's supposed to allow the sankaku gallery page to exceed 1k items.

Interestingly, after I did network>downloader definitions>manage url class links and applied `sankaku gallery page parser fixed` (which I assume is from the png I imported) it consistently stops after only 24 items.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1d8061 No.13086

>>13085

Hmm. That is the correct procedure for switching over. I expect you'll want to switch back until this gets figured out.

I am afraid I do not know anything about the 'fixed' parser other than it is supposed to pull a different gallery page with some 'startswith' id or something, so I can't talk cleverly about debugging it. I assume you are logged into sank if you were originally getting 1k results, otherwise I would say 'try logging in'.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0ced6e No.13107

Is there a way to use Hydrus with sites that make you click "Agree" or "Enter" before you can use them, e.g. giantessbooru.com?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3b3d66 No.13108

File: ef5654866847151⋯.png (22.22 KB,358x439,358:439,error.PNG)

Whenever i try to download tags from the repo i get this error. It's always the same keyerror. I tried looking in the DB files for they key in case it was conncected to a faulty path but i can not find it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0ced6e No.13111

Hydrus doesn't download anything from DeviantArt, no matter if I log in or not. I understand that you're supposed to have the client download something without logging in before you can do it with an account, but how?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4 No.13114

Gelbooru added some gay default tag blacklist you have to modify to access certain tags. Any way to remove them from hydrus?

Here's an example.

https://gelbooru.com/index.php?page=post&s=list&tags=loli

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4 No.13115

>>13114

Also, when you go into the options you can enable all site content and it gives you a special key you should be able to throw in with searches to get the full results like "&api_key=anonymous&user_id=9455", but I couldn't get it to work with anything.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026 No.13117

Speaking of tagging systems, what are idealistic searches you wish could be usable on boorus?

>eg. "on-model"

>eg. "girly/femboys but no masculine males in the image" (possible solution would be having negating tags for like "non-femboy")

>eg. "horsecock but only on horses"

As a side note, I found an imageboard which used anime girls for an image captcha. I wonder if that could be utilized for tagging boorus, like how Google uses theirs for AI training.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1093d No.13118

File: 65b6599d7235be7⋯.jpg (73.93 KB,499x454,499:454,1445ea260a25cf25d2b5ab5709….jpg)

is there a way to exclude a tag from a 'everything' search?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0ced6e No.13121

>>13111

Never mind, I figured it out. I needed to clear cookies first, through the login menu, then try again. Now that I have downloaded some pictures without logging in, I should be good to go.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4 No.13122

>>13107

Yeah, should be able to. In the new downloader system, that process would count as a 'login'. You can create one of these yourself, although it is not trivial to do. You also need very basic web development knowledge, such as what a cookie is.

The help for all this starts here:

https://hydrusnetwork.github.io/hydrus/help/downloader_intro.html

There is supposed to be some login help here:

https://hydrusnetwork.github.io/hydrus/help/downloader_login.html

But I just discovered that I never wrote any! If you would like to write your own login script, skim some of the downloader help to see what my approach is and then check out network->downloader definitions->manage login scripts. Your client should have 'hentai foundry guest login click-through' as one of its defaults, which basically does the same as what you want here: if it doesn't see a session cookie in the HF session, it simulates hitting the main page and clicking-through.

I think if you duplicated the HF login and then edited it to your other site you might have some luck. That particular HF script also posts an HF content form to unhide spicy content, which you can remove. I am happy to help you if you run into trouble, and if you figure something out, please post it so other users can benefit!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4 No.13123

>>13108

Thank you for this report. Please pause the repo if you have not already. Can you then go to your log (install_dir/db/client - 2019-7.log) and scroll down to this traceback and copy the last bit? (If you still have the popup, you can do the same thing with the 'copy traceback' button). I think I know what is causing that error, but I'd like to check the whole error be certain.

It looks like your files manager is/was unable to find one of its subdirectories. Have you ever moved your database around with database->migrate db? If you have, please go to your moved 'client_files' dir. If you have not, please go to the default at install_dir/db/client_files. Are there 512 directories in there? There should be 256 in the format fxx and 256 with txx (files and thumbnails, respectively), with the xx counting in hex from 00 to ff. Do you have the 'f4c' directory? I suspect you do, because if it were missing, hydrus would complain on boot. If so, it sounds like your db has somehow lost its internal store of db locations.

If you do have the f4c folder, please make sure your client is closed and then jump up to your base db dir, normally install_dir/db, and run the sqlite3 executable. Copy paste these lines into the terminal that pops up exactly:

.open client.db
SELECT COUNT( * ) FROM client_files_locations;
SELECT COUNT( * ) FROM client_files_locations WHERE prefix LIKE "f%";
SELECT COUNT( * ) FROM client_files_locations WHERE prefix LIKE "t%";
.exit

You should get 512, 256, 256, but I assume some will be less. Please let me know what you get. We can fix it manually if only f4c is missing (i.e. if you get 255), but if there are more missing I won't walk you through manually fixing this issue, since it is a pain in the ass going through 256 entries to figure out what is missing. If this is your issue, I will write a new repair routine on boot that will catch this unusual problem and try to fix it automatically. I am 95% certain I can have this in for 359.

You should not import any files to your client in the meantime. That missing directory issue will affect any file imports. The cause for this issue is also a concern. I cannot think of any way that one of those directory entries would be removed. Anyway, let me know what numbers you get, and the full traceback, and we'll get further into this.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4 No.13124

>>13111

>>13121

Thank you for this report. I have a report from another user that suggests the DA login script is broken, and I am due to give it a look this week. I am not sure what your cookie issue was, but please keep me updated if you learn anything new.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4 No.13125

>>13114

>>13115

I don't have a gelb account so I can't talk too cleverly, but I think your best bet here is to have hydrus login to gelb for you. I recommend you use a new throwaway account. Am I right in saying that gelb has a blacklist tied to your user account? I assume you can set your throwaway account's blacklist how you want, and then hydrus will inherit that when it logs in.

To add that api_key to searches in hydrus, you'd have to alter both the gelb GUG and URL Class in hydrus. It might be a bit of a pain if you aren't used to the system, so I recommend trying the login first. Let me know if that works for you. Related help at the bottom here:

https://hydrusnetwork.github.io/hydrus/help/getting_started_downloading.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0ced6e No.13126

>>13121

>>13124

OK, so I can download stuff if I'm not logged into DeviantArt, but once I do log in (and it does say I'm logged in just fine), it doesn't download anything. The login says "valid", and I get no error messages; it just doesn't import any files.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0ced6e No.13127

>>13122

Thanks! I'll look into it and see if I can make it work.

I actually sought help from that downloader_login page before posting, but it wasn't too helpful, as you can imagine. :p

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4 No.13128

>>13117

I don't tag on boorus, but for hydrus I am all for longer/combined tags to clarify these situations. I think tag parents in hydrus can help tag quicker too. For instance:

horse penis on horse

could have all these parents, some being grandparents:

horse penis

equine penis

animal penis

penis

animal genitalia

species:horse

species:equine

whereas 'horse penis on futanari' or '-on human' or whatever other pseudo-suffix would have a different set of nice parents. It is neat as your brain can work on this higher level of recognition and it lets you add a bunch of valid tags in one action. Same with something like "girly/femboys but no masculine males in the image" or the various "small female big male" stuff you come across sometimes. With these specific tags, you can define precisely the core of what you are interested in in the image without being awkward or overbroad in the search. Also it lets you quickly and precisely dismiss what you don't want with a single negation tag like ['horse penis on horse', '-horse penis on human' ], rather than ['horse penis on horse', '-species:human'] or ['horse penis', 'species:horse', '-species human'], which are not the same thing.

I think you could easily go crazy trying to define these forever, however, so I will fall back to one of my main tagging rules: only tag what you are personally interested in. If you want to see horse penises only in their natural God-given state, then pursue this 'horsecock but only on horses' path, and perhaps the '-on intersex' companions you might want to exclude with a negation tag. But don't try to tag everything on everything on everything or you'll go nuts.

I have spent thousands of hours creating an engine that burns trillions of cycles of CPU to precisely categorise and search different types of bepis. Didn't expect I would be here ten years ago, jej. I am certain that within ten years we will be feeding this info into AI tech of our own with some fun results. :^)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4 No.13129

>>13118

Yeah, when you start typing a tag, put a hyphen "-" before the tag, so like "-footwear". This will discount any file with that tag from the search. I think all regular tags will work this way, including special tags like "creator:*anything*". If you have some tags in the 'selection tags' area on the left, you should also be able to right-click them and say something like 'exclude this tag from the current search' and some other options.

Note that 'system:everything' is secretly implied to any search that has >0 tags, so if you just enter for '-footwear' or whatever, you don't need to add the 'everything' yourself.

Also, when you do big searches like this, try hitting the 'searching immediately' button (ctrl+i shortcut as well, I think), which will pause the search-on-new-tag and lets you queue up a bunch of tags before actually firing off your big search.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4 No.13130

>>13126

Thank you for this info. No idea what's going on here–maybe the Eternal Cloudflare has struck again, or maybe once you log in they now deliver differently 'shaped' gallery page data or something. I'll look into it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4 No.13132

File: a2ccb844c47ccc4⋯.png (15.2 KB,776x164,194:41,1.png)

File: 6cbce6fcb8c5015⋯.png (50.98 KB,670x886,335:443,2.png)

File: 35dcad48dc090c7⋯.png (29.74 KB,710x296,355:148,3.png)

File: 3fc92ddf5f9c710⋯.png (14.49 KB,754x452,377:226,4.png)

File: 084662c329cf53c⋯.png (57.45 KB,750x244,375:122,5.png)

>>13125

>Am I right in saying that gelb has a blacklist tied to your user account?

Yes, but it also has a default blacklist which you need to disable in your "account". Your account being any account including temporary anonymous ones.

Anyway, I tried it before but I forgot to modify the GUG before, so I tried it again and still nothing. The URL throws the extra information in properly, but still no results. (1-3)

I also tried creating a temporary account, it logged in properly, but still no results despite the cookies looking like they're working. Throwaway account and literally billions of possibilities for the remainder of the hash, so there's no problem with the fifth pic. (4,5)

I tried restarting hydrus just in case but still nothing.

The API thing doesn't work on the browser, so I guess that's Gelbooru's fuck up, but logging in with an account with the content enable does, so that might be some fuckery with how hydrus downloads stuff I'm guessing.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3b3d66 No.13136

File: f1c3809c8ac6ea6⋯.png (7.86 KB,628x141,628:141,query.PNG)

File: 07121d68648065d⋯.png (68.68 KB,817x840,817:840,traceback.PNG)

>>13123

I have 250 'f' folders and 256 't' folders. I did migrate my db from one folder to another at one point and suspect that something got messed up in the process. I have also noted that when i run the 'file integrity' tool other folder names appear, which i presume are other missing ones.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3b3d66 No.13137

File: 7f211373e6f9651⋯.png (65.26 KB,702x889,702:889,traceback2.PNG)

>>13136

Another example.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a4dd9b No.13139

>>13085

That parser was faulty, I believe. I made it work for me some time ago, should be at >>13138 – please tell me if it works.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bf766a No.13146

To hydrus dev specifically: how do you manage all the tasks involved with a project of this scale? I find it staggering that you're able to keep on top of it all so well, presumably as one person. Apologies if there's already an answer around.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ff8fee No.13159

I asked some time ago about how to restore old downloader functionality(e.g. "get X more new files"), and I tried simply increasing the "stop after this many files" values(search and gallery parser) and resuming the search, but Hydrus doesn't do anything with it.

Is there anything else I can do or must I switch to searching with a high value initially and pausing when I'm satisfied?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

939696 No.13168

File: 04eda2644078404⋯.png (3.97 KB,379x126,379:126,display all site content.png)

>>13125

I've been messing around with it a little bit. Even though I was able to log in to Gelbooru successfully in Hydrus, it didn't allow me to download any blacklisted tags, neither in a download page nor subscription. However, following the instructions from the help page you linked and exporting a cookies.txt from browser into Hydrus did allow it to work. I suppose that means the relevant info is not tied to your account but is in a cookie.

I think this is the setting in Gelbooru's options that's causing the issue.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4 No.13170

File: 00fca99fb072ea5⋯.png (16.73 KB,312x287,312:287,ClipboardImage.png)

>>13168

I thought I had looked through the cookies before but I'm guessing I'm retarded.

Pic related is enough to get it working.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d9f4f4 No.13171

I've been working on creating my own parser recently, and have been successful at the parser bit, but it only downloads the first page of a gallery link. How do I make it download the entire gallery?

My gallery parser correctly returns a "gallery next page url", and I have a url class of type "gallery url", yet only the first page is being downloaded.

Am I missing something stupid?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

086196 No.13180

Whoahhhh so what the fuck am I looking at here? What exactly is hydrus? Can someone please give me a run down

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4 No.13182

File: 9ffbd02c9c77292⋯.png (83.15 KB,752x786,376:393,ClipboardImage.png)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4 No.13183

File: c904bcea9b3e1e9⋯.png (1.34 KB,294x29,294:29,ClipboardImage.png)

File: cf8817ce53633ae⋯.png (64.47 KB,1136x238,568:119,ClipboardImage.png)

>>13171

Try setting it to no limit and download a tag you know goes past one page, then look at the gallery log. The last URL should give you a note that'll explain what's fucking up.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

68ca54 No.13184

Is it possible to have the tag autocomplete enter existing namespaces when tagging with a word that isn't a non-namespaced tag? e.g. entering "splatoon" automatically tags as "series:splatoon" since I don't have a "splatoon" tag in my tag services. I know it's possible with tag siblings but adding an additional pair for each tag is a lot of extra work.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4 No.13185

File: 29c08084f92875a⋯.png (4.43 KB,300x166,150:83,ClipboardImage.png)

>>13184

It already autocompletes in the searches if that's what you mean. If you're talking about tagging it as "splatoon" and having the tag translate to "series:splatoon" then yeah, you need sibblings.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a No.13191

>>13132

>>13170

Hey, I am sorry I was late to this. I think another user (sorry if it was you somewhere else, and I am forgetting) was telling me that this cookie is not tied to the account, but the session. I still have a job to add cookie export to the Client API. This will make this operation a one-click action for anyone with the browser add-on and clear the hassle from this shit.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a No.13192

>>13137

>>13136

Hey, I am sorry getting back to you late here as well. I believe this now auto-heals on boot in 359, so have you had luck with that?

For some reason, six entries have been dropped here. You might like to check install_dir/db/help my db is broke.txt just as some background reading. It might be worth running a db integrity check on client.db at least, just to see if there are any other dropped rows in that file.

If the new version did not fix this automatically for you, you can fix it manually by using this line:

INSERT INTO client_files_locations ( ?, ? ) VALUES ( "fXX", "client_files" );

For each dropped fXX prefix. fb9, f4c, and four others that I imagine you can figure out by booting and running some more file integrity checks or similar. If your actual location for the files is not install_dir/db/client_files (as per the "client_files" parameter there), the client will ask you to relocate to the correct place on boot.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a No.13193

File: 3aa3b38b6d702ab⋯.jpg (139.19 KB,900x1200,3:4,3aa3b38b6d702ab341f50bcac3….jpg)

>>13146

I use ToDoList by AbstractSpoon to 'keep track' of everything, and I strongly recommend this software for any project tracking (I also use it for daily schedule management), but in truth I am always drowning and overwhelmed, ha ha.

I am just one lad, and once hydrus picked up a little, my todolist queues quickly turned infinite, almost always growing faster than I can process them. You can see the current running todo count at the top of this page–1,783 items atm jej. I sometimes cut that number by a handful after a good week of small work, but it typically grows by 15 or so items a week. The ride never ends.

I have found the trick, given the unrelenting queue, is to instead bottleneck my own schedule so I don't burn myself out. Marathon not a sprint, etc… Although I love working on hydrus, I get tired pretty easily. I usually do about 30-40 hours a week on hydrus, and while I could easily 'schedule' 100, I'd probably still not catch up on my to-do for another twenty years and I'd kill my soul pretty quick and quit the project. Strict schedules have been helpful.

Although I am fairly clever, I am not conscientious. I am not normal or professional, so I have some different ways of working compared to typical devs on other projects. I don't do many widely considered good practises and the back of my mind generally recoils at them, viewing them as glow in the dark. Coming to know what I am weak at–and then not try to do that–has been difficult but important. I know that I burn out and end up in drama-bombs when I work with others, so I have a strict solo-dev policy for hydrus, and that has worked really well for me. Weekly updates is another–I am a 'do homework the night before' kind of person, so whenever I have tried monthly updates, I end up playing an MMO for 2.5 weeks and then scrambling to catch up. This schedule keeps me focused and allows for quick prototyping iteration and bugfixing turnaround. Focusing on larger jobs has been difficult, but the 'big job' poll, giving me a day or so a week to push on a longer-term thing, has been great. And having the new 4-weekly rotation of small jobs, cleanup, small jobs, ongoing polish has been good at ensuring I don't get lost in the muck of some bullshit for six weeks and forget about other important things.

The most difficult thing that I still fail at is keeping up with messages. I am not social and don't enjoy stepping out of being Anon. It is real work for me to deal with posts and DMs and emails, and it isn't something I can avoid, so I mostly catch up when I have the motivation and guilt to get it done.

It helps having an Anon-centered userbase. I love imageboards and enjoy having tech-fluent users whose hand I do not have to hold. It means I can put out scrappy tech and make autistic but functional UI rather than needing to make things nice and easy and pretty, which I am not good at, and still get great feedback on complicated topics.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a No.13194

>>13159

Yes, unfortunately there is no way at the moment to get that old queue-building method back.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a No.13195

>>13168

>>13170

>>13191

I guess since this is divorced from needing an account, the cookie is not actually remembered account-wide. Thank you for this information.

If this is a tick-box with a typical html POST form to save it, I should think it would be possible to both write an account-less login script and extend the existing account-credentialled gelbooru login script to also do this step, if they user wanted. The Hentai Foundry scripts do something similar here, to do the initial home page 18+ screen click-through.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a No.13196

>>13171

>>13183

If you are testing the URL in the normal URL downloader, this page doesn't have the pipeline tech to do a gallery walk, so it will only ever parse the single gallery page you drop/paste into it. If you want to test a full gallery walk with 'next page' stuff, you will need a GUG for it and do it in the gallery downloader.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a No.13197

>>13184

>>13185

Yeah, my first siblings system proved only half the job here. I want to make sibs and parents much more portable so users can just grab like a 500KB text file from somewhere and just pull a whole bunch of decent 'series' siblings from the PTR to their local tags in one go. Doing it as one person by hand is a huge pain, and the current system does not resolve subjective disagreements well (e.g. is "bodypart:", though technically accurate, actually 'good'?).

The next step of tag sharing will be better en masse definition curation, and making it more shareable.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cc0406 No.13206

>>13193

Thank you so much for this fascinating insight. I've been struggling with organisation a lot recently, so I thought I would ask you how you do it, as you're clearly able to continually deliver despite all the challenges you mention. That's where I'd like to be, and I have every respect for you. That software looks like just what I've been looking for (and not finding) - thank you again.

I thought that keeping up with messages might be difficult, as you have so many scattered locations to check. I expect it to be a shortcoming that imageboard communication is naturally ephemeral, so you may feel pressured to follow up with people in a timely manner. Perhaps that helps you keep on track though. I expect I've taken quite a bit of your time at any rate, for which I apologise. Certainly don't feel like you should respond to me.

I have held off on suggestions for the software in respect of the fact that the userbase is something worth preserving, as you say. Any improvements to the UI, for example, would probably lower the barrier of entry and result in an influx of lower-skilled users, causing more hassle for you. I had wondered, though, if it was possible to write hydrus in such a way that frees you up to work entirely on the backend, leaving frontends to interested users. But I'm sure it's a lot of work for not much payoff.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cd5321 No.13209

Am I required to run hydrus as an administrator? I don't recall needing to several versions ago.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a No.13211

>>13206

Thanks m8. I appreciate your comments. I like to insist I am just a lad though, no one special. I don't want to get a big head.

It has taken years for me to figure out a good schedule and attitude to doing this work, and there is no magic trick I have found except keep working every week and recovering over and over from failure. If you want to figure out your own system, my best advice is to keep at it, and if you are young, don't expect it to come as fast as you want. Stress is part of the whole package, like any other work, and another resource to be managed.

Stoicism works for me. It is a little cliche in some imageboard circles, but if you haven't read it, I recommend Marcus Aurelius:

https://archive.org/details/meditationsofmar00marc

Now that hydrus is functional and does a whole bunch of things, my overarching plan is to move myself and the hydrus code to be more platform-focused, providing a base that users can make things for, rather than pushing towards a complete package I have to continually tinker with. The downloader overhaul last year was the first really big stab at this–moving the whole thing to be completely user-configurable so regular users can populate and maintain it, rather my having to hardcode and update all download info myself. I now want to extend the Client API, maybe play with add-ons of some sort, and 'generalise' the code to allow more user-configurable behaviour. This lets me worry less about infinite specifics and spreads the workload without my sperging out at working in a team.

I'll be doing some PTR/tag repository work after this current duplicates stuff. Main push will be not dissimilar: giving clients more control over what they pull from the PTR, and starting work on improved local tag siblings management.

Completely decoupling the UI from the db is probably too big a job to consider at the moment, but I know some users are planning pseudo-booru wrappers using the Client API. We'll see how that goes and iterate on from there.

I am all for new suggestions, btw, so please don't be afraid to say something. I just have trouble keeping up with them a lot of the time. I'm not about to add a Facebook plugin, ha ha, but thoughts on improving and cleaning up workflows is always helpful. As the guy who understands the cogs behind everything, it is often difficult to picture what is confusing or unhelpful to users.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a No.13212

>>13209

Shouldn't do, unless your db is in a protected location like 'C:\Program Files'. The client needs to write to its db dir, which is by default in install_dir/db, so if that is in a protected or otherwise read-only location, it might not work as a regular user.

What error messages are you getting, if any, that drove you to try 'run as admin'?

Here's some general info on the db and how to move it about, if that is helpful:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cd5321 No.13216

>>13212

Ah okay, the install directory got moved recently after discovering Chocolatey and installing hydrus through it. I'll see about moving it back to the root of C:\ where it was before.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cc0406 No.13217

>>13211

Yes, that makes a lot of sense. I agree about stoicism. I happened upon your recommendation of Marcus Aurelius to another guy, and it's good stuff. Some self-discipline is certainly in order, which cannot be provided by software.

I certainly intend to give some feedback after I've had a chance to organise my thoughts. I'm sure most of it, if not all, will have been said already though so hopefully it's not too tiresome reading duplicate suggestions. Can't really see a public to-do list (or the like) around to help prevent that, though.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3b3d66 No.13220

>>13192

Updating to the latest version seems to have solved the issue. When i started the client i got the prompt that the db entries were missing, and i could then point them to the folder where my files are stored. Will do some integrity checks now to see if anything else breaks.

Thanks for the help!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1d1a8c No.13221

File: be3abb69baee671⋯.png (330.79 KB,594x640,297:320,1557349478670.png)

Is there a way to force-download the whole tag set of the PTR? I increased the max bandwidth for Hydrus services but it's saying

>service has exceeded bandwidth - next request in 5 days 17 hours

I know there was a way to do it in a previous release but I can't find it now.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1d1a8c No.13222

>>13221

Ah, never mind. I found the option to import.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4 No.13226

File: 251535f27a89464⋯.png (948.46 KB,791x725,791:725,251535f27a894644416f8e13c6….png)

I already have the PTR downloaded, is there any way I can export it and import it on a separated hydrus?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

481666 No.13238

Did the dev ever add the ability to manage multiple databases? I know there was talk of it at one point, but I haven't kept up with development in a while. I like the idea of having separate databases for SFW and NSFW or some similar scheme.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a1195b No.13241

Why is there still a search plugin for the Mishimmie? It's dead, it was never exported and Dreamhost has no interest in restoring it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516 No.13249

>>13217

Yeah, I'm afraid my to-do is too big to reasonably convert to a public system. It has personal info in it too, from user submissions and so on, so I'd have to sanitize rather than just copy/paste. I also haven't found an online issues system that doesn't make my head explode either–for a while, I had github issues activated, but for me it just resulted in dozens of long discussions about how potential new features might behave, and I lost more and more work time to it until I couldn't keep up and abandoned it. All that shit is driven towards boosting social media engagement, which is the very thing I hate, kek.

I know some users on the discord tried to make some sort of wiki to act as a 'frequently asked issues' or something, but I don't know how it ended up.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516 No.13250

>>13220

Great, thanks for letting me know.

>>13226

Yeah: turn on help->advanced mode and then check the repo under services->review services. Hit 'export updates' to pull them out, and then on the new (also advanced mode) client hit services->import repo update files to push them in. Then do one sync with the PTR on that new machine so it knows the updated files it wants, and it should recognise it already has them and be able to start processing right away.

Be very careful doing this. It is a lot of CPU work all at once, and I didn't design the processing pipeline to deal with it. I recommend you let it run for x minutes processing on a shutdown maintenance run (which will have strict timing rules and be able to cancel itself eventually even if it runs super slow), and then restart the client and run database->maintenance->analyze->full. This will make the db do some optimisation stuff on the new populated db tables so it can do the rest of the processing at decent speed.

I overall do not recommend hitting 'process now', if any user has told you about it, on a fresh sync. Some users have tried to do a full processing run in one go from nothing, which usually locks up the client due to mis-optimisation and slow work.

Let me know how you get on!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516 No.13251

>>13238

Not yet. It'll be a pretty huge job, unfortunately. I'd like to have multiple local tag services coming in after this duplicates work as part of a significant en masse tag management overhaul.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516 No.13252

>>13241

Thank you for this report. I'll take it out of the defaults.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

dae166 No.13258

Is there a way for me to filter by filetype? I know you can sort by filetype as an option, but say if I wanted to only see .jpg files of a certain tag, is there a way for me to do that besides manually adding filetype tags to every image?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1f5fbd No.13263

Guys, how do you set up a Derpibooru login? I do not know what kind of login script it uses. My grabber just uses the API key.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6aca6e No.13265

>>13258

yes you can filter by filetype same way as you apply other filters to a search

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7d6a76 No.13269

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d693b0 No.13273

is there a way to know the last time hydrus was open? I went to a repair center and I'm afraid the saw all the nasty shit I have.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6aca6e No.13279

>>13273

Maybe the log files? Check last time modified, or if there are any dates written to them.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9403ab No.13280

>>13279

fuck, they did.

fuck fuck fuck fuck

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9403ab No.13281

File: 25f30059e6c85c3⋯.png (165.33 KB,1366x768,683:384,scrot_2019-07-23-23:33:59_….png)

>>13280

>>13279

fuck I'm not going back there never again.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6aca6e No.13289

>>13281

Holy shit, there were looking at your stuff for like 7 hours? Maybe they want a copy

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

79c1de No.13299

File: e52f2c611cfaf72⋯.jpg (522.28 KB,1920x1080,16:9,cunt destroyer.jpg)

If my filenames already contain character delimited tags like "dog, corgi.jpg" how can I have Hydrus import those words individually when I select "add tags from filename", rather than creating a single "dog, corgi" tag? I assume there's a regex trick.

Also any general tips for scrapping tags from an existing image library's filenames?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3f7141 No.13300

>>13281

F

This is why we use passwords

(And not repair centers)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3607e9 No.13301

Hopefully ther's an easy way to do this.

I have two seperate instance of Hydrus with different files, tags etc.

Is there a way for me to merge them into a singular Hydrus instance with all tags intact? Could I start the server on one port and connect to it from the other instance and grab everything?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f95d3 No.13302

File: 41ce6a73d3d4559⋯.jpg (180.82 KB,750x750,1:1,1509677821_1392204711558.jpg)

The menu bar on Hydrus is missing. Not the sidebar, the one on top. Is there a way to get it back? I don't remember removing/disabling it.

On a semi-related note, is there a list of built-in keyboard shortcuts?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

623fd2 No.13307

File: 8aeab596c788f21⋯.png (406.58 KB,1500x1000,3:2,ClipboardImage.png)

File: fd1e0ddb7dacb5d⋯.png (1 MB,1920x1080,16:9,ClipboardImage.png)

I wanted to make a thread, but it would probably be useless if the BO/devs don't agree to the proposal, so I think I will just post here.

Due to the recent take down of ExHentai, I'm aware some anons use Hydrus, either as a database for their own pics or not, myself included, since I use it to tag hentai doujins I store for easy access so there's a chance that they have doujins databased on hydrus of their own.

What chance there is to convince the Hydrus dev of making something out of this? So anons makes a collective database of mangas and doujins so they can be easily shared there and to some other place?

It could work like p2p, so no extreme bandwidth use in Hydrus servers, hopefully, and if it works good, it'd work like a torrent client of sorts, but to display these doujins directly. If it goes well, big circles can host their doujins themselves there.

I might be extremely wrong, though. But just want to know if there's a chance.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4 No.13312

>>13307

Hey, I am afraid I have a ton of IRL shit going on today, so I can't catch up with my messages properly until tomorrow. I will try and check in again later this evening. Please feel free to make a fresh thread. There is certainly a lot to talk about on this subject.

If Anons can get use out of hydrus as an archive tool, I am happy. As I say a bit here >>13305 , to me this episode is a reminder that you can't rely on content, particularly spicy content, remaining in a location you do not control. Hydrus was in part designed to solve this problem by being a 'local' booru.

That said, hydrus is not currently well geared towards paged content like comics. Increasing support is planned, and I am in no way opposed to adding cbz/cbr import support and improved paged browsing, but at the moment the overall hydrus workflow works best for single images like you would find on most boorus and imageboard posts. Any Anons who would like to know more, please feel free to ask other users here or check my 'getting started' help here: https://hydrusnetwork.github.io/hydrus/

In regards to IPFS or using Hydrus File Repositories or something to effect better sharing on large scale, I think this will be increasingly possible over the next few years, but hydrus is not currently there. The IPFS integration (which also lays the groundwork for other small-file p2p support) is under active development now and is still mostly for advanced users who are already familiar with how hydrus works. There isn't an easy switch to throw and have doujins flow nice and easy again. In future, I'd love to do eventually add IPFS/hash/tag cross-referencing to be able to search IPFS or other file sharing domains using tags and file identifiers harvested from other sources.

I am open to more opinions and thoughts. I am not big into the doujin scene, so I am largely ignorant of the actual mechanics and the overall size and shape of the problem to solve. I saw some posts saying Sad Panda was about 50-200TB. If there is a db out there with all the file hashes and tags and so on, it wouldn't be super difficult to convert it to hydrus tags and import it to a client. Current PTR is 620 million tags covering 31 million files, so if those are the sorts of numbers we are talking about, I think the current hydrus db can handle some jobs related to this at least.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516 No.13326

>>13265

>>13258

Yeah, try system:filetype in a regular search page. You can mix it with any other tags or whatever.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516 No.13327

>>13263

I don't use derpibooru–I assume there is a derpi login script on the github that another user made? If you have imported it ok, go network->downloaders->manage logins and find the entry and click 'edit credentials'. Put in your username/password and hydrus will try and login as you the next time you make a request on the derpibooru domain.

Most of the downloaders in hydrus download as if you were browsing. Again, I don't know if the derpi login script (and accompanying downloader you have) works on an api key, but I imagine it just uses user/pass and simple gallery/file post page requests like most of the others.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516 No.13328

>>13269

What's the problem? Did some go missing?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7b9f2c No.13330

>>13307

ipfs is the solution, and it's being developed, but the timeline is very long and not likely to come out any time soon.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516 No.13333

>>13281

If you like, close the client, go to install_dir/db, and enter this:

.open client.db
SELECT dump_name, datetime( timestamp, 'unixepoch' ) FROM json_dumps_named WHERE dump_type=13;
.exit

That'll give the timestamps of the x most recent 'last session' and 'exit session' timestamps, just in case any are left over from that 07/23 period. It probably won't add anything more useful than that log, but you never know. It depends on your idle settings, but since tag repo processing kicked in two and half hours after boot there, my guess is they opened the client and then for a period they didn't move the mouse, probably starting about 1pm or so, then moved it again at 1:30 to cancel processing, then left it alone again. I assume your client had caught up with the PTR by the 3:41 processing job since it stops there, or maybe they were continually moving the mouse until 18:40 looking through all your stuff, maybe, and then closing the client safely.

If your session does not seem to have changed in your boots since then, the worst the guy probably did is click through your open tabs. If the files (or empty pages) are the same as before, he didn't hit f5 on anything.

In an unrelated note, hydrus runs well from an encrypted partition. Ciphershed is an easy-to-use solution:

https://www.ciphershed.org/

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516 No.13334

>>13333

Shit, should have said, 'run the sqlite3 executable' in the db dir. That'll give you terminal that you can paste those lines into.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516 No.13335

>>13299

It is generally a pain in the ass every time, since it depends on what the exact file format is, and regex is not the most pleasant tool to work with, even if it is excellent.

For commas, something like this should work:

(?<=parentfoldername\\).*?(?=,)
(?<=, ).*?(?=,)
(?<=, ).*?(?=\.)

But I am a bit rusty.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516 No.13337

>>13301

No simple merge system yet. If you only want to move files and tags, it isn't super difficult–basically, export all the files and reimport them to the new db, and hit help->advanced mode and use the advanced service-wide update button on both clients' appropriate local tags panels on review services to export and import a Hydrus Tag Archive of your tags.

If you need to move ratings or complicated (i.e. other than 'everything is archived) archived status, there aren't easy solutions. For inbox/archived, you can split it into two jobs, but ratings have no import/export yet.

Let me know how you get on. I can figure out a hacky fix to transfer the data or maybe some simple import/export finally.

Subscriptions and several other objects should be exportable and importable from their dialogs, either to clipboard or png files.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516 No.13338

>>13302

The menu that has file | pages | database and so on? That should not be removeable in hydrus, but perhaps your OS or Window Manager allows you to clear/hide it? What OS are you on? Can you take a screenshot of what you see?

The shortcuts system is in a mess of several different systems. Some is hardcoded. The new system is available under file->shortcuts, which you will be able to check out once we get your menubar back!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5d1b82 No.13347

is there a requirement needed for ubuntu lts minimal to get the server to run? the client appears to run fine but getting an unhandled error on the server when i close it from the command line. end goal is to move the ever growing db off my pc and onto this box that has way more storage.

when its running i'm not able to connect from another computer and the firewall is disabled ubuntu.

Unhandled Error

Traceback (most recent call last):

File "threading.py", line 914, in _bootstrap_inner

File "threading.py", line 862, in run

File "site-packages/twisted/internet/base.py", line 1267, in run

File "site-packages/twisted/internet/base.py", line 1276, in mainLoop

— <exception caught here> —

File "site-packages/twisted/internet/base.py", line 902, in runUntilCurrent

File "include/ServerController.py", line 407, in TWISTEDDoIt

StartServices()

File "include/ServerController.py", line 333, in StartServices

HydrusData.Print( 'Starting services\u2026' )

File "include/HydrusData.py", line 970, in Print

print( repr( text ) )

File "include/HydrusLogger.py", line 159, in write

self._log_file.write( message )

builtins.ValueError: I/O operation on closed file.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0c232b No.13350

Hi to all,

There are any bad side effects by installing hydrus (and it's archive) into a share program folder like mega or dropbox?

I wanna try to do this in order to access and update my archive from multiple computers

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d2c03a No.13351

>>13312

Feeling sad about this, also https://8ch.net/animu/res/111544.html#q112288 (not using crosslinks to prevent bumplocks)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5d1b82 No.13352

>>13347

Threw a bunch of python modules that it was missing. still not run able.

Unhandled Error

Traceback (most recent call last):

File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner

self.run()

File "/usr/lib/python3.6/threading.py", line 864, in run

self._target(*self._args, **self._kwargs)

File "/home//.local/lib/python3.6/site-packages/twisted/internet/base.py", line 1272, in run

self.mainLoop()

File "/home//.local/lib/python3.6/site-packages/twisted/internet/base.py", line 1281, in mainLoop

self.runUntilCurrent()

— <exception caught here> —

File "/home//.local/lib/python3.6/site-packages/twisted/internet/base.py", line 902, in runUntilCurrent

call.func(*call.args, **call.kw)

File "/mnt/hydrus/hydrus network/include/ServerController.py", line 407, in TWISTEDDoIt

StartServices()

File "/mnt/hydrus/hydrus network/include/ServerController.py", line 380, in StartServices

HydrusData.Print( 'Services started' )

File "/mnt/hydrus/hydrus network/include/HydrusData.py", line 970, in Print

print( repr( text ) )

File "/mnt/hydrus/hydrus network/include/HydrusLogger.py", line 159, in write

self._log_file.write( message )

builtins.ValueError: I/O operation on closed file.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

623fd2 No.13353

File: 43371e74cb75ddd⋯.jpg (29.39 KB,500x541,500:541,43371e74cb75ddd1dbe054d998….jpg)

>>13312

Alright, mate. I understand that Hydrus infrastructure is not quite ready for something like what I have in mind, but as I understand, Hydrus is being developed collectively, isn't it? So what I'm going to do is make a thread to rise awareness of this, so people can have it in mind and help with the development. All you said so far sounds correct, and I have the tags saved somewhere.

Thanks for your reply.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5d1b82 No.13357

>>13352

Nevermind i'm just going to move it to an external drive

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3607e9 No.13359

>>13337

Worked like a charm, thank you! I only needed the tags and the file associations so that was very easy.>>13337

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1d1a8c No.13360

File: 5526854a8a58447⋯.png (658.92 KB,1656x997,1656:997,2019-07-26-105215_1656x997….png)

>>13338

I'm on Debian, using 2bwm and bspwm mostly. I wasn't able to find a way to restore the menubar, but I was able to export all my files with their associated tags and re-import them into the new version of Hydrus, so all's well that's ends well. When I did have the menubar missing, it looked like this. It wasn't an environment thing because other GTK programs had the menubar present and when I tried to copy

db
into the new Hydrus release the menubar disappeared there, but before I did so (and now, after exporting, deleting the db files, and re-importing) the menubar was present.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a No.13363

>>13352

>>13347

>>13357

Thank you for this report. Another user told me that 361 is having trouble starting services correctly–they only seem to start for him when he hits ctrl+c, which then quits the program. I did change this this past week, and it was all fine on my test machines, but something is messed up with the initial event queue for certain platforms. I am going to give it another go this week. Please let me know if 362 works for you, or, if it is convenient, see if 360 works ok.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a No.13364

>>13350

Yeah, I generally recommend not putting the actual db on a remote system. SQLite likes to have some clever hard drive locks, and these remote locations tend to offer 'fake' versions of the lock commands if they even offer them at all. It might eventually cause malformed errors, or it might just not work for you at all.

It is also good to have the actual db on a local system for latency purposes.

You can try it if you like, and I'd be interested to know how it goes, but make sure you have backups of everything, and don't try to run two separate clients from two separate computers on the same db at the same time.

I know several users who enjoy putting just their media files on a NAS or even in the cloud. Keeping the db files and thumbnails locally but putting files on remote storage tends to work ok. Here's some related help talk about your options:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a No.13365

>>13353

I'm afraid it is just me, one lad. I have trouble working in teams, so I have a strict policy of working on my own for hydrus. I am happy to help other Anons with their various situations, but I am always drowning in work, ha ha. I can never get things done as fast or well as I'd like.

That said, the overall push of the past couple of years has been to make hydrus more 'platform' based with the completely user-configurable downloader system and the new Client API. This lets other users fix their own problems without having to wait for me to hardcode-fix the danbooru downloader or whatever. I'd like to continue this trend going forward.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a No.13366

>>13360

Thanks. That's odd. On my end I just make a MenuBar object an attach it to the main frame, and then wxPython and your Window Manager are supposed to handle the rest according to native controls. Some Window Managers put it up on the main OS frame. My best guess is 2bwm was trying to put it into a location wxPython couldn't handle and it bombed out. Increasing compatibility for different Window Managers is a constant fight, so please let me know if you discover anything new here.

>>13359

Great, thanks for letting me know.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f1c4c7 No.13370

If I were, for purely cosmetic reasons, to change the current media destination's location drive letter (it's the same drive, but now E instead of L) how would I go on about that? I can't really migrate the database as it's the same drive. Just tell Hydrus where to find it again via adding a "new" location?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

623fd2 No.13374

File: 5e4571719738bbb⋯.png (1.08 MB,1439x852,1439:852,ClipboardImage.png)

I have a really old version of hydrus. If I update it will I lose all my tabs?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bc2c89 No.13375

Hey,

Thank you for your hard work.

This is an amazing tool.

My question: why not upload the tag database to something like megaupload/mediafire and lets us import it that way, instead of through chunks of 64mb/day?

Is there something preventing you/anyone from doing this? Of course it wouldn't be up to date but that would eliminate a large portion of the waiting from our side /bandwidth from your side.

(I also asked this through tumblr, feel free to ignore my question there)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ad3314 No.13376

>>13375

Already been done here:

https://cuddlebear92.github.io/Quicksync/index.html#two

In Hydrus use services -> import repository update files

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f1c4c7 No.13377

Oh, got another one:

Is there a faster way to get a single file out of hydrus than right click > share > copy > image (bitmap) if I want to post it somewhere?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4 No.13379

>>13374

Ideally no. Theoretically, it should be fine™. A bunch of stuff will change, like all those thread watchers will become multiple watcher pages each with one page, but nothing should break, hopefully. There's a max page limit of about 145 or so, now, and I am not sure what yours currently is, so pare down your session if it is larger than that.

The main thing is make a backup before you do it. If it all goes wrong, you can always roll back.

I think we are talking more than 100 versions. The client won't let you update more than 50 versions at once due to bitrot (I can't guarantee the update will do anything). I recommend trying 10 versions at a time. If you are on 250, try 260 and boot it. If no errors, try 270, 280, and so on. If it breaks, try 275 or whatever, half-way between your jump. Let me know if you run into trouble.

If you are a madman, you can try a couple of 50s or whatever, but whatever you do make sure you have a backup before you try. I know one user just did like 320->360 and he said it was ok, but this is a complicated subject and every client is different.

>>13375

>>13376

Yep: https://hydrus.tumblr.com/post/186658823864/hey-there-thank-you-for-your-amazing-work-i-was

Let me know if you have any problems.

>>13377

I think you can set up a shortcut for that, but it might only work in the media viewer. Try file->shortcuts->media and add a new shortcut for 'copy_bmp'.

However, if you are uploading to 8chan or any other normal site with an upload box, just drag and drop the thumbnail to your browser. It is a lot nicer to send the real file rather than a bmp/png copy. If you are trying to DnD to discord, try turning on the discord BUGFIX option under file->options->gui.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bc2c89 No.13383

>>13376

>>13379

I see, thanks!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ab698d No.13400

File: 22dd349b8ad0e47⋯.png (927.71 KB,1920x1080,16:9,ClipboardImage.png)

Trying to add the public tag repository and I don't see a place to put an access key. Any idea what I'm doing wrong?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ab698d No.13401

File: a032848cb0838f6⋯.png (805.51 KB,1920x1080,16:9,ClipboardImage.png)

>>13400

Scratch that last question I guess, figured it out, but now it doesn't want to connect. Turned off my firewall momentarily to see if that was the problem, but it's not.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7a7385 No.13402

For someone with a ton of images is there some way for me to limit my inbox search to, say 1000 images so everything doesn't slow way down trying load all the images in the inbox?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

98f3ac No.13403

File: 78984882c0fd25f⋯.png (276.43 KB,1200x1300,12:13,ClipboardImage.png)

File: 4a727eab6a1ad73⋯.png (273.89 KB,1034x1320,47:60,ClipboardImage.png)

File: 19a126812845b4f⋯.png (89.25 KB,1040x2028,20:39,ClipboardImage.png)

From /v/'s draw thread.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

98f3ac No.13404

>>13403

someone had requested a mascot for the software.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6aca6e No.13405

>>13401

>>13400

just do the option on the help menu to set it up automatically for you

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ab698d No.13407

>>13405

I did for both before coming here, and it still wasn't connecting.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6ab37a No.13416

File: 220f1f2f49c75ae⋯.png (9.79 KB,468x445,468:445,a.png)

>>13402

System:limit is what you want. Once you click on it you can specify how many files to limit it to.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

12d396 No.13419

>>13407

I figured it out, I was using an older build. New version works just as advertised. Also seems to be downloading only 6000+ tags, will that number increase?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

12d396 No.13420

>>13419

Also, I have all of my images sorted into fairly general but still useful folders, is there a way to have Hydrus keep those images assigned to those folders while uploading?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

155fb9 No.13421

I am very slowly bumbling my way through learning the program and figuring out how to use it properly, and only seven hundred or so images in did I find the automated tag lookup script in the options menu. I noticed, however, that the script does not also pull the urls from the pages of the art, as the more roundabout way I was using does. Is there any quick and easy way to make it so the urls get pulled and attached to the image along with the tags as well?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ab698d No.13422

>>13420

Nevermind, figured that out too, turns out it's an option in file>import folders.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

43c1dd No.13430

File: 541653bd0d0baca⋯.png (132.3 KB,833x360,833:360,hydruschan.png)

File: 96cc024f2cc9807⋯.png (131.72 KB,810x810,1:1,hydruschan2.png)

File: 660e0ae7d387a62⋯.jpg (79.33 KB,1280x959,1280:959,hydrus-tan.jpg)

>>13403

Hey, this is great! Here's some other ones made by users from a while ago, mostly just having a giggle. When previously asked 'What should a hydrus-tan have?', I've generally said it should have an error somewhere, or long UI hangs, ha ha ha. Making her thick works with hydrus having a bloated install and db, I like it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

43c1dd No.13431

>>13421

That's a good idea. That tag lookup script stuff was the prototype for my later downloader engine overhaul, and the way it fetches and parses its results is not great. I'd really like to rework it to use the newer parsing system, at which point it could definitely associate URLs in a sane way. That whole system is also pending an eventual throttled 'auto-check' system that you'd be able to queue up and have work in the background in some sane way that didn't end up eating bandwidth and CPU.

Best way to associate URLs right now is to run a gallery download. A bunch of the site parsers now pull md5 or similar hash and recognise when the client already has the file (and hence don't end up downloading it again, but will get tags etc…), so they work a bit like the lookup scripts anyway. You might like to do a test on a site you like for an artist you like and see if that solves your tag&url problem quicker. Help on this if you are new to it:

https://hydrusnetwork.github.io/hydrus/help/getting_started_downloading.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

155fb9 No.13432

>>13431

Thank you for your attention! That is what I have been doing, feeding the hashes of the existing files I'm importing from my own folders through the downloader to get their tags and urls, the only problem for me is that the process for that is being excruciatingly slow for me, taking upwards of two hours to get through about 150-ish files, meanwhile the tag lookup is nearly instant.

I should assume that is purely a connection problem on my end, yes? And also thank you for all your documentation, it has been an immense help so far.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5f34d9 No.13445

Is there a way to delete unused tag info? Hydrus's two .db files sum to 19.5gb filesize, while my actual images are only 4.4gb. It seems like it's storing a ton of info about tag associations for images I don't actually have. Is there a way to tell Hydrus to go through and prune all data that doesn't match to an image on disk?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4b2510 No.13446

File: c999422e0e0a97a⋯.png (8 KB,287x134,287:134,m4DW4X0[1].png)

Forgive me if I've missed any explicit support for it even, but is there a workable way to parse and download images from Pillowfort blogs, akin to Tumblr support? I ask because a few artists have moved there ever since Tumblr's porn crackdown.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d49cfa No.13447

>>13432

Ha, sorry for insanely late reply. Since you see normal tag searches go fast, I think your connection is fine. There is a built-in limit for 'gallery' URL hits in hydrus, which might be what you are running up against. Normally, I strongly recommend keeping this limit in, as it makes regular booru searches work at a sensible pace, but if you change the setting under options->downloading, under 'gallery downloader' section, the 'additional fixed time…' setting, which is likely 15 or 30 seconds, take that down to 3, and these url queries, if they are indeed waiting on that, should go through quicker.

I wouldn't recommend taking it down to 1, but you can try if you want. Gallery hits tend to be CPU heavy for servers, so hydrus tries to also be polite by not hitting them too frequently. Hash lookups may be ultra-fast serverside, but if they have set up bandwidth banning based on frequent heavy requests to the gallery script, you might get hit without meaning to.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d49cfa No.13448

>>13445

If you are synced with the PTR, unfortunately you need to get all the PTR definitions and data as well, which is about 20-odd gigs now. As you guessed, it gets all the mappings (~700 million at this point!) the server has, so all computation happens clientside and with complete privacy. This tends to be less of an issue if you have a 1TB+ total collection, but for your situation, it is not great.

If you wish, you can delete the PTR, which will clear most of that data after a vacuum call. It will not delete master records, which I have a planned solution for as per here, which I answered earlier today:

https://hydrus.tumblr.com/post/189534434774/ive-been-using-hydrus-and-trying-out-the

>Unfortunately, there is no way to clear out ‘orphan’ master definitions for hashes and tags yet. This is becoming a bigger problem with various larger dbs, so I expect to have it in in the next couple of months. I know what I need to do, I just have to write a decent maintenance routine that won’t gobble up too much CPU. The ideal is to have it work in the background, so you won’t have to do anything–it’ll just shrink your db naturally over time.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d49cfa No.13449

>>13446

I haven't got one in by default. The main repository for user-made download scripts is here:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/Download%20System/All-in-Ones/Single-Sites

Unfortunately, I do not see one for Pillowfort there. I just checked out the site and saw it is in beta and needs a registration fee. Let me know if I missed a way to view as a guest. If you have a login, do you know if they have an API to talk to? Is their browsing experience similar to tumblr? I do not make downloaders myself any more, but if you could provide some example html/json/xml data for the Pillowfort equivalent of a booru's gallery tag/username search and post pages, perhaps the guys who do make downloaders now can figure something out.

If this is a bit technical for you, feel free to get into the discord and see if one of them will walk you through it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291 No.13450

So im getting disk I/O errors because my hdd is presumably dying on me and I neglected to make any backups of my database. I move the entirety of the db over to a new drive successfully except for client.mappings, I get errors whenever I try to move or copy it. Am I fucked?

Also the captcha on this chan is fucking cancer holy shit

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291 No.13451

>>13450

Also I have tried cloning it in sqlite, seems like its getting stuck at current_mappings_10.

The program also still works fine, no tags seem affected though I dont really know what client.mappings handles. The only reason I noticed something was wrong was because I suddenly couldn't process repository updates.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

df8eda No.13455

My yande.re tag parser has disappeared for some reason. Can anyone post an import file for me?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d567bf No.13459

Is there a way to re-import files from my collection at mass in order to change where their tags are stored? I made the mistake of saving all import tags to just the public repository, now im noticing now that a huge amount of images are getting tagged completely incorrectly by it so now everything is jumbled. Therefore I just want to make a new local tag system for specifically the imported tags but re-importing everything manually would take weeks.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d8ca57 No.13463

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d8ca57 No.13464

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d8ca57 No.13465

>>13459

I am not totally sure about your problem here. When you say to re-import the files to get their tags again, did you add tags to your files when you imported them? Was that from regex filename parsing, or neighbouring .txt files, or something else? Or were the files downloaded, and these tags are parsed from boorus? If you were to do this the slow way, what would you be doing?

services->tag migration may be of use here, but if things are too jumbled, maybe not.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d8ca57 No.13466

>>13464

>>13463

>>13465

d8ca57 is me, fuck this multiple imageboards with different BO trip systems, ha ha ha.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291 No.13467

>>13465

Parsed from boorus mostly. The slow way would be to manually download the images again using the urls that are saved with them, one at a time.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

980238 No.13471

>>13464

Thanks! This works when I do a test parse, but when it comes to actually importing a new image from yande.re it's not pulling any tags for some reason. I double checked that the url class link was correct, but still no luck.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

57a349 No.13475

anyone recommend a good external viewer for mac? don't really know whats good

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5fab5b No.13479

How could I go about pulling "date taken" data from my pictures and get that converted to a tag?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

74f865 No.13480

I'm going to merge two different databases together with one being on an old hydrus version.

Is just exporting all the files them importing them with the tags enough or is there a better way to go about it?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5fab5b No.13481

>>13480

that's probably the most straightforward way of doing it

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cc9b37 No.13483

>>13471

I got it. It turns out that my default tag import options had been reset at some point.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

74f865 No.13484

File: fe3e16eb82ae240⋯.png (5.63 MB,2520x3564,70:99,ClipboardImage.png)

Here's a bunch of questions since I just got a SSD and cleaned my HDD.

First I was thinking of migrating everything(hydrus, thumbnails, DB files) but the actual files to the SSD, but I'd rather not bloat it straight away since it's a really small SSD. Is there any way to check the size of everything before moving?

And about merging DBs, I remember reading something about having different sets of tags in the same DB, is that a thing I can do and I'm just stupid or has that not been implemented yet? Can I hide a set of tags by default? And is there a way to apply a certain tag to every file currently in the DB other than manually?

Thanks in advance.

8ch is shitting the bed as usual, pls don't doublepost

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5e009a No.13485

I wonder if it would be difficult to create a thread image ripper like the ones for 4/8ch for other *chans running tinyboard + vichan or whatever 7chan is running? Could one just edit the 4chan one or is it a more involved process?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1d7680 No.13486

Anyone have a GUG for 4plebs? It seems similar to an imageboard setup but watcher, URL, and simple downloader all fail, yet the full images are there when I click manually and are just stored on its own CDN like on 4chan.

Or is there some workaround?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

679f38 No.13488

File: 37cca738f805a78⋯.jpg (81.32 KB,1280x720,16:9,36475866324.jpg)

Is there a way for:

-automatically tagging like "page:1-page:n:" for downloading from booru images, not only imported from local files

-transferring part of the files to another hdd (for example large films)

-handling external subtitle for video

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9f76e1 No.13489

>>13467

Ah, yeah. Hydrus does not remember the origin of every tag, so you can't do a local-only retroactive mining of tags based on tag source. Best way is to set up a new url downloader page and feed the urls in again (doing a right-click on a big ctrl+a of the files you want should let you copy all urls that match a certain site, and then you can paste back into the url downloader). In order to get the tags again (normally hydrus will just skip urls it has seen before), you'll want to set the tag import options to 'get tags even if urls/hash already known'.

Do a test of a dozen files first here, to make sure you have it all set up how you want, and then do a bigger batch.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9f76e1 No.13490

>>13479

Hydrus does not read EXIF data at the moment. I plan to add this sometime and add search predicates and tag conversion for it.

I do not know when I will add this, so if you want to do it now, you will have to use an external program to pull it before import and then convert the tags you want to a newline-separated 'sidecar' 'filename.txt' file to import beside the file. I know some users who have done this, although I am afraid I have forgotten the name of the EXIF extracting software they used. I think they had a system where the software converted to xml files, and they used a script to convert that to newline-separated as hydrus understands for import. If you are not very technical, it may be a pain in the ass, but I am happy to help if you want to give it a go.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9f76e1 No.13491

>>13480

>>13481

Yeah. You might want to do it in two batches: one for inbox files, one for archive, so you can quickly regenerate that information in the destination. If both clients have services->tag migration, that's the best way to import/export tags atm, using Hydrus Tag Archive files.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9f76e1 No.13492

>>13484

database->migrate database has good estimates of db size, ok estimates of media size, and ok estimates of total thumb size. If the SSD is very tight, you might not want to move the db files–they need a drive with free space to work well. I recommend 1-2 times as much free space as the size of the largest of your four db files. This is no big deal if you don't sync with the PTR, but if you do, your client.mappings.db is probably 18GB or so.

You can now create multiple local 'my tags' services. Hit services->manage services to add more. The new services will appear in any place you can add/edit tags as separate panels or tabs.

services->manage tag display is a new-ish panel that lets you hide tags for single services or all services.

There is no way atm to apply a tag to all files in a file domain. I recommend you set up a page with:

system:limit=1024

-the_tag

And then do several rounds of f5-ctrl+a-f3-ctrl+v-enter-enter. You can push the system:limit a bit higher to do it in larger batches, but you'll take an inefficient CPU hit dealing with too many at once. If you have 100k+ files, this may be too much work.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9f76e1 No.13493

>>13485

I keep meaning to knock up some quick parsers and url classes for the bunkers, and then I do not find time for it. It would not be super difficult for anyone familiar with the download system. Most imageboards offer good APIs or nice html, and as you say, if you have a parser for one vichan board, you probably have support for 95+% of them running the same software.

8kun is difficult because of the DDoS stuff.

>>13486

I don't know anything about it, I am afraid. How do they fail, is it a 503? If they have aggressive CloudFlare protection, this is a common cause of hydrus not being about to get files when a browser can.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9f76e1 No.13494

>>13488

- Not yet. I would like to add this some time.

- Try database->migrate database to set up multiple storage locations. It allocates files across multiple locations randomly according to set location 'weights', so if you just have a couple of multi-GB files, you can't guarantee it will be smart about putting them in one place, but as your collection increases in size, it will be more accurate.

- No support for this, unfortunately. I am not sure there will be a neat solution in the near future. You are free to do what you want, but I recommend you not use hydrus for movies. I designed it for imageboard-style files, where you need to keep track of hundreds of thousands of smaller files, so most of the workflows and maintenance timers are built around that. Files that have sidecars just don't work well with the hydrus storage system.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

04beb1 No.13496

is it possible to delete the deleted files record for specific files, rather than the entire database? I know I can do the latter via `services > review services > local > files > all local files`, but I want to keep the whole deleted files history, except for maybe ~120 files worth.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1dd1e8 No.13497

Alright I think I'm an idiot. I'm trying to download the images from an artist on pixiv, and despite having other pixiv downloads that work this one just doesn't and I can't figure out why.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

37bed4 No.13499

File: 1887535f707ca9b⋯.jpg (108.84 KB,838x1064,419:532,__shark_girl_(yukimoto_shu….jpg)

I want to use Hydrus to manage an existing collection of pictures downloaded from boorus, and nothing else. If I only want to get tags from those boorus instead of importing the entire PTR, is this https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/wiki/2.-How-to-fetch-tags-in-bulk-with-md5-hash the best way to go about it?

There was also something about Hydrus tag archives in the manual, but the archives linked there are several years out of date.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d9a9ce No.13504

I would really like to use hydrus, but I feel I'm losing the browseability of folders. How do you "browse" certain core tags easily?

Like, right now, one folder I browse frequently is based on specific characters where the directory structure is: "my-images/series-name/character-name/". I've imported a bunch of these folders and given them tags, but now I miss the ability to see, at a glance, which series I might want to browse.

I figured out that in hydrus I could mark certain tags as "favorites", which is a start. With my old folders I could quickly navigate the hierarchy using the enter+backspace keys to go up/down and arrow keys for moving around, and I can't do that with hydrus tags. It's also clear that the list of favorites would be unwieldly at some point.

Let's forget the hierarchy for a moment. Each character is a nice conceptual "album" and it'd be great if I could just browse my "character" tag and easily switch from one character to another. I can double click on a favorite tag to bring up that tag search, but if I double click another tag it does an "AND" search…I first have to navigate to another part of the window to remove the first character's tag.

So I'm stuck. I really want to move to tags though so I can easily browse by artist or other tags which cuts across my "/series-name/character-name/" folder structure. But I want those tags to provide the same browseability had I used a "/artist/character-name/" folder structure.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291 No.13507

>>13504

I personally have a bunch of tabs set up with tags to organize some my db without having to search every time. For example, I have a blanket tab for all 2d art which then has other tabs in it that are tagged for whatever else, including an Artist tab that then has a bunch of tabs that are organized by artist. It wouldnt be hard to do the same for what you want, though it is maybe a bit less convenient than a simple folder structure though you gain way more than enough to make up for it.

Speaking of, a feature i'd really like to see is the ability to add a "master tag" to a tab so that that tag is required for all searches in that tab and all others that are in that branch. Would make it a bit less tedious to organize that way.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291 No.13510

>>13507

And though im not sure its within the scope of the program, actual folder-like functionality would be great to see. The way I would imagine it would essentially just be creating tabs and giving them a presence in the actual gallery screen instead of just the tab bar, while being able to navigate through them like an actual file explorer.

Organizing everything through tabs is kind of hard on the eyes and awkward to use if you have a bunch of them, so this would be a godsend. I'd probably lose some time going full autist and setting up my own folder structure to organize absolutely everything, though.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d9a9ce No.13511

>>13510

The way I would imagine it is that you can specify a folder structure in a separate input box, below the input box for the search string. So you might say:

folders: <namespace1> / <namespace2> / <namespace3>

which would automatically populate a virtual folder structure with the results of your search. Items that were returned by the search but don't have a tag for namespace1 would be put inside a 'misc' folder at the bottom. (and recursively, there would be such 'misc' folders inside each level of the hierarchy)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

74f865 No.13531

File: d0ade66e08166da⋯.png (8.19 KB,704x91,704:91,a.png)

Is there any way to make the duplicate filter value pngs over jaypegs? I tried to set it to negative but it only goes as far as 0.

Also, since when was this implemented? I just found it now when searching for backup options.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a3189e No.13546

I accidentally deleted a tag, my intention being to replace it. Using the show deleted option in the tags menu, I can still see the tag, but it isn't applied to the files it once was. Is there any way to re-apply a deleted tag to files?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8adcb7 No.13552

new to hydrus, is there a way to change a namespace's color? I added two new namespaces recently and they both were the color of one I already had.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

84886b No.13553

I am unsure on whether to post this here or in the bugs thread, but after updating the program to the latest release, I noticed that I no longer can press Ctrl+C to easily copy the file's path. However, the hardcoded shortcut section still references that as the shortcut to copy files into the clipboard. Is this change by design, or is it truly a bug?

>>13552

If I'm not mistaken, you can do that by going through File > Options > Tag Presentation

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0cde32 No.13555

Where is the place with the most current info on getting downloaders to work? I am searching but I keep finding outdated info.

I am not good with computer. I just want to download every single picture of my waifu on the entire internet.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2900fc No.13561

Hey lads, apologies again for lateness. New schedule calls for me to check this four times a week, let's see if I can stick to it this time.

>>13496

Under options->files and trash, turn on the advanced file deletion dialog. It provides an option to delete a file without leaving the deletion record.

If the files are already deleted, there isn't a way to do this I don't think. Maybe you can do something clever like searching for the files in 'all known files' domain and then re-deleting them. You know what, I'll make a job to add a 'clear deletion record' thing to the right-click menu for these files.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2900fc No.13562

>>13497

As I understand it, Pixiv now prohibit nsfw results from non-logged-in searches, and the hydrus pixiv login script silently broke some weeks ago, I think due to them adding captcha. Please use Hydrus Companion to log into pixiv now:

https://gitgud.io/prkc/hydrus-companion

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2900fc No.13563

>>13499

Yeah, that md5 lookup process is decent. The other solution is the Public Tag Repository, which I specifically designed to solve the problem, but if you are new to hydrus, you might want to hold off on that for a bit. The PTR is now a sledgehammer, and if you only ever want to tap out tags for a couple thousand files, it might not be right for you.

>>13531

I folded that into a changelog a little while ago, I think. I basically just exposed the internal values I was using for the score to users.

I will make it so you can value pngs over jpegs. Out of interest, can you talk a bit about your situation where that is useful? Is this for, say, a set of patreon rewards, like 'raw' ultra-high-quality png versions? In general, most users I have talked to about this tend to have many more situations where the png is some clipboard.png imageboard copypaste post of the original jpeg, so they generally prefer jpeg to be the higher weight be default.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2900fc No.13564

>>13546

Some deleted tag storage and workflow is borked, so let me know if this doesn't work:

Open up the file(s) in manage tags dialog, set to 'show deleted', and then double-click the 'deleted' tags that have '(X)' after them. The double-click should apply a re-add action.

I think you can also just try to add the tag as normal, and the delete will be overwritten.

I am assuming this is for 'my tags' tags, and not something like the PTR. Undeleting tags for a tag repository is more complicated and really needs a janitor.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2900fc No.13565

>>13553

Ah, thank you for this report. I am pretty sure that is old–I think all the copy stuff is now under file->shortcuts->media, with action names 'copy_(bmp/file/path/sha256_hash)'. I'll clear out that old text.

>>13555

Here's the main repository:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/Download%20System/All-in-Ones/Single-Sites

Open up Lain in network->downloaders->import downloaders and drag those pngs on her. It isn't always perfect when there are object conflicts, but it generally will import everything automatically.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291 No.13569

So I have a couple questions here, first I was wondering if its possible to automatically add a specific tag to every imported file that DOESN'T have another specific tag thats parsed from wherever im importing the file. For example, I'm trying to categorize my collection by 2d and 3d files, 3d is usually tagged on boorus but 2d usually isnt so its kind of a pain doing it manually.

Second, how do I completely delete the ptr so that I can re-download it? Im fairly sure my ptr is corrupted because a lot of tags are mixed around where they shouldn't be. I've tried deleting it from the manage services menu and making a new one but that just processed the repository updates that ive already download again and doesn't fix the jumbled up tags. Dunno if re-downloading it would even fix the problem but I figured i'd try.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

389651 No.13570

Is there a way to clear all previous deletions without reason?

I've done some fuck ups in the past and now that there's deletions explanations it's a good time to unfuck my hydrus.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

21bc67 No.13571

>>13569

I don't think you can do this 2d/3d tag logic atm. My best suggestion for now is to have a search page set up with:

-2d

-3d

(possibly system:known url=somebooru.org)

i.e. 'all the files that are from site x that don't have 2d or 3d)

And then once a week or so go:

F5

Ctrl+A

F3

Type '2d', hit enter

Ctrl+R

On it, to do this manual job in a single batch.

For the PTR, I am generally confident that your PTR is not corrupt, but the tags were applied to the file incorrectly due to human error or a mis-parse. Sometimes, for instance, a website changes its layout and a bunch of 'favourite' or 'users also liked these tags' get sweeped into a bunch of files, which slowly gets corrected on the hydrus side from user petitions and janitor approval. Simple mistakes are fairly common, and the chance of a PTR file being corrupt and that not being noticed by the client during processing is infinitesimal, so my bet is on the former. The client knows exactly what the update files should be, and if there is a single byte difference from what the server has, it throws all sorts of errors.

If you notice completely the wrong tags, with no correct tags, on many many files, particularly if it is the same kind of swap (like everywhere you expect to see 'series:touhou', you actually see 'character:samus aran'), it is rarely possible that your db has switched up some master tag definitions, possibly due to a hard drive fault. Can you provide an example hash of a file that has bad tags, and what tags you see on it? Right-click it and hit share->copy->hash->sha256. I'll check it my end and let you know if I see radically different tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

21bc67 No.13572

>>13570

I do not think so. If you are feeling brave, and you are not connected to any file repositories, and you absolutely have a good backup in case this goes wrong, try running the sqlite3 executable in your install_dir/db directory and copy/pasting each line here in turn:

.open client.db
delete from deleted_files where hash_id not in ( select hash_id from local_file_deletion_reasons );
.exit

Then boot the client. Do a sanity check, a test import on a previously deleted file or something, just to make sure everything seems good. Hang on to the client.db backup just in case this screwed something up and we need to reimport its old data.

Oh, and run help->debug->data->clear db service info cache, which will force your client to recount deleted file numbers next time you look at them.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

74f865 No.13576

File: 330029baa87ad33⋯.png (433.97 KB,707x900,707:900,330029baa87ad33816894e99f3….png)

>>13563

>I have talked to about this tend to have many more situations where the png is some clipboard.png imageboard copypaste post of the original jpeg

I scrap boorus and yiff more than image boards, so pngs are usually actual pngs instead of Trojan jaypegs. I like pngs better since they're usually cleaner even when scaled down, I only opt for jpgs instead if the resolution difference is big enough to make it worth it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

100de7 No.13597

Do parent tags do anything if you have "auto-add entered tags' parents on add pend action" disabled? I disabled it because I don't like how it will add them if you enter the child tag, but it won't remove them if you undo it before clicking apply (for example, if you entered the child tag by mistake) and that's caused me to apply non-applicable tags to files on a few occasions without me noticing. I figured that turning the option off would instead add the parent tags once you actually click apply, but instead, it seems to just ignore parent/child relationships entirely now.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

be64c5 No.13598

>>13576

Thanks, this is interesting.

>>13597

Yeah, at the moment it only applies on add/pend actions, which means 'add a tag to my tags' or 'pend a tag to a tag repository'. 'delete a tag from my tags' or 'petition a tag from a tag repo' actions, which remove tags, do not apply parent associations at any time.

The reason for this is hydrus generally doesn't know why a tag was added. So if you had the situation of:

add character:asuka

(series:evangelion added as a parent)

add character:rei

(series:evangelion would be added, but it is already there, so no action)

'oh, actually rei isn't in there, I'll remove it'

remove character:rei

series:evangelion is not touched

You don't want to remove 'series:evangelion' because it could be there for legit reasons, and the code isn't clever enough to work through the human reasons for it being there.

Potentially, I could improve this logic by, say, only removing a parent if there are no valid children, or adding a history to manage tags dialog so it remembers that a tag was added by parent implication and not direct human action, but this is a complication on already quite complicated behind-the-scenes logic. I leave it to the human for now and will revisit it when the siblings/parents system is more intelligent.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

de39ab No.13623

Is there any way to export .jpgs files with the tags embedded in them so you could view said tags with windows explorer or android or other programs? If anyone would know a way to add .jpg tags from .txt files that would also be grand.

Is there a way to disable certain tags/namespaces from being written to the .txt on export?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4455e0 No.13626

File: 2bfe1c61cf7f184⋯.png (19.77 KB,1615x116,1615:116,ClipboardImage.png)

I'm trying to import https://www.deviantart.com/karbo/art/A-snack-in-a-trap-828646610 via hydrus companion but it's not working, any idea why? All other deviantart posts import just fine

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

afca12 No.13627

Just added Hydrus Companion add-on to my Firefox, and I can't seem to figure out how to make the add-on extract tags from, say, Pixiv when sending the page to Hydrus. From what I'm seeing in the settings and advanced features it requires putting in your own regex or something, and I have no idea where I'd begin with that. Is there an easy way to just allow me to directly pull tags from pixiv when I send a page to Hydrus?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5b632f No.13629

File: 65d82677eabcb7c⋯.png (51.08 KB,852x420,71:35,ClipboardImage.png)

File: 7dca5ede429008c⋯.png (28.62 KB,918x227,918:227,ClipboardImage.png)

My Windows updated and I can't run hydrus anymore, what could it be? This appears on the logs. The second one is the error if I try running client.py through cmd

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9d7547 No.13636

>>13623

I know there are programs that can do auto population and scraping of NTFS metadata, but I do not have any personal experience so I do not know any application names. Other lads who have done it needed to write scripts to convert .txt to/from xml files.

Yes, unfortunately you cannot filter the tags exported to .txt yet. It is on my list.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9d7547 No.13637

>>13626

I am afraid it works for me, which is actually a surprise because I understood the hydrus DA parser/login stuff was having trouble recently, particularly with nsfw–can you say more about the error? What happens when you try to import, does it give 'error' or 'ignored'? If you check the import list on the downloader, does it have an error in the 'note' column? Can you right-click->copy and paste that here?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9d7547 No.13638

>>13629

Did you upgrade from a very old version? I forget which one it was, maybe around 330, I jumped from 32 bit to 64 bit, and those errors look a bit like that. The instruction back then was to do a 'clean install', which sounds simple but you absolutely have to do it correct:

1) Make a backup.

2) Delete everything in your install directory except the 'db' directory and its contents.

3) Install as normal.

This clears out the old 32-bit dlls from your install directory while leaving your database intact.

If you are updating from a very old version, more than 50 versions at once, you may have trouble even if you can get it to boot. If you know what version you were before, I recommend only updating about ten versions at a time. So if you were 325 before, try doing 335 and then booting the client. If that works, then try 345. And make sure you have a backup, just in case something goes wrong.

Let me know how you get on, I am happy to help.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9d7547 No.13639

>>13629

Shit, I just read your post again–you updated Windows, not Hydrus. Sorry, can you explain a bit more about which version of Windows this was? I am not sure what is going on here at all. A clean install may help nonetheless.

Have you ever set up hydrus to run in compatibility mode anywhere? Was it set that way maybe by the Windows update?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5b632f No.13641

>>13639

I updated an older version of windows 10 to a newer one, so I don't really know, it doesn't seem to be running in compatibility mode. I tried doing a clean install and the extract-only zip but both give me the same problem.

I have python installed and tried uninstalling it from my computer but that didn't really seem to fix it. All of the other software I had on PC works fine after the upgrade, the only one that gave me problems was hydrus.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4455e0 No.13644

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4455e0 No.13645

File: e2fde8865cde5e7⋯.png (22.41 KB,812x242,406:121,ClipboardImage.png)

>>13644

>>13637

oh right, actually all NSFW posts on deviantart get ignored like this, I have cookies though

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5b632f No.13655

>>13629

Okay, so I managed to fix it. I had to install the windows media feature pack ( https://www.microsoft.com/en-us/software-download/mediafeaturepack ) because my new windows is this N version. Just leaving this here in case someone else has the same problem.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b22eff No.13674

>>13645

Ah shit, I just logged into DA in my hydev browser and got a 'we just updated our layout, here's a video to explain the changes' thing. I wonder if they have done what pixiv did a while ago, which is to roll their new layout out to usernames at different times, perhaps as part of a kind of A/B test. If the new layout breaks the hydrus DA parser, then it will start failing for users at different times as they get switched over.

I am not sure if this is true though, since it only seems to be hitting you for nsfw, and I am still ok, I just tested it again.

I am not a DA user IRL. I recommend you let these downloaders/subs get their 'ignored' status, to keep them in memory. Once we have this figured out, all downloaders should have a button or menu option somewhere to 'retry ignored', at which point you will be able to retroactively get the missing files here. It may also just be some weird cookie sync thing that'll fix itself in a couple weeks. Please let me know if you discover more about this, and I will keep an eye for more user reports to see if this is a more widespread issue.

If I didn't say, have you tried clearing your hydrus cookies for DA under network->data->review session cookies and then resyncing with Hydrus Companion? Maybe there is an old 'hasn't clicked nsfw age-gate' cookie still in your hydrus that is tripping things up.

>>13655

Thank you for this follow-up. I had never heard of 'N' edition before. I will write this down for future lads who run into the same thing.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3e57fb No.13684

>>13674

might be something with my firefox, looks like all cookies, not just from DA stopped being imported correctly (they're empty) but hydrus companion dev tried with fresh profile and it worked so something on my end probably

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3e57fb No.13685

>>13684

nevermind, managed to import cookies correctly but still get the ignored status

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

21c58e No.13690

>>12641

is there a way to query deleted files from the client API?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f7ca0f No.13693

File: 84e6b23e39eed9e⋯.png (28.76 KB,440x339,440:339,Screenshot_20200222_001807.png)

Hi, I have a few neewbie questions:

1. Just installed Hydrus, downloaded and imported 2 GiB of updates. Processing is almost finished by now, pic related shows the directory sizes in …/conf/Hydrus/. So, my virgin Hydrus with no media imported consumes 30+ GiB of disk space. Is it really that big from the beginning? Should I maybe perform some cleaning?

2. …/conf/Hydrus/client_files contains 4 GiB of files with the names like "da2e0e38098fc73502574fbba8832ce7a6f6a88c475ff33e53c152246b0e2afa". What's that?

3. Archive with the repository updates is called 1PTR_UPDATE_FILES-20200216T131032Z-001.zip, so apparently it's from February, 16. I expected that it should cover all the data I need to download, but after the import was done services→review services→repository sync→downloaded was about 70%. So I still had to spent few days waiting while the rest will be downloaded from the repository in 512MiB chunks. Is that a normal situation? Did I do something wrong?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

000000 No.13695

Some faggot has spammed the "female" tag on thousands of unrelated images to shit up the PTR, any way to deal with him besides manually petitioning tags?

At this point I'm pretty sure it's intentional sabotage and not just a broken parser script.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8d0ee0 No.13701

>>13690

Only what is available here:

https://hydrusnetwork.github.io/hydrus/help/client_api.html

I don't think you can do deleted files yet, and now I look, I don't think the file metadata call does file service status (like 'is in my files') yet. I expect this to be added as I work on the api.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8d0ee0 No.13702

>>13693

1) To keep the client fast, network resources low, and to improve user privacy, the Public Tag Repository shares all tags to all users. Essentially your client is mirroring all its data so all work happens clientside, on your CPU. The PTR has been running 8 years or so and now has about 750 million tag-file pairs, which is about 28GB. It will grow a little bit every day.

Adding file import data to the client does not add much space. My IRL 1.7 million file client is 30.7GB total.

Future improvements of this system will allow you to better filter what tags you get from the PTR, I estimate reducing total db size for users who want fewer tags about 67-90%.

2) The extensionless files are the update files your client downloaded from the PTR. They are kept in case the data needs to be reprocessed. If you are interested, they are zipped json, mostly very long lists of numbers.

3) Is this from QuickSync? I do not know enough about the QuickSync archives to talk cleverly about them. It may be missing some. I do not think you did anything wrong. The client will figure out what it needs and fix you up automatically by downloading anything missing and doing more processing when ready.

As you did a very large processing sync all in one go, I strongly recommend you do database->maintain->analyze->full. It will take a bit of time but will ensure that your db is optimised and running fast when you start importing files. Normally, this optimisation happens naturally over time as a client processes the PTR over weeks, but the QuickSync archive tends to happen a bit faster than it kicks in, and now you have hundreds of millions of rows here, that can accidentally lead to extremely slow searches for a couple weeks unless you run that maintenance routine.

Let me know if you have any more trouble.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8d0ee0 No.13703

>>13695

The main janitorial team for the PTR are on discord, if you would like to talk to them directly:

https://discord.gg/3H8UTpb

Best solution now is to petition the tags as you see them. If do not want to go to discord, I recommend you set a 'reason' for the petition with the basic problem here, and the janitorial team will get on it if they do not already know.

There are not excellent janitorial tools for hydrus servers yet. As the PTR grows and more users use it, I expect to improve how all this works. At some point, users may well be getting their own access keys, so bad uploaders can be cut off.

These tags could be malicious, but when I ran the PTR, accidents due to mis-parses or ESL (English as Second Language) confusion vastly overwhelmed any other problem.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

74f865 No.13706

File: e91af8de30ff506⋯.jpg (2.32 MB,1958x3321,1958:3321,64d222943d5e00e4e1e9908b67….jpg)

Is there any way to force redownloading files?

There's a few pictures that are broken due to HDD failure, but the only way to redownload them is to permanently delete the file then either import it or redownload from an URL ignoring deleted files.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

aeee7a No.13707

How does Hydrus handle continuous downloading of resources with anti-chronological post order? Such as the boorus where the recent posts appear on page 0.

I mean, suppose Hydrus has downloaded X pages and was stopped, in the mean time Y new images appeared, so now when it tries to resume from page X it won't be at the same spot where it left off.

Is this all accounted for and I shouldn't worry when making my own parser?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

74f865 No.13708

>>13707

It checks the pages in order and saves the URL it finds to download. Accessing a page is pretty quick so it isn't any problem unless you stop at X page while there were X new pages of content, it'll ignore the old files but download new ones.

Say there's 5 files per page, you download two pages and stop, while you're not downloading 15 new files get uploaded, it'll continue from that second page next time, meaning it'll skip the first 5 recently uploaded and download the Y files from then on, including the ones you already downloaded.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8ab7c4 No.13719

Is it possible to delete all PTR data from my local database? I just use hydrus to organize my reaction images and realized I don't really want 25+gb of tag data from the PTR on my hard disk

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

62e53e No.13722

>>13706

Thank you for this report. For hard drive failure, there is: hit database->maintain->file maintenance->review scheduled file jobs, then go to the 'add new work' tab.

You add jobs to files as selected by a normal search, (so sometimes you might want to say 'all jpegs' or something) but since your problem probably hit all files randomly, you probably want to queue up 'system:everything' (or if you do not see that, do this for inbox, then archive) and then hit 'run this search' to load those files to get the job.

In the 'add job' section, make sure 'if file is missing/incorrect and has url, move file out and try to redownload' and then click 'add job'. Then, in the background, your client will check every file in your db that it is what it thinks, and if it is not, it will queue it up in a new download page. Depending on the size of your collection, this may take a very long time and take a whole heap of HDD and CPU since it needs to load and think about every file you have. You might want to break it into batches or pause/play it under that same 'file maintenance' submenu only when you are happy for it to go.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

62e53e No.13723

>>13707

>>13708

Yeah, if you are making your own parser, as long as the site you are parsing provides files in 'newest first' order, you are good. You only need to concentrate on pulling urls from each page and then any available 'next page' url. Subscriptions have a whole bunch of logic to think about this stuff and decide whether they need to get the next gallery page or if they have caught up to where they were on a previous sync. Then only requirement is that the site serves files newest->oldest like a typical booru.

Note that regular download pages are one-shot and do not do this sort of syncing logic. You can have any kind of 'highest rated first' or whatever sort order and it doesn't matter–these just can't be turned into reliable functioning subscriptions.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

62e53e No.13724

>>13719

You can delete most of it. Hit services->manage services and find the 'hydrus tag repository' entry for the ptr and delete it. It may take the dialog a minute or two to close as it finishes the big delete job, so give it time if it hangs.

Then, when that is done, it is usually a good idea to restart the client. Then hit database->maintain->vacuum->full, which again will take a few minutes, but will recover your hard drive space.

Unfortunately, the 'master' definitions are not currently removed, so your db will still have a ~7GB client.master.db. I have a plan this year to write maintenance code to recycle this info and recover the space, but it is not in yet.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6bd03e No.13746

File: a9387c57f50bec0⋯.jpg (46.42 KB,1365x124,1365:124,4745745.jpg)

How do I set hydrus to aline my thumbnails again? Can't remember what update changed this but I remember hydrus was able to aline thumbnails perfectly whenever I clicked on or highlighted an image in the top row. I could never describe it good enough to ask but it became a little annoyance to me.

Another annoyance, whenever I switch between pages/tabs. Say I selected a certain number of images that I want opened to another page/tab. If I delete any images for the new 2nd tab, the 1st tab will reset the scroll bar back to the top. If that makes sense. Before it didn't used to do that and just kept its scroll position like when switching between any other tab and now it doesn't. So basically I want it to keep my scroll position like before.

Both these little annoyances started happening after the last big hydrus project/update so I'm not sure if their was a setting or something I missed.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3e79ae No.13751

File: 595f8f89ce34246⋯.png (97.9 KB,860x937,860:937,processing.png)

I updated by about 20 versions to version 386. I remember there being some changes associated with the public tag repository around the time that I started my old version of Hydrus. I had been able to sync with it, then it stopped working and Hydrus notified me that I had to update to a new version in order for it to keep it synced. I have only updated two weeks ago so I have a lot of processing of the PTR to do, apparently.

My questions are: What is the nature of this processing? What does a single update correspond to, and what does a single definition correspond to? Why does it only do a double digit number of rows per second?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3e79ae No.13752

Sage for doublepost. I looked into the help files, especially help/database_migration.html. Fucking hell, it was worth taking the effort to move Hydrus to my SSD and keep the media on my HDD. It's now processing 100x faster.

I have some feedback for that help page and the process in general. It seems to me that moving your media files and their thumbnails away from the Hydrus install directory is a more common use-case than moving the SQLite database away from the Hydrus install directory, but that page places the 'informing the software that the database is not in the default location' heading first. I was getting confused while trying to figure out what to do, thinking that I'd have to pass it a command-line argument at some point. But if you set the non-portable locations of your media files and your media thumbnails and move those files, you can afterwards just close Hydrus, move the entire Hydrus directory from your HDD to your SSD, open it again, and it'll just work. Shouldn't that be the thing that is emphasised on that page, and the detail about moving the database to a non-default location should come last? I also found the thumbnail location override setting confusing, because the thumbnail directory path is listed in two places thanks to that setting, and because I don't see why specifying the thumbnail directory's path should work any differently to specifying the media file directory's path(s).

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

aeee7a No.13753

File: 3c581d01a076370⋯.png (24.72 KB,794x349,794:349,ClipboardImage.png)

File: 29808ea66330da5⋯.png (15.3 KB,455x267,455:267,ClipboardImage.png)

How do I parse this kind of JSON in my custom parser?

The server returns either a photo object with type=photo and more data in a subtree by the "photo" key with URLs to download images, I can handle that fine. (first image)

Or it can return a "doc" type object with "doc" subtree with an URL to download it. The doc can be any file, I only want to download it if has "ext":"gif". However the URL doesn't have the extension inside of it and I don't know how to do a check by a different key before passing URL to the downloader. (second image)

I tried to make a Veto rule which looks up .doc.ext=='gif' and vetoes if it doesn't match, but then it won't download the regular photos, since they don't even have the .doc subtree.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1e64f No.13755

>>12641

what's the best way to subscribe to an entire booru site? I'm training a resnet to score files based on my preferences so I'll need to do hardly any work. If I try to do this in a subscription will my client catch syphilis and die? I was going to just do sort:id:asc as the query in the gallery downloader and slowly raise the file limit, occasionally deleting the query when it becomes too big and incrementing the start id like "sort:id:asc id:>10000". Also, since I'd be downloading *everything* I wouldn't want to grab the latest stuff first and accidentally download something weird the moderators haven't purged yet.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1e64f No.13756

>>12641

>>13755

also, regardless of how I setup syncing, I'm probably going to download deepdanbooru2019 first to speed things along and use the client api to import the urls + tags -> hashes

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1e64f No.13757

>>13752

>Fucking hell, it was worth taking the effort to move Hydrus to my SSD and keep the media on my HDD. It's now processing 100x faster.

intel ssd optane 900p is literally 10x faster than an nvme SSD so if you gotta go fast you could throw your db on a $300 optane

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8 No.13759

easily the most 'tism pleasing program i've ever encountered. magnificent software is magnificent. also at the risk of sounding gay as shit the "Act from need, not toward purpose" line in the help pages was profoundly impactful. super nice little unexpected bonus, that. all my gratitude are belong to hydev on both counts.

regarding questions, am i ok updating version 370 -> 380 -> 387 or should i hit 373 first and go from there? does the move to qt mean anything for the 10max update interval?

next, i can't seem to get a downloader for 4plebs working. test parse all checks out. keep getting file ignored error 403 no matter what i try or how i tweak things. halp?

and lastly, could you take a look at the code which manages tag parents/siblings via the context menu next time you're poking around in there? specifically the part that preloads whichever tag was right clicked on so that it shows up as already entered in the left-hand column? dunno if it's the preloading that does it or the fact that it's present on every tab/service but those mappings set up via the context menu have a tendency to end up getting cross applied to other services. kinda seems like maybe stuff like "both fields required for new pairing" and/or "don't auto apply uncommitted changes" is getting bypassed somehow.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

63c965 No.13768

I may be the world's worst googler, I apologize in advance. Has the nijie login script been updated lately? Since the recent changes to it none of the 3 I have are doing the job.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6c6282 No.13778

File: 0972dfbaf2bcbf7⋯.jpg (143.71 KB,1773x1468,1773:1468,0972dfbaf2bcbf75849637d0ba….jpg)

Is there any way to, when importing from a Twitter URL, to generate a "creator:" tag with the username?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

18488b No.13779

>>13778

add this to the twitter parser


[30, 4, ["creator tag", 0, [27, 6, [[26, 1, [[62, 2, [0, "div", {"class": "user-actions"}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]]]], 0, "data-screen-name", [51, 1, [3, "", null, null, "example string"]], [55, 1, [[], "parsed information"]]]], 0, false, "creator"]]

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

63c965 No.13781

Has the OR and AND dialogue been replaced completely? If so, how would I go about using OR in a search? My old tabs using it are still around, but I cannot make a new search using it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

340098 No.13782

>>13779

Works perfect, thank you friend!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

288a15 No.13783

>>13781

hold shift while pressing enter on a search tag to add it to an OR search, or you can click the OR button on the right of the autocomplete popup for a interface for an advanced OR search.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

de0135 No.13784

Does anybody know how I can add the login details for e621?

I've tried some Q&A threads but I can't find the details on how to do this.

I tried the danbooru login script but it doesn't work, I get:

The login for "e621.net" failed! It will not be reattempted until the problem is fixed.

The failure reason was:

Missing cookie "_danbooru2_session"!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3e79ae No.13786

File: 01a746c070aea18⋯.png (18.26 KB,205x295,41:59,fileextensions.png)

What's with these files in my database folders that have no extensions? Adding the usual image file extensions didn't make my image viewer recognise the file as an image, and searching the hash directly didn't find anything either. Are they repository updates from the public tag repository? I did notice that I have 31034 files under 'all local files', 23018 under 'my files', 7979 under 'repository updates', and 37 under 'trash'. The first number is the sum of the rest.

Speaking of which, how exactly do these repository updates work, and why do I have 7979 of them?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8c7a30 No.13788

>>13786

>Are they repository updates from the public tag repository?

Yes

>Speaking of which, how exactly do these repository updates work, and why do I have 7979 of them?

As new tag mappings are added to the repo, they server creates update files and they are downloaded and processed into your DB.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

43d1aa No.13790

>>13784

Yes seconding this e621 is useless without login! HOW DO YOU LOGIN?!>>13784

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.13793

>>13746

Should be options->thumbnails, "EXPERIMENAL: Scroll thumbnails at this rate…". It'll tick down one thumbnail height at 1.0. Also there is a thumbnail % visible scroll check, which might actually be what you want here–set that to 0% and it'll always scroll.

EDIT: Seems like that isn't working right? I'll look into this in case it is a bug. Might have been Qt that changed that, or some other thumbnail scroll-to stuff I did recently.

I have heard about the 'scrolling to top' issue. I did do some work on this,but I will check this as well. If you are on an older version than current, updating may fix this.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.13794

>>13751

tl;dr: Let it work, it'll take about a month of background work in idle/shutdown time to catch up. Read EDIT below.

I don't have time to do full technical explanation, but: The tag repo sends all tags to clients, basically mirroring the server db to the client, so that all work happens on your hard drive and your CPU.

Every day or so, the server generates compressed update files of all the changes in the past cycle. These are zipped up json, mostly lists of numbers. Then this portable data needs to be integrated into your client, lining up serverside file and tag definitions with what your client knows, and then adding the actual file-tag pairs so you can see PTR tags for your files. There are also auxiliary operations like maintaining the autocomplete count cache and other metadata.

EDIT: Since you are getting double-digit rows per second, are you on an HDD instead of an SSD? The modern PTR has 700 million mappings, and does not work well on a high latency drive like a hard drive. If you cannot move your hydrus install to an SSD, you should pause it now as you will never be able to sync up to date and keep up to date.

Info on migrating a db here:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.13795

>>13752

Fugg, should have read more. Great, I am glad you got yourself sorted.

Thank you for the feedback on the help. That makes sense. I will make a job to give that help page and the UI a pass.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.13797

>>13753

I think you want to make a 'subsidiary page parser'. This is advanced and not well documented, but basically it lets you split a post up into sub-posts. This part isn't important for your case, but what is important is a veto on a subsidiary page parser only cancels the content of that subsidiary parser.

So, something like:

MAIN PARSER
photo content parsers
content parsers anything photo and doc have the same structure for, maybe tags
SUBSIDIARY PAGE PARSER
VETO if no gif
doc specific content parsers like url

In this case:

If you have a photo, it pulls the photo content, then runs the subsidiary parser, which terminates because no gif. = photo url

If you have a gif doc, it pulls no photo content, then runs the subsidiary parser, which pulls gif doc content. = gif url

If you have a blah doc, it pulls no photo content, then runs the subsidiary parser, which terminates because no gif. = ignored import status

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.13798

>>13755

>>13756

I very strongly advise against downloading an entire booru. If you have not seen this relatively new document, I have written several of my thoughts, which are based on experience with many users, here:

https://hydrusnetwork.github.io/hydrus/help/getting_started_downloading.html

That said, some users want to mirror for archival reasons, or because they just want a specific small niche booru, or they are madlads, if this applies to you:

Most boorus will accept '*' as an input query, which will just return everything. You would do as you are thinking here, setting up multiple download page queries in batches, variations on 'id>x', to fill in all the gaps, and then when you are caught up, you would set up a sub for '*' to catch all new content thereafter. If the booru is big like danbooru, it probably runs too fast for a hydrus subscription to to keep up with reliably. My guess for a technical limit, for a client that runs 24/7, would be a booru that spikes during a busy period at more than 100 files an hour.

I will again say getting everything is almost certainly impossible for any human to sift through properly. If you haven't done much archive/delete processing, I recommend you try doing 1,000 files. Time how long it takes to do properly, then multiply that by how many files you are expecting to get. If you want to chase spicy content before it is deleted, I would recommend you try to filter your query to catch that specifically, or find a different site to download from that simply is ok with hosting it. Maybe search for filetypes, if the booru accepts it, or 'does not have a creator tag yet' if the booru supports it, anything to help you separate the wheat from the chaff.

The most productive way to set up subscriptions is for artists you like. You will soon be overwhelmed with that like everyone else, and all the happier that you don't have a 30:1 trash-to-good ratio.

Let me know how you get on!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.13799

>>13759

Thank you, I am really glad you like it. I made it for other Anons like me, so we are brothers.

You should be good on the Qt update step, if you haven't tried it yet. There were no special install instructions, thankfully. As always though, the best thing to do before a big update is just to make a complete backup beforehand. You can then go nuts and even try it all in one step, since if it all breaks, you can just roll back, no worries.

For 4plebs, if you open the 'file/watcher import status' window on the downloader (hit the 'table' icon button), whichever is breaking, and find the url that is failing in the list, is there extra info in the 'notes' column? Right-click->copy notes on that to get it in your clipboard, then paste it into a document editor. It should give more info, and hopefully a full html readout of the error page, which should give more info. 503 is often CloudFlare problems, but 403 is Forbidden, which usually means you are hitting a malformed url. Did you make the downloader yourself? Does it work in the parser testing window? Is there maybe a api url converter in the url class that is hitting the wrong thing? help->debug->report modes->network report mode can help here.

You can also pastebin/email/whatever it to me, and I'll have a look.

Thank you for the report about siblings and parents. I can see how something like that would happen, I'll check it out.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.13800

>>13768

I am afraid I do not know much about it. The one here:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/blob/master/Download%20System/All-in-Ones/Single-Sites/nijie-artist-2018.12.14.png

Looks to be more than a year old, so if the site has just changed, I would not be surprised if it has broken. If so, I will remove it.

For now, for any non-simple-booru, I am recommending people use Hydrus Companion to log in. It is a browser add-on that lets you log hydrus in by copying your browser cookies across. The page with instructions is here:

https://gitgud.io/prkc/hydrus-companion

However, if the layout has changed and login script is broken, the downloader itself may also not work. Let me know if this is true, I'll give the guys on discord a buzz and see if anyone wants to update it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.13801

>>13784

>>13790

For now, I recommend you use Hydrus Companion, a browser add-on that lets you copy your browser's logged-in cookies straight to hydrus:

https://gitgud.io/prkc/hydrus-companion

This is a good blanket fix for sites that do not have native hydrus login script support.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1e64f No.13807

>>13798

>If you want to chase spicy content before it is deleted,

nono, I specifically would want the opposite of that

>I will again say getting everything is almost certainly impossible for any human to sift through properly.

that's why I'm training a resnet to do it for me :^)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

74f865 No.13810

Is there a way to have the duplicate filter only show the pixel by pixel duplicates?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8 No.13812

File: 7bef75e044d352f⋯.png (431.09 KB,1256x1294,628:647,4plebs_error.png)

>>13799

cheers lad

welp. said fuck it and went for it. i updated backup then updated hydrus clear through to 388 in one go. nothing 'sploded; db booted; all seems well. happy days. new qt version looks and feels sharp af too. i liek.

regarding 4plebs downloader, still no dice there. i was half hoping the update might just end up sorting it out but no such luck. anyhow. the list below isn't exhaustive but hits the main bulletpoints of what all i tried:

-new custom downloader made from scratch → test parse are successful; download is fail

-edit of first downloader stripped down to bare-bones → test parse are successful; download is fail

-duplicate (structurally speaking) of first downloader dialed in for a different but similar website → test parse are successful; download is successful

-import of cuddlebear downloader with smol modification → no test parse because am jsonlet; download is fail

-reimport of cuddlebear downloader left alone this time → plug-n-play mode so no test parse; download is fail

-simple downloader on default setting after nuking all traces of 4plebs from url classes and parsers → plug-n-play mode/no test parse; download is fail

>"help->debug->report modes->network report mode" can help here.

neat, that's handy af. picrelated. left side is using bog standard unaltered cuddlebear (json parser & 3 url classes from github repo). right side is the one i made (parser and 2 url classes, link here https://pastebin.com/F4kA9kxw ). the 4plebs urls don't "look" malformed and browser will navigate to them just fine but i can't seem to get anywhere with it when i try going through hydrus. all paths lead to 403.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6bd03e No.13815

File: 8323f2b1bb0a947⋯.jpg (69.21 KB,1361x432,1361:432,45745747.jpg)

>>13793

>Should be options->thumbnails, "EXPERIMENAL: Scroll thumbnails at this rate…". It'll tick down one thumbnail height at 1.0. Also there is a thumbnail % visible scroll check, which might actually be what you want here–set that to 0% and it'll always scroll.

Yeah this is what its always set on for me. I tried changing the "Do not scroll on key navigation" percentage but I haven't really notice anything or at least relating to my issue and setting the "EXPERIMENAL: Scroll thumbnails at this rate" to 0% gave me errors, if that's the one you mentioned to set.

Also I updated to version 389.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6d4bc3 No.13822

>>13801

Friendly neighborhood anon here, I whipped up a login script for e621 based off of the danbooru login. Not sure if it's 100% efficient(never made a login script before) - but it works, at least.


[73, "e621.net login", 2, ["e9438b0ebc12025f09faf1366734b927fef9832ce6d7d37d68f52da37512d0ea", [21, 1, [[], [], [], [[[51, 1, [0, "remember", null, null, "remember"]], [51, 1, [3, "", null, null, "Ph2fFwrvF7603984"]]]]]], [26, 1, [[72, "password", 1, [1, [51, 1, [3, "", null, null, "example string"]]]], [72, "username", 1, [0, [51, 1, [3, "", null, null, "example string"]]]]]], [26, 1, [[74, "hit home page to establish session, ignore passthrough", 2, ["https", "GET", null, "/", {}, {}, {}, [21, 1, [[], [], [], []]], [26, 1, []]]], [74, "hit login page to parse auth token", 2, ["https", "GET", null, "/session/new", {}, {}, {}, [21, 1, [[], [], [], []]], [26, 1, [[30, 4, ["get authenticity token", 14, [27, 6, [[26, 1, [[62, 2, [0, "input", {"name": "authenticity_token"}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]]]], 0, "value", [51, 1, [3, "", null, null, "example string"]], [55, 1, [[], "parsed information"]]]], 0, false, "authenticity_token"]]]]]], [74, "post login form", 2, ["https", "POST", null, "/session", {"password": "password", "username": "name"}, {"commit": "Submit", "remember": "1", "utf8": "\u2713"}, {"authenticity_token": "authenticity_token"}, [21, 1, [[], [], [], [[[51, 1, [0, "remember", null, null, "remember"]], [51, 1, [3, "", null, null, "$5betre%0"]]]]]], [26, 1, []]]]]], [["e621.net", 2, "Login required to access special content (clear your blacklist)."]]]]

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f04aba No.13825

>>13810

Not yet. Pixel dupes are recognised in realtime at the moment, but I have a plan to make a cache for this data and let you search for them specifically. I do not know when I will get to this.

I expect pixel dupes to be the first scenario you will be able to set up auto-duplicate resolution rules for. (e.g. 'if potential pair is pixel dupe and one is jpeg and one is png, auto-select the jpeg') With luck, we'll have auto-resolution of many imageboard clipboard.png files.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f04aba No.13826

>>13812

Thank you–I think I figured this out. There is a little-used feature under network->data->manage network headers that lets you set the User-Agent (or any other http header) of hydrus for different network contexts. There should be a couple of examples in there you can basically copy.

If I add a new User-Agent entry for archive.4plebs.org set to my web browser's string, the download works. Please forgive the bad UI–this is even more hacky code than usual. My guess is they have set up something that says 'if the User-Agent is anything other than these known browser strings, return 403 forbidden'.

When you export a complete downloader package under network->downloader definitions->export downloaders, there is a button to include these custom http headers, which other users will be able to import too under network->downloaders->import downloaders.

The files under the domain i.4pcdn.org do not seem to need the User-Agent.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f04aba No.13827

>>13822

Thank you for this work! I will roll this into 390.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8ab7c4 No.13830

Is there a way to give two IMAGES (not tags) a parent-child relationship or similar?

Similar to the "this image has x child images [link]" feature on borrus. Basically I want some way to link e.g. a two-to-four image set together

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

57d276 No.13831

Does anyone have (or is the Dev planning) a gallery downloader for AGN.PH? Nintend0h has been getting uppity with takedowns, and I wanna rip pokemon shit before they get shitcanned… just in case.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7a1a55 No.13833

>>13826

This is a slightly better version, now that I messed around with the login script settings more. I can't improve much else until the new version where the wx->qt bugs are fixed.


[73, "e621.net login", 2, ["e9438b0ebc12025f09faf1366734b927fef9832ce6d7d37d68f52da37512d0ea", [21, 1, [[], [], [], [[[51, 1, [0, "remember", null, null, "remember"]], [51, 1, [3, "", null, null, "Ph2fFwrvF7603984"]]]]]], [26, 1, [[72, "password", 1, [1, [51, 1, [3, "", null, null, "example string"]]]], [72, "username", 1, [0, [51, 1, [3, "", null, null, "example string"]]]]]], [26, 1, [[74, "hit home page to establish session, ignore passthrough", 2, ["https", "GET", null, "/", {}, {}, {}, [21, 1, [[], [], [], []]], [26, 1, []]]], [74, "hit login page to parse auth token", 2, ["https", "GET", null, "/session/new", {}, {}, {}, [21, 1, [[], [], [], []]], [26, 1, [[30, 4, ["get authenticity token", 14, [27, 6, [[26, 1, [[62, 2, [0, "input", {"name": "authenticity_token"}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]]]], 0, "value", [51, 1, [3, "", null, null, "example string"]], [55, 1, [[], "parsed information"]]]], 0, false, "authenticity_token"]]]]]], [74, "post login form", 2, ["https", "POST", null, "/session", {"password": "password", "username": "name"}, {"commit": "Submit", "remember": "1"}, {"authenticity_token": "authenticity_token"}, [21, 1, [[], [], [], [[[51, 1, [0, "remember", null, null, "remember"]], [51, 1, [3, "", null, null, "$5betre%0"]]]]]], [26, 1, []]]]]], [["e621.net", 2, "Login required to access special content (clear your blacklist)."]]]]

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8 No.13834

>>13826

that did it! thanks so much again man

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a59a6e No.13835

>>13830

Not yet. The next big push on the duplicates system will expand the 'alternates' file relationship into a full-featured file-association tracking system, as you say exactly like boorus do.

Ideally you'll have multiple new relationships to set like 'recolour', 'costume change', and hopefully some basic indexing as well so you'll be able to order several WIP (work in progress) versions of a file.

It'll also have ui support, so if you are looking at a file in the media viewer that has a parent/master, there will be a quick way to navigate to that without losing your place in the navigation carousel.

>>13831

In general, I do not write downloaders any more. There are too many sites for me to keep up with and get my regular work done. If you have not seen it yet, the main repository for user-made downloaders is here:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/Download%20System/All-in-Ones/Single-Sites

There are several users on the discord working together to make downloaders as well, here:

https://discord.gg/3H8UTpb

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a59a6e No.13836

>>13833

Thank you, I have copied it. As I said in the bugs thread, I have fixed those two things (I assume you) highlighted, and I gave a quick click through all the other edit login script UI. I think it is all good now.

I'll put off rolling this in for tomorrow's release, just in case you would like to do more. Please let me know once you are up to 390 if you are happy with this as-is or if you have a new one.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

fb696e No.13844

>>13836

I've updated to 390 and everything looks like it is working as it should - the second script is good to go as is.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6b8283 No.13847

I started getting an error when downloading from Danbooru:

"Could not find a file or post URL to download!"

This just started happening today, and I didn't change anything on my end other than updating to v390. Any ideas?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6795b4 No.13849

Is there a way to move a selection of files from one database to another while retaining the tags? I could do it one tag at a time, but it'd be great to have an easier way of doing it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c48a70 No.13852

Hi, I'm looking at some programs to see what fits my needs and didnt see this in the docs. How does hydrus handle not downloading the same image twice? If I delete a crap image, will I have to worry about it being redownloaded again from the same place? (as an extra detail, I'm guessing you can't help it if it's a different site with the same image though)

Lastly, as a test, I trying to import 25+ mp4 and webm videos that all fail with "stream missing" errors. Is this a known thing? Thx and sorry if it's a silly questin.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0aac4a No.13854

>>13852

Hydrus will recognize the hash of files that you already downloaded and skip them. You'll see "already in db". Files that you previously deleted will be tracked the same way.

Files that are similar but not identical will show up in duplicates processing, and you can choose how to handle them yourself.

The help section on the Hydrus website has some good info on the basics like this.

I'm not sure about the .mp4 and .webm problem. Stream missing makes it sound like the codec isn't parsing the files properly, but that's just a guess.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0aac4a No.13855

>>13847

Fixed: I changed the danbooru file page parser to look for "class=image-container" instead of "id=image-container".

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1c92d No.13857

File: 3da07e7426cf5d6⋯.png (11.62 KB,1089x102,363:34,Capture.PNG)

First time user here.

Already add my login credential for both gelbooru and sankaku sites.

Why the gallery download doesn't works with loli tag for gelbooru? it said "The parser found nothing in the document!"

It works normally with other tags for gelbooru.

Gallery download works normally with sankaku for loli tag, but for some reason the tag imported doesn't add loli tag into the downloaded images, everything else got added, I even got loli_face tag but not loli tag.

Also can I changed the position of the windows?

I want to switch the place of preview and thumbnail windows.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1c92d No.13858

File: fc68a50925a706a⋯.jpg (171.42 KB,1806x964,903:482,Capture.JPG)

Also is there a way to make Hydrus preview the thumbnails first before downloading the original image just like Grabber and then I can pick which image to download later one by one using right click or shortcut?

Because downloading each image and then picking which image to saved and delete take too much time and bandwidths.

Grabber can do this but this app doesn't support local database and tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

669e9c No.13859

Is there a way to autotag the images I import without using the bloated public tag repository? I'd only like for the artist to be added, everything else is unnecessary for my purposes.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d9b111 No.13861

>>13854

Oh, the hash makes more sense. Yeah I have no clue about the stream missing, I have the newest ffmpeg codecs I could get from the ubuntu 19.04. Just let me know if there's something I can try.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2a4917 No.13862

>>13861

Responding to myself: looks like the hydrus client didn't detect my ffmpeg codec in the "about" page. I tried a separate client download and it recognized and work.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

63d096 No.13865

After a recent client update a tiny portion of my files have begun showing up glitchy in the client (low to massive distortion). So far I've found 5 out of 200, all of them are webm or mp4.

They ONLY glitch when viewed through the hydrus client, opening them in any other program has them showing up fine.

Is there a way to fix this?

If not, is it possible to edit a file's metadata (mainly import date) so I can import them again and put them back where they belong?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64 No.13867

File: a9d62e7bc64c220⋯.png (7.59 KB,345x100,69:20,ClipboardImage.png)

Regarding the URL classes that redirect from a web page to an API call, which checkbox settings take effect? The page class or the API class ones?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d8fdde No.13868

>>13857

Gelbooru needs the cookie fringeBenefits=yup to view that content, make sure you have it in hydrus.

>>13858

No

>>13859

Parser's can get tags from the site they are downloading from. Most parser's should already get some sort of tags, you can choose which one's it gets in tag import options.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ee6909 No.13869

>>13868

The cookie.txt fixed the problem.

Since there are no way to changed how the GUI looks, is there a way to make Hydrus act like a normal booru so I navigate them from a web browser?

Hyrdus is a great tool for scrapping image from boorus, but the UI navigation is really bad.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

100de7 No.13873

If a hydrus database is corrupted, but in a subtle way that isn't noticed immediately by the client on boot or during automatic maintenance procedures, if that can even happen, then is it possible for the backups that you make to become corrupted too, or is there some kinda check in place to make sure that the backup is safe before writing it to the backup location?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d2387f No.13875

>>13869

There are some third-party tools that make a web UI for Hydrus. This one is probably the best at the moment.

https://github.com/mserajnik/hyve

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d47a16 No.13876

Is there any way to change the font size of the GUI? I can't find it within the options.

I'm on a 4k monitor and I am going to go blind

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6adfe8 No.13877

>>13875

The last time I tried this it said, 404 when I tried to connect the API.

And there are no tutorial for non tech savvy people.

What is this even mean?

HYVE_API_BASE=/api: the base path of all the API routes. No trailing slashes.

HYVE_MEDIA_BASE=/media: the base path of all the media routes. No trailing slashes.

VUE_APP_IPFS_GATEWAY_BASE_URL=https://ipfs.io/ipfs: the URL to the IPFS gateway you want to use for IPFS links (only available when connected to hydrus client). No trailing slashes.

VUE_APP_HYVE_API_URL=http://localhost:8000/api: the URL to your hyve HTTP API. No trailing slashes.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

543601 No.13879

Is there anyway to change where the temp dir is? I'm trying to free up some space on my ssd and there are ~2600 Hydrus files and 14 empty folders in the AppData\Local\Temp folder. Would like to move the temp dir to another drive or reduce the size if possible

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

000000 No.13880

How do I run Hydrus over TOR?

Assuming I don't want my ISP snooping on my DNS queries to figure out what weird sites I scrape from.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d2387f No.13881

>>13876

You could make a qss and change the font size in there, there is a readme in hydrus root dir/static/qss. Something like this.


QWidget {
font-size: 100px #whatever font size
}

>>13879

You can change it with a launch flag. client –temp_dir="temp/dir/path"

>>13880

You should be able to add the TOR proxy into hydrus proxy settings in options>connection

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d2387f No.13883

>>13877

>HYVE_API_BASE=/api: the base path of all the API routes. No trailing slashes.

Thats the path for the API. Say you wanted the API at localhost:8000/ass, you would set it to "/ass"

>HYVE_MEDIA_BASE=/media: the base path of all the media routes. No trailing slashes.

Same thing but for the media is servers. e.g. localhost:8000/media/ass.png

>VUE_APP_IPFS_GATEWAY_BASE_URL=https://ipfs.io/ipfs: the URL to the IPFS gateway you want to use for IPFS links (only available when connected to hydrus client). No trailing slashes.

This shouldn't be important if you don't use IPFS.

>VUE_APP_HYVE_API_URL=http://localhost:8000/api: the URL to your hyve HTTP API. No trailing slashes.

The URL to the API you set in HYVE_API_BASE for the web front end to access.

I agree that it's not the easiest for non tech savvy people but then again neither is Hydrus.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

543601 No.13884

File: 7fcffa918988142⋯.png (6.16 KB,442x166,221:83,hydrus.PNG)

>>13881

The client won't launch if I use this as a launch flag

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

543601 No.13885

>>13881

>>13884

I got it to work with two dashes instead. Am I free to delete the old temp files now?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7905a9 No.13888

Is there a way to control, how many images are downloaded per second in a gallery download?

Seems like now it does 1 image per second, and even though I'm in the advanced mode, I can't find a way to change that.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6adfe8 No.13890

>>13883

>but then again neither is Hydrus.

I got no problem in installing hydrus and it got a wiki site explaining on how to use it.

Hyve only got a readme full of tech jargon.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6adfe8 No.13891

>>13868

Loli tag still not imported from sankaku even with cookies.txt added.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d31e4f No.13892

>>13890

you could try this one then, I thought it was pretty easy to setup, though no documentation. All you have to do though is run npm install then ng serve in its folder after installing npm and angular-cli.

https://github.com/floogulinc/hydrus-web

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7203bd No.13893

File: 0c202846c26f034⋯.jpg (164.19 KB,1889x981,1889:981,Capture.JPG)

I tried Suika https://github.com/Suika/hydrus-docker Because it got build in Hydrus client in it, no problem during installation with docker compose.

I can download images from boorus and browser them with Suika's built-in Hydrus using VNC remote, but when I open the Suika's Webui I got 404 in the API page and no image in the gallery page.

My Suika compose setup https://pastebin.com/wthuSjYE

Since I already got Suika's Hydrus installed and some database and images in my NAS I tried Hyve next.

I added Suika's db folder and client_files folder into Hyve's docker-compose.yml.client.example then ran it with my Portainer stack.

Open webui and got this

>ApiUnavailableError

>Could not connect to hyve API. Please check your .env configuration.

My Hyve client compose https://pastebin.com/3Vfj4TuC

I changed hyve-sync ports: from '8000:8000' to '7000:8000' because my Portainer webui also used the same ports and I can't changed them.

Next I tried to import Hydrus images and tags into Szurubooru, with Szuruporter, dunno if i'm just a brainlet or was it because Szuruporter never got update anymore because nothing works.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8 No.13895

>>13849

https://hydrusnetwork.github.io/hydrus/help/advanced.html

Hydrus Tag Archives under the heading for tag migration sounds like it'll accomplish what you're after.

>>13859

check the link above, read the section called importing and adding tags at the same time. if that's N/A to however you've got your files named or whatever your folder structure is, then probably the public tag repo would be your best bet. can't say for sure as i don't use it myself but there might be a way to whitelist the artist tags and thereby reduce the bloat.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a3a007 No.13901

When I set a custom path for an external program, Hydrus does open it, but it doesn't open the file in it. Do you know why this is?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

72b0bd No.13902

File: 41f8b03b2e8b6c3⋯.png (90.91 KB,1314x570,219:95,2020_04_04_23_44_38_file_i….png)

>>12641

Why do most of my downloads from danbooru get ignored?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cfaa81 No.13904

Is there a way to count the total number of tags, or similar statistics other than the "how boned am I" window?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4f77c8 No.13905

Is there anyway using a parser to add to an images notes.

For example, some images have a story in the description and so I currently have to manually copy it into the images notes.

Thanks

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64 No.13906

File: e1e432889f51678⋯.png (23.65 KB,507x254,507:254,typo.png)

There is a typo in the character type dialog, "minUmum".

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d34df0 No.13908

>>13902

Are you on the latest version? Danbooru changed some stuff so the parser had to get updated.

>>13905

Unfortunately not yet.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64 No.13909

>>13905

You can use tags for that. You'll lose line breaks, but at least it'll be searchable.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c7dfdd No.13910

Just wondering, is there a guide in video form for info about Hydrus Network?

The main guide is helping me a bit but i thought a video of footage of the software being used could help too

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

644098 No.13919

>>13877

Till today, i still don't know how to make this run.

Still got "Could not connect to hyve API. Please check your .env configuration."

I don't even know where do I get my API url from.

Here my compose setup https://pastebin.com/jdZK0ite

What I want to do is setup a Hyve on my NAS server using docker, let all my booru images and hydrus database saved inside my NAS.

While using Hydrus windows client to download my images and send them to Hyve server folder on my NAS.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d43b0a No.13921

Hiya. Recently got into hydrus and am loving how easy it is to download and sort/process the thousands of images/manga etc that I usually download manually, so thanks for that!

Just looking for a solution to the following issue:

Version - 392

Situation - Downloading an artist's work via "pixiv artist lookup" gallery downloader

Problem - Some posts are being ignored with comment "Ugoira veto".

Is there some way to disable this veto and have it pull the ugoira files too? I can't find anything in the changelog about ugoira since version 329, and can't find any settings that would relate to it. Ideally I don't want to have to download and tag them manually.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

644098 No.13922

>>13919

Wrong quote.

>>13883

Fucking captcha.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2207a6 No.13923

>>13921

>Is there some way to disable this veto and have it pull the ugoira files too?

Not with just disabling the veto, It would require an additional parser but even then ugoria isn't a video file, you could only get it as a zip of images.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c4b652 No.13926

Hello, is there a way to automatically hide the cursor after opening the media viewer instead of having to move the cursor around?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

063396 No.13927

Is there a shortcut for undoing a deletion and adding it back to the current view? Neither Ctrl+Z or setting a hotkey for "undo" undeletes the file, I have to manually right click to undelete. If not, am I using the wrong workflow? Thanks

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30 No.13938

The cycle of me being unable to check and respond to my messages every day continues, worsened by current events. I apologise. I'll keep trying new ideas here on schedule and stress budget, the ideal is 24-hour turnaround.

>>13865

Are you viewing these files through the hydrus native viewer, or the new mpv window, do you know? The new mpv player has audio support. You can also see what you have set to display files under options->media. Can you post one of the files, or otherwise link me to it, so I can test my end?

There is no way to easily edit import time right now.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30 No.13939

>>13867

Good point. I think those mostly apply just to the URL in the import queue, the actual URL being actioned, which will typically be the Post Page URL, not the API. Usually, the API conversion happens very late in the download process and is temporary, it only changes what URL is actually hit and what parser is used.

I think only the raw URL stuff, the top three checkboxes in your image, would apply to an API URL, since that stuff applies to the URL itself and recognition and parser match-up.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30 No.13940

>>13873

Unfortunately, a full integrity check is fairly slow, usually 1-10MB/second, so this is not done automatically. You can force a check under database->check->db integrity, and if you would like to set up a routine to regularly do so, it would increase your safety.

Otherwise, your scenario can unfortunately occur. The good news is that if the subtle way isn't something you or the client immediately notices, it is usually something like 'a power cut that turned off your computer while you were committing PTR data malformed a 4KB database page of 'stained brown shirt' tags on a selection of obscure.booru.org files', and since you likely don't have those files, you probably never load them and never run into the problem, and in any case PTR data, which is about 98% of all your db data, can be recovered with reprocessing, so it isn't so big a deal.

Check out 'help my db is broke.txt' in install_dir/db if you would like more background reading on this.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30 No.13941

>>13876

I am working on nicer solutions here that you can select in the client. It'll involve changing the qss as >>13881 suggests, but with 'pt' instead of 'px', and I need to fix some custom text I draw in the program that doesn't yet follow qss rules.

>>13885

Yes, you can delete anything in a temp folder if there isn't a hydrus client currently running and looking at it. Nothing in there is persistent across boots.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30 No.13942

>>13888

All of that is controlled under network->data->review bandwidth usage. It is advanced, so be careful. I strongly strongly strongly recommend against just deleting all bandwidth rules, as you'll likely catch some IP bans and also possibly overwhelm your client DB and UI.

Most of the rules that are in there by default–the 'edit default bandwidth rules' button–are set to ensure new users don't make a mistake and download 5GB of trash by accident.

I recommend you keep the 1 file per second global domain rule, but it is up to you. If you are interested, I have longer thoughts on the philosophy of not overwhelming your inbox and sanity here:

https://hydrusnetwork.github.io/hydrus/help/getting_started_downloading.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30 No.13943

>>13849

Yeah, I need to write a proper 'how to migrate files with tags' guide, as this keeps coming up, but the main solution here is as >>13895 says: use services->tag migration with Hydrus Tag Archives. Check it out, be careful because it is advanced, and let me know if you run into trouble.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30 No.13944

>>13901

It probably all depends on how the program wants to accept that launch path. They aren't all in the format:

program_exe "path"

But sometimes:

program_exe –open="path"

Or it might sperg out about paths with spaces in–don't forget the " marks.

If you open up a cmd/terminal and do a test, does the same format you have hydrus set to do work, or does it similarly just open the program? If it works in the command line, then try turning on help->debug->report modes->subprocess mode and then launching the program. This will popup a bunch of debug messages about what is about to do, and the error may be apparent. If it isn't, let me know what you'll get and I'll have a look.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30 No.13945

>>13904

Not yet. I'd like to expand this stuff in future and have some graphs and things.

>>13905

I hope to have an expanded notes system with note parsing in downloaders in the next couple of months. It will happen in a 'medium sized job' week.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30 No.13946

>>13906

Thanks, fixed.

>>13910

Not that I know of. I agree though, although I am sorry to say I don't have the time or expertise to do it myself.

>>13921

Unfortunately, there is no Ugoira support in hydrus yet, so the veto is the current 'graceful' way to error without an error. I have plans to add it, but it is a tricky format, so I can't confidently say when.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30 No.13947

>>13926

Thanks, I didn't realise that hide tech didn't kick in until an initial cursor move. I assume you are navigating by keyboard? I'll check that out and see if I can fix it.

>>13927

Shift-delete I think is the default. Please forgive this shortcut being a bit hidden, delete keys are funny, so these actions are still hardcoded and not yet moved to the full customisable shortcut system.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

096d01 No.13949

>>13947

Thanks regarding shift-delete. This still requires switching pages, tracking down the deleted item which is troublesome especially with no "order by time deleted" filter, clicking the image, and then shift-delete on it.

My ideal inbox workflow would be (text after * isn't possible currently):

A (delete bad img and auto move to next image)

L (archive image and *auto move to next image)

* Y (undelete image and auto move media browser back to undeleted image rather than tabbing to trash where the images are out of order by deletion time)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30 No.13951

>>13949

Thank you. There is a two part solution here, I think:

1) (plan to work on better trash timestamp stuff deleted) EDIT: I was talking to another user about sorting by trash time not working, but I just tested it now and 'sort by time imported' works ok for trash, as long as you are looking at the 'trash' file domain, rather than 'my files' . I can see how this doesn't work when you are looking at a 'my files' page that happens to have some trash in it, though, so I'll think about this.

2) I have a barebones content undo/redo system, but it only works on a couple of things like archive/inbox, and that not well. The main thing I have to do there is keep pushing on shortcuts, which involves converting program actions from hardcoded calls to a generalised 'command' action system that all works through the same pipeline. As I move more of the program to that, I'll be able to hang those undo-able commands on an undo system plugged into that same single pipeline. Just have to keep working.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6dcd94 No.13956

Hello all,

I stopped updating hydrus after the 8chan debacle and never really went to search on how to continue getting news about it.

I noticed the current version is 398, mines is 318. Could you let me know a list of major things that were added or changed since then?

Also given this situation, I was thinking of deleting the whole database, settings, etc (except the pictures) and start over.

If I copy the client_files folder under Hydrus/db/ would that be enough to start over and keep the pictures or I need to do something else?

I used the Windows extract only version.

Thanks for all the help.

PS. Not required but is there a copy of tags of videogames/cartoons/anime I can borrow from anyone?

Current mine it has a lot of junk tags. Just looking for characters and series names.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c91700 No.13960

File: 81e08b1e8244238⋯.png (2.85 KB,811x29,811:29,Capture.PNG)

>>13956

Hello and thank you for everything that you have done for us,

It is my first time running Hydrus on this computer and it's not starting up, it's just taking up CPU power in the background. I don't recall it taking this long to load. I have all the prerequisites checked.

Is this normal? And if so,hould how long sit take depending on whatever pc specs you imagine? I apologize for the stupid question, it's just a little frustrating.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6dcd94 No.13961

>>13960

I think you meant to write to Board Owner/Dev. I'm not him, i'm a normal user only.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9 No.13962

File: abc953eef72057f⋯.png (14.54 KB,541x480,541:480,ClipboardImage.png)

I'm trying to make a parser for nyanpals.com/pettankontent but the site needs javascript to show the posts. Is there any way to load the page with JS or am I fucked in that regard?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9 No.13963

>>13956

>Could you let me know a list of major things that were added or changed since then?

If you want to waste 20 minutes reading, you could just go to the github and read the first few lines of each release.

>If I copy the client_files folder under Hydrus/db/ would that be enough to start over and keep the pictures or I need to do something else?

No, hydrus would consider them all junk files without the database file. If you don't want to update you can either export all files with tags(if you want to keep the tags) or just open the new hydrus and drag all the files in to import(you can just drag the db folder for that).

>PS. Not required but is there a copy of tags of videogames/cartoons/anime I can borrow from anyone?

If you don't mind bloat you could just download the PTR and either censor all tags except the ones you want or move all the tags you want to a new tag repository then deactivate the PTR.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

728c15 No.13969

>>13962

> Is there any way to load the page with JS

No, use your browsers development tools network tab to find out where they are loading the content from with JS and parse that instead.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2010c9 No.13971

I'm having the same issues as >>13960

It's not starting up. I'm running the latest version, fresh install.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9 No.13972

>>13969

I managed to track it back to a JSON page and convert it to

{url:"/img/pettankontent/50a00b3bb9934e314dcbd1e6c614f2ab.png",tags:"miqo'te,keiko"}
{url:"/img/pettankontent/535c19b6fe565f1b90262cbf978285ad.png",tags:"miqo'te,keiko"}

but now I'm fucked. I don't think hydrus has any way of converting this. It's not HTML so that doesn't work, but it's not JSON either so that breaks.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291 No.13975

So I have an issue, I use a lot of personal tags and have them set up with parent tags exclusively using My Tags but they're automatically being applied to the ptr tags as well. Its pretty irritating because I specifically did it to more easily tag things for myself because the ptr has a lot of mistagged images but theres not much point of the tags are automatically being applied to the ptr as well because of the parent tags I set up in My Tags.

For example, I set up a tag called #:3d so that it automatically sorts to the top of the tag list and so I can easily organize 3d images/videos. I also set up parent tags for it (in my tags) so that anything tagged with 3d rendering and the like will get it, now like a week later #:3d is getting applied to everything in the ptr too, among others. Im not sure if thats just on my end but its pretty damn annoying either way seeing as I have to commit ptr tag changes every time I want to get rid of them. Anyway I can stop this without getting rid of the parent tag entirely?

Also, is there an option to make tags from the tag suggestion box in the manage tags window not disappear after you apply them? Before you could scroll through images and tag things without closing the manage tags window, but now the tags disappear from tag suggestions after you apply them until you re-open it. Makes tagging things a bit more inconvenient to do.

I feel like a dumbass typing tag 5 times per sentence by the way

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

64349a No.13976

I think i messed up my installation/db by just copy pasting the install folder to my HDD with more space. I believe I have it all recovered and have access to the db again, but I can't drag and drop images into hydrus anymore, neither from Chrome or desktop/explorer. I think UAC now complains every time i run the exe. I don't remember if last time it just worked or had to change something to enable DnD.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

951b56 No.13980

>>13952

I recommend you update rather than starting over. There is lots of data in your existing db that won't carry over, and you can't really just switch out your client_files and continue.

I recommend you update 10 versions at a time, as described here:

https://hydrusnetwork.github.io/hydrus/help/getting_started_installing.html#big_updates

Let me know if you run into any trouble.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

951b56 No.13981

>>13960

>>13971

I am very sorry to hear this. I updated from python 3.6 to 3.7 in 393, so that is my biggest guess here. Can you try a couple of things for me?

If you go to install_dir/db/client - 2020-4.log and scroll to the bottom, how far does hydrus think it got in booting? What was the last step? Are there any errors there?

If you download the extract release, if you don't already have it:

https://github.com/hydrusnetwork/hydrus/releases/download/v393/Hydrus.Network.393.-.Windows.-.Extract.only.zip

And extract it to your desktop, does that completely bare and new client run?

It looks like >>13960 is Win 10. What OSes are you on, and is there anything special about your OS? Is it behind on many updates, or have an unusual .NET or Visual Studio Redist situation, or is it Windows 10 'N'?

Could you have an anti-virus that quarantined one of the hydrus dll files?

I recommend you roll back to 392 for now.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

951b56 No.13982

>>13962

Yeah, this >>13969 is the trick here. See what json they are pulling with their js–it usually is really nicely formatted, and actually easier than parsing html.

On Firefox, right-click and hit Inspect Element, then hit the network tab and hit f5 or scroll down to load the next gallery page or whatever. You'll see it come in as it happens.

There are a few 'api' parsers in a default client install that work like this. Check them out, and their GUGs and URL Classes (which use API transformations) to see how it all works. Let me know if you run into trouble.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

951b56 No.13983

>>13972

Ah, can you post an example URL, so I can have a look?

EDIT: I looked it up:

When I fetch:

https://nyanpals.com/kontent?include=pokemon&exclude=nsfw&allow=&limit=10&method=uploaded&offset=0&order=DESC&token=null

I get some nice JSON like what you have here, but it seems to parse ok. The only problem is that comma-separated tag string. There isn't a way atm to get hydrus to parse that single item into multiple strings, afaik, although I do hope to add this.

An easier first solution is to just get the URLs. If you are unsure how to split that into rows in a hydrus parser, you want to change to a JSON formula and get the 'rows' value, then every item of the subsequent list, and then every 'url' entry of those. It should give you a list of URLs.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

951b56 No.13984

>>13975

I am sorry for this trouble. There aren't excellent tools yet to change what parents and siblings apply where. I want to change this in the near future, allowing you to set per-service rules and sibling/parent application.

For now, the only option is in options->tags, uncheck 'apply all parents for all services'. This should keep your parents separate, but I do not think it is perfect. Let me know where it isn't perfect, and I will fix it.

I accidentally made the new tag-removing suggestion boxes not refresh correctly on new media. I will fix this. Once it is fixed and you give it a go, please let me know if you would still like the option to have the tags no longer remove. The ideal here is that all the lists will constantly synchronise to the current media so they never show a tag that is already 'current'.

Tell me about it. Sometimes doing this I have to say the weirdest shit over and over.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

951b56 No.13985

>>13976

Windows doesn't like it when you DnD between programs of different privilege levels. The most common example is one program running as admin and another not. I understand there are more subtle levels that can apply as well, but I do not know the details. Could that be the problem? It sounds like UAC is asking about admin rights there–is hydrus now trying to write its db to Program Files or a similar protected location?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9 No.13988

>>13983

>An easier first solution is to just get the URLs.

That works, but it means I can't get the tags since the URLs link directly to the images, which is the entire reason why I wanted to write a parser.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

64349a No.13990

>>13985

That must have been it. I did have it installed in program files x86 just now. Reinstalled somewhere else and now DnD works. I guess i forgot where I had it installed last time.

Thanks for the assist.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

951b56 No.13991

>>13988

Right. Unfortunately I don't have a 'split one text into multiple' lego block ready to slot into the pipeline yet. The current string transformer is strictly 1-to-1.

I've just had a look at the code to check. I currently have:

for every text parsed by the formula:

test against string match

convert using string transform

I could try to slot in a new 1-to-n 'string splitter' object before the transform, but perhaps the real answer here is to also make a 1-to-n 'string processing' container object that can hold m match, converter, and splitter objects in any order you want. Then if you want to test a match after conversion you can, or after splitting, whatever you want. I could spam that processing object in several other places, along with the splitter.

I will have to think about this. This specific problem of string splitting keeps coming up. I think it might be worth the extra investment to make a flexible, pipeline-simple system that will solve more problems than try to jam in more hardcoded stuff.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

100de7 No.13992

Are known urls for files supposed to be urls that point to the same exact file (hashes match) or just files that look the same? Like, is it something I would add myself when a come across a file on the web that looks identical to one in my database, or do I just leave adding known urls to the downloaders and such, who can check that the url points to that exact file, and not just one that looks like it?

I wondered this when I saw the known url merging options on the duplicates processing page. I thought that it didn't make sense to ever merge known urls because they're different files, but then that got me wondering if known urls just need to point to files that are "close enough" to be okay. Of course, if that's the case, then that might cause trouble with downloaders skipping downloads of files from urls that are already known urls, especially if the file that would've been downloaded was a higher quality version.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

17c2e1 No.13993

Is there a GUI setting in 393 i need to tick to fix this error? Having the error on tbib and danbooru so far. Also I migrated my image db and upgraded like 50 versions, and I notice one of my older parsers transferred to. Maybe my parsers are outdated? (srry if that doesn't make sense, not 100% on this stuff yet). Here's the error anywho:

Looks like an unsolvable CloudFlare issue: Cloudflare reCaptcha detected, unfortunately you haven't loaded an anti reCaptcha provider correctly via the 'recaptcha' parameter.… (Copy note to see full error)

Traceback (most recent call last):

cloudscraper.exceptions.CloudflareReCaptchaProvider: Cloudflare reCaptcha detected, unfortunately you haven't loaded an anti reCaptcha provider correctly via the 'recaptcha' parameter.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

hydrus.HydrusExceptions.CloudFlareException: Looks like an unsolvable CloudFlare issue: Cloudflare reCaptcha detected, unfortunately you haven't loaded an anti reCaptcha provider correctly via the 'recaptcha' parameter.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

566336 No.13998

>>13993

I should mention my whole comp routes over tor and that hydrus is running via torify

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0b3892 No.14000

File: 8eddfb5283fe190⋯.jpg (254.98 KB,1200x1200,1:1,Romantic_vampire_eb2bfa_74….jpg)

Hello guys ! Newbie here, I'm reading the help but I'd like confirmation if I have understood everything correctly: everytime you move an item to the db (drop&load or manual import) hydrus is in fact creating a copy of your item and a thumbnail. Those newly creating files are stored somewhere in the program both in form of hash, thus allowing it to quickly access the data and the thumbnail but not really to access it easily by the user using your regular framework (like going into windows file explorer) ?

If this is correct, can someone tells me how much datas he generated for one item ? I would say it roughly doubles it but I'm not sure. I'm asking because I have a fairly large amount of files that I would like to process but not being able to allowed that much disk space I'd like to estimate the total amount it would required.

I'm of course thinking of investing in some new storages and I'd like to know if someone ever tried to store their datas onto usb drives like it is mentionned in the help. I always have my doubts about that method because of how "temporary" is the nature of those but this seems like a good way to stock new items.

Thanks everyones and special thanks to the dev, this stuff is a dream comes true.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

db4114 No.14002

Is it possible to assign extra mouse buttons as shortcut triggers somehow? The GUI doesn't seem to recognize them.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

23788d No.14003

>>14000

The files are just copied so yeah double, plus a little bit extra for info about them stored in the database but this is minuscule.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

692d3a No.14005

File: 54a6c1dab7ef451⋯.png (78.34 KB,793x667,793:667,Screenshot_from_2020_04_24….png)

Hello. I apologize if this is a very simple problem but I can't manage to figure it out on my own.

I want to use hydrus on multiple computers and I want them all to be synced together. The way to do it is to use file/tag repositories, correct?

I can't get it to work though, I've set up a server as it's detailed in the help pages and then added a file repository service to it, added an account to the file repository, uploaded a couple of files to the remote service then I added that repository on another device, yet am not seeing any of the files on my other device.

What am I doing wrong?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

572163 No.14101

>>13992

Ideally, it is for exact hash matches.

The internet being what it is, plenty of sites point to other referral URLs that have resizes, or CloudFlare optimises a file in transit for one site but not another, so sometimes two files in hydrus get the same known url, or one file gets two different urls for the same site, but everything works best when each file url only has one file associated with it. When there are url merges or dupes, hydrus has downloader logic to recognise this and ignore that data when it does the 'have I already seen this file?' check.

The duplicate known url merge is not recommended for general use. It works for weird one-off jobs, usually based on user workflow preference where they want to have every possible url available to click in the media viewer, but every time it is done, it weakens the 'already in db' logic and ends up wasting html or file bandwidth. I wouldn't have added it, but urls are always a bit of a clusterfuck, and if you have time to put in to do custom dupe merges or have a weird local network proxy redirect downloader setup that mirrors some crazy urls to the wrong place, it can make sense.

Normal users shouldn't touch it, I think. And in general, I think I'd say leave known urls management to the downloaders. You can add a missing one if you want, if you discover an important one that somehow got skipped, but only if you have the time to get that finicky.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

572163 No.14103

>>13993

>>13998

This is a very new (393) system. Hydrus now has the ability to recognise CloudFlare challenge pages. For simple challenge pages, hydrus can now do the five second wait you sometimes see on your browser. For difficult captcha challenges (which you are likely seeing with TOR, and sites that are under active attack show to everyone), hydrus cannot currently do it and now registers a CF-appropriate error. There is an extension to the new 'cloudscraper' module to add a captcha solver service, which it is complaining about in your error, which I may do.

Did this site work for you in the old version of hydrus? My belief is that it shouldn't–hydrus would have hit the CF page and just recorded a 503 error.

I will update that error message to be less technical, thank you for posting it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

572163 No.14105

>>14000

Hey, yeah, >>14003 is correct. Hydrus owns its files and stores them in its own file structure. It only needs that one copy of the file, but that file cannot be nicely seen with Windows Explorer, so if you still want to view a folder of files with nice filenames, you will want to keep a copy of the original folder after you import to hydrus.

Your situation is common to new users. As I say a bit in the help, I recommend you keep using hydrus for a few weeks with a thousand or so files, and don't delete any of your old files and folders. Most users who like hydrus figure out which files work for them in hydrus (usually booru files are good, but paged comics are currently bad) and find they do not want to keep the nicely named folder-based originals and delete them.

As for extra metadata, hydrus generates a thumbnail for each file, which usually ends up being about I think 1.6% of total file storage space, and the database itself, which if you do not sync with the PTR will start out a few MB and even with a large collection will normally be less than 500MB. If you do sync with the PTR, it will be 25GB or so. If you are not on an SSD, do not sync with the PTR.

For any situation with 100,000+ typical imageboard and booru files, the total effective hydrus size is basically 97% taken up by file size. There isn't all that much bloat.

If you are interested, there's a little more on my thinking about storage stuff here:

https://hydrusnetwork.github.io/hydrus/help/faq.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

572163 No.14106

>>14002

Not yet. I am slowly converting my five different old hardcoded shortcut systems to the unified system under file->shortcuts. Every now and then, I extend it and add mouse hooks for a new window, but support is limited atm. It is currently mostly the media viewer and its related filters I think.

That said, are your extra mouse buttons like OS-level buttons, like 'back' and 'forward'? Now I think of it, I don't think I recognise these yet, and I bet Qt has some nice support for it. I will look into this. It would help if you can list all the non-standard buttons you mouse can do.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0ca948 No.14111

File: dab3ed9b4c0104e⋯.png (5.37 KB,302x155,302:155,ClipboardImage.png)

1) How do I give filenames to my images?

2) Is there a way to make the letters bigger? (See pic related)

3) How do I favorite an image?

4) How do I use the collection feature for doujins/comics/sequential images?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b5d17a No.14114

Is there a reason why Hydrus doesn't support importing Word documents, spreadsheets, PowerPoints, etc.? I really like the idea of getting away from folders, but right now, I have to turn my documents into PDFs to import them.

I assumed it was because these files are essentially made to be edited, and that editing them would mess things up for Hydrus, but I found out today that it does import Photoshop files just fine, though I haven't tried editing them yet.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

674e43 No.14133

>>14103

Close–the site didnt work in old hydrus and actually showed 403 forbidden instead of 504. I'm looking at a proxy to use inside hydrus to hide my tor usage. Any thoughts on services b4 i throw money at this? Thanks again>>14103

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8 No.14156

>>13927

file > options > files and trash > remove files from view when they are… uncheck both boxes for now. next help > help and getting started guide > getting started with files > filtering which streamlines the workflow for archive vs delete processing and functions basically as >>13949 describe here only with different keys: your A maps to delete, your L to spacebar, and your Y to backspace, plus there's one more for skipping any files you may wanna leave inboxed.

>>13975

do you typically set up parents via the top menu (services > manage tag parents) or the context menu (right click > add parents to tag)?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8 No.14157

File: bdb5a343571141b⋯.gif (96.12 KB,1177x829,1177:829,string_break.gif)

>>13988

>>13991

>flexible, pipeline-simple system

pre-parsing conversion maybe? dunno json well enough to get much further than this but picrelated. might could work if entered in at the level of a sub-subsidiary page parser.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8 No.14158

>>14111

1) manually type them in prefixed with filename: or click import with tags when you drag files in from a folder

2)see >>13941

3)tag it with favorite or use ratings (services > manage services > add > local rating system)

4)my method is to give each page a unique page:# tag and every page the same title:whatever tag, then set hydrus to collect by title

5)read the help pages m8, particularly advanced usage - general and getting started with ratings, 3/4 of your questions are covered there and explained thoroughly

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3 No.14161

How do I download an entire pixiv of an Artist with Hydrus? Also how does it sort images that are in sets?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3 No.14162

>>14161

I'm in URL Import tab, and pasted in a pixiv link pressed enter and it didn't do anything says "Looks like HTML – maybe the client needs to be taught how to parse this?"

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3 No.14163

>>14162

>>14161

nvm found out, for pixiv I had to install some extension API thing. Hydrus seems very technical, and I've just started using it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3 No.14164

File: 39417cd4bc2f762⋯.png (56.6 KB,445x1028,445:1028,ClipboardImage.png)

How do I edit the tags that get automaically fetched into the gallery? For example it fetches "creator:", how do I make it so it automaically changes this to "Artist:" instead? Also some tags such as "long hair", "medium long hair", "short hair", I want to change it too automaically become just "hair" as well. Would be greatful, the Help guide is quite complex for me

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

de61ca No.14165

>>14164

Such changes require you to edit the parser itself locally. changing the creator namespace to artist is quite simple and only requires a single thing to be changed. as for the hair stuff. more tricky. could be done with a regex replacement prob on the tag content parser.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3 No.14166

>>14165

How do I edit the parser locally? I've also no experience with regex expressions

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3 No.14167

>>14166

okay found out how to change the parser tags using siblings. But still don't know how to change creators to Artists.

Also have a big problem of downloading from pixiv all the tags are in japanese and I'd rather have the Gelbooru and Danbooru tags instead. But I'd have to download from there instead of pixiv which usually has oredered images and source quality.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

de61ca No.14169

>>14166

really is best to join the Discord for that sort of thing as it will prob require a large amount of handholding.

But you can also read the docs on how to do it. :)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6be94a No.14170

So i got hydrus companion, when i go on the pixiv website i click on send cookies from this site to hydrus but i still cant have it parse the pixiv images. Can someone tell my retarded ass what im doing wrong?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8 No.14171

File: f10357834673048⋯.gif (1.66 MB,1292x875,1292:875,tag_namespace_change_in_pa….gif)

>>14166

>>14164

>>14167

here lad. best advice i could give you is to read the help guide start to finish so it gets into your head even if you don't fully understand how it all works together yet. don't worry about being able to relate everything in there to what it actually does in the program, just make sense of how the words relate to each other and the info will file itself away for later and make it easier to learn the program just by poking about and seeing what happens.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291 No.14179

Is there a way to sort files so that alternates are grouped together?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291 No.14180

Is the favorite tags window not sorting properly now for anyone else? After I updated to the latest version all the tags were jumbled up, instead of sorting by namespace and alphabetically.

>>14156

Late response but I typically use the context menu to add parent tags

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8adcb7 No.14181

I'm sure this is in the help somewhere I can't find, but is it possible to add page tags without going to each image and just plugging away?

Some of these longer paged things are killing me.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291 No.14182

>>14181

If you're adding images that have the page as the filename then you can use that with a page namespace as you import. Can use regex commands if theres more than just the page in the filename too.

Other than that though im not sure of any other methods other than just doing it manually.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c2b97f No.14183

>>14181

If you have or get the hydrus companion browser extension, you can go to utilities and use the sequential image tagger. you just put in the image hashes in order 1 per line and it will sequentially tag them. So if you have them in order in hydrus you can just select them all, copy the hashes and paste them in.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c2b97f No.14184

>>14170

Doesn't sound like you are doing anything wrong, You set it up with your API key and have cookie permissions on that key right?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8b2b0d No.14186

$$\{red}{redtex}$$

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

11beeb No.14187

Redtext

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

541472 No.14188

""Bold""RED BOLD

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

541472 No.14189

We are researchers who deal in open-source information, reasoned argument, and dank memes. We do battle in the sphere of ideas and ideas only. We neither need nor condone the use of force in our work here.

VINCIT OMNIA VERITAS

SEMPER FIDELIS

WWG1WGA

Q Proofs & Welcome

Welcome to Q Research (README FIRST, THEN PROCEED TO LURK)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

11beeb No.14190

"VINCIT OMNIA VERITAS"

""SEMPER FIDELIS""

$$WWG1WGA$$

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

11beeb No.14191

"VINCIT OMNIA VERITAS"

""SEMPER FIDELIS""

$$WWG1WGA$$

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

11beeb No.14192

$$\colorblueblurtext$$

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8b2b0d No.14193

$$\color{blue}{blurtext}$$

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3 No.14194

Has anyone else used imgbrd-grabber before? How does it compare to Hydrus?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

de61ca No.14195

>>14194

Hydrus is different. It downloads all that grabber does for sure with its tags.

It is easier to edit and add support for new sites on Hydrus and you have more control in general across the board.

But Hydrus is much more than just a simpler download client.

Its a full library manager with a ton of features for those types of users.

You won't find any better than Hydrus for booru/internet content of its kind.

Please feel free to jump on the Discord if you want a more direct convo about what Hydrus can do for you.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

100de7 No.14198

When using hydrus companion (on firefox, if that matters) to download from Pixiv, how do you get it to download a specific file from a post, rather than the entire post. If I go to the direct file url, (the i.pximg.net one) it just returns a 403. With post urls this doesn't happen, so I know it's not a login problem.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e98755 No.14201

>>14198

the direct file url needs a referer header of pixiv.net. In hydrus you can make a pximg.net url class and add a referer of pixiv.net to it or just add it to the http headers without making a url class.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

100de7 No.14202

>>14201

Thanks. I knew nothing about how url classes worked before this, but after banging my head into the wall until it worked reading about it on the help pages to figure it out, I managed to make one after a few tries, and the downloading works now.

I feel like a url class for pximg is something that should be in the defaults. Is there somewhere that you're supposed to submit things like this to get added to the defaults, or at least a third party repo for people to download it themselves?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e98755 No.14203

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64 No.14205

Is PTR up and running?

I tried adding the service as mentioned in the help, but testing connection to ptr.hydrus.network gives an error.

Back then before the big change in PTR it worked for me, then I disabled the sync once the owner changed and now I'd like to enable it again.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ed6436 No.14207

>>14205

Yes it works. Rather than setting it up manually it's easier to just go to "help > add the public tag repository". Try that.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8adcb7 No.14211

>>14182

This sounds like a a good enough way to do it, thanks

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

35f78b No.14213

I noticed that "Source Time" information is included when pictures are downloaded from galleries, so the parser does know how to read and store that information, but there doesn't seem to be a system tag to search by source time, nor a sorting method in pages.

Is it at all possible to search by source time? I think it would be cool to look at various artists or franchise works from a chronological point of view (or at least as close as the boorus get to that)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64 No.14215

>>14207

I did exactly that but still testing the connection gives an error and no progress is being done in syncing.

Also, do I have to remove the old PTR service, that points to the non-existent server now, or I'll lose all information associated with it?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3 No.14218

How does running Hydrus on an SSD (2TB) compared to a hard drive? Will it make any difference when you have 10000s of images?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9e7558 No.14220

>>14218

Just having the database on an ssd will make it significantly faster.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7b65e6 No.14222

Is anyone using Hydrus Companion on Vivaldi 3.0? It was working but after the update the "send for tags then send to hydrus" context menu option does not display the prompt to input tags.

Any tips on how to fix?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e5852a No.14225

File: e6aec069ef9c36a⋯.png (24.64 KB,563x412,563:412,ClipboardImage.png)

Expired links

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e5852a No.14227

File: 3fe88c2972dbd44⋯.png (101.59 KB,680x564,170:141,ClipboardImage.png)

>>14220

>a SSD is strongly recommended

>when it's obvious you would use a HDD for huge picture stocking

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cd48ee No.14245

>>14244

Meanwhile I have 10TB HDDs

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ade59b No.14256

>>14114

Hydrus file import and maintenance is currently predicated on the idea that files' filetype can be determined from file content (not filename). As it is generally more difficult to determine a .doc is a .doc compared to recognising a jpeg or an mp4, these types have been slower to add. This is also the reason why I do not yet have a 'any other file' filetype.

Ultimately, to get these files supported, hydrus either has to recognise them a bit better, or I need to improve the maintenance and import code to deal with more complicated and less certain situations. I haven't looked too closely, so perhaps most .docs have a nice file header I can recognise, and I know some open source document formats are really just xml that I can ultimately parse and pull the right tag from, but some, like .txt, are going to be more tricky. Mostly it has just not been too high a priority yet, so I haven't invested the time–I am not opposed to it.

I recommend you not edit files stored in hydrus! Another axiom of the hydrus file storage system is that files are read only, and changing file content of existing files could give you a couple of recoverable but annoying problems down the line. Do an export->edit->reimport cycle if you want to edit.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ade59b No.14257

>>14133

I am not sure. I am afraid I do not have much experience in much of this. I'm still improving hydrus/cloudscraper integration, so I hope there will be better/more tools in future.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ade59b No.14258

>>14179

Not yet. That will likely have to wait for the next big iteration of the 'duplicates' system, which will expand alternates into a more fleshed out 'file relationships' system. In that step, I will have thumbs load with their dupes and other file relationships so UI stuff can access it for collecting and sorting. At the moment, thumbs don't know this info, so it is actually secretly fetched from the db every time you right-click on a thumb!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ade59b No.14259

>>14180

Sorry, I have fucked this up about three weeks in a row now. Everything is sorted correct in 396 except favourites. That'll be fixed next week.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ade59b No.14260

>>14215

Some users in Russia can't access ptr.hydrus.network. Some DNS ip range ban bullshit iirc.

If you remove the PTR service from your client, yes, you will remove all the mappings. The answer here is to change the settings in manage services to something that works. Can you load https://ptr.hydrus.network:45871 in your web browser? It should make a welcome page with an ASCII lady.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ade59b No.14261

>>14213

Not yet, but I hope to integrate more of that data into the 'modified date' row that files now have. At the moment, source time is not recorded.

>>14225

Thanks. Some weird dude got upset and started spamming TOS-violation stuff to the server along with some friends, so the old invite url got turned off. Assuming the users who run the discord server are ok at that point about putting a new invite up, I'll fold the new on into 397 and all my help links.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ade59b No.14262

>>14218

>>14220

>>14227

>>14244

>>14245

The main deal here to get great performance is to put the hydrus 'db' (i.e. the four client.xxx.db files) and your thumbnails on your small expensive fast SSD and your media files on the large cheap slow HDD. DB and thumbs benefit from low latency, jpegs and webms do not.

Doing this takes a small amount of work, but if you have an SSD, I strongly recommend ultimately moving to this setup when your client gets big. Here is the help page on this:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9 No.14264

File: aa45824d6ed87c2⋯.png (15.97 KB,936x392,117:49,ClipboardImage.png)

Was the way namespace searches work changed recently? I used to be able to do namespace:* to see all the tags that belonged to that namespace, now it only shows with at least three letters or two and a *.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

60ca44 No.14265

File: d1f92024439e1a3⋯.png (825.63 KB,1075x1518,1075:1518,overwhelming_sex_knowledge.png)

I haven't used Hydrus in a while. What did Pixiv fuck up this time and how to I get Hydrus to download lewd images from there again?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3e1190 No.14266

>>14265

They added captcha to login so you have to add cookies manually or using hydrus companion (https://gitgud.io/prkc/hydrus-companion) to login.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8f1ccb No.14268

>>14256

That makes sense, thanks!

Speaking of filetypes, is duplicates processing for gifs, videos and other filetypes on the horizon?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8f1ccb No.14269

Can you add support for Mozilla Archive Format (MAFF) files?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64 No.14272

>>14258

As a cheap and dirty replacement for now you could implement sorting by the thumbnail filesize, it will group similar images together, since when you make all images the same dimensions and compression level (which is what thumbnail generation does), the file size becomes a function of content.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

69ed8b No.14275

I'm using version 392 right now. Are there any hidden factors that lead to subscriptions stopping early? (besides the "i've seen these links before" system).

Am I right in assuming that the "on normal checks, get at most this many newer files" is a per-query and not per-subscription limit?

I'm working with a decent number of queries right now, with the intention of having them checked about once per week. I force-checked everything today, and after I set them going, the sub looked through a bunch of queries, then just stopped entirely. I've had some queries stop at like 70/100, some don't even start and are like 0/31.

there aren't any other ongoing simultaneous download or import jobs, no bandwidth limits are being exceeded. The popup just disappears, checking current network jobs shows a blank screen, and going to the subscriptions page shows that my files are for example items: 945/1224, nothing in the "recent delays / error" column.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

94ce1a No.14276

My sort is currently set to "artist-series-comic-chapter-page", but while the pages sort in descending order, chapters sort in ascending order. Is there a way to make both sort in descending order?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

32bafb No.14277

File: f6bcef2dcbc5cf4⋯.png (137.02 KB,869x959,869:959,123.PNG)

Am I a big dumb, or are there no default downloader for newgrounds art galleries? There's only ones available for games and movies galleries.

I tried making one myself but it didn't get all the urls, I think due to the fact that the newgrounds art gallery page doesn't load the whole gallery at once and requires you to scroll down to load the rest.

https://matt-likes-swords.newgrounds.com/art/

Notice that it doesn't load past about halfway through the 2016 submissions until you scroll down there, and as a result doesn't grab urls beyond that point. Pic related shows the test url parse results, and at the bottom is one of the urls that is beyond the loading point - it doesn't appear in the test results. It seems making the parser deal with that is beyond the scope of what I can figure out just from clicking around.

Are there premade downloader definitions for newgrounds art galleries already made? If not, is there a way around this issue?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

21d745 No.14281

Anyone ever have a tag search that stops working? Gelbooru one worked fine yesterday but now just results to "done" without fetching any files. Is there a new downloader?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1fae9 No.14282

File: 1e88bb7bb7be30b⋯.jpg (41.83 KB,640x626,320:313,selling_deer.jpg)

>>12641

How do I set my database backup to ipfs?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

60ca44 No.14284

>>14266

I did that and it said that the cookies were successfully added, but I still only get safe images when I do an artist search in Hydrus. If I do "send this tab to Hydrus" it can download lewds though.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6bc377 No.14285

File: ba508d1f9cfec6f⋯.jpg (503.28 KB,748x800,187:200,ba508d1f9cfec6fe795eb02e28….jpg)

I have a kind of specific question: How can you delete a file including all of it's potential duplicates from the trash?

I did download using my own scripts a couple of years back and … well I managed to get about 4 million files, no idea how many of them I still want.

I imported them into hydrus because I wanted to actually look at them - tagging, the PTR, filtering and duplicate processing are a real godsend.

So now I can (thanks to the PTR and scraped booru tags) finally search and delete everything I do not want to keep. The only issue is that the PTR will not tag all of the resizes and thumbnails and whatever else I managed to download.

Duplicates processing usually finds those files, but I can't really delete them, because they are not tagged.

I *could* go to duplicates and search for files in trash, and that *would* find all the files (even ones with only one duplicate in the trash), but they are not displayed on the media panel so I can't really delete them all at once.

I have to hit "launch the filter", then go through each pair hitting delete on the keyboard, then hit "both files" in the dialog box and finally hit "permanently delete".

Is there any chance I can build a query for that? Otherwise, I will just work something out with xdotool and the "show a random pair" button.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

100de7 No.14286

Is there a way to search for files that have an identical known url, to make sure that I'm not accidentally putting them on the wrong files?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b5d17a No.14288

>>14286

Yes: system:known url

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9 No.14290

>>14281

I'm getting the same error. The gallery parser stopped working for some reason, maybe gelbooru changed something.

At first I thought I had been blocked from running 3 simultaneous gelbooru subscriptions.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9 No.14291

File: 52e1c18e561b303⋯.png (4.68 KB,512x115,512:115,easy_import_downloader_png….png)

>>14290

Turns out it was just a simple retarded change. They changed the thumbnail class from "thumb" to "thumbnail-preview" for no reason.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab No.14295

>>14264

Yes. I cleaned up the pipeline here and removed it, as it was always a hack that could cause problems for some users. I hope to add in an option this week to return it if you want, and an option to allow 'ser' to match 'series:metroid' and all other 'series' tags.

>>14268

Not right now. I am currently recovering from a huge conversion of the UI library from wx to Qt. tl;dr is: all UI is much better now, but I have a thousand small things to do before I can focus on a big expansion of anything.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab No.14296

>>14269

Can you email me (hydrus.admin@gmail.com) an example of one, or just point me to one here? If the file has an easy reliable file header, I can recognise it and add it sooner rather than later.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab No.14297

>>14272

That's an interesting thought. I don't easily 'know' thumbnail size atm, but I do know filesize. The 'sort by bitrate' does size / num_pixels for static images, but I'm not really sure if that is really similar to what you are going for.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab No.14298

>>14275

If you are getting 70/100 kind of numbers, that sounds like bandwidth limits to me. The subs are getting the files they are supposed toand yeah, the 'get this many files' options are per querybut they are quitting those queues early. Subs can stop early for a couple of odd things, like pausing network traffic or opening the manage subs dialog, but if they are doing it on their own, it is probably due to bandwidth running out for that context or, I think, five file errors on that query happening in one run.

If you catch a sub stopping early, open up manage subscriptions and look at it. It should have a 'recent delays?' column that'll talk a bit more about why it stopped and when it expects to continue.

SHIT, EDIT: I just read the rest of your post. Hmm, I am not sure. If you hit help->debug->report modes->subscription report mode and then tell a sub to run, it will spam a whole bunch of popups on its decisions, also writing them to the log in install_dir/db. This may help us figure this out.

It could be a weird bandwidth rule, maybe something like a 15s or 30s delay somewhere, where the sub is saying 'oh, I should stop', and then the delay times out real quick, but some other hardcoded delay isn't requeueing it for ten or twenty minutes.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab No.14299

>>14276

Hmm, they are both, I think, supposed to sort in ascending order, e.g. 1, 2, 3. Exactly so you can read comics in order. If you go into options->sort/collect and add a new namespace sort just for 'chapter-page', does that work correct? Do the files have multiple series/creator? Some of this stuff is very old tech, and it doesn't work well for those situations.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab No.14300

>>14277

Newgrounds has always had spotty support. Their galleries and embeds are weird and sometimes change. You might be in luck, because I see a new one here:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/blob/master/Download%20System/All-in-Ones/Single-Sites/newgrounds-2020-04-07l.png

It says it does art, but I know nothing about it. If it works well, let me know and I'll update the hydrus defaults.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab No.14301

>>14281

>>14290

>>14291

Thanks mate, I'll fold that into 398.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab No.14302

>>14282

There is no in-hydrus way to do this atm the moment. You would have to figure out a backup regime that somehow pinned your db or its media files to your ipfs daemon.

I may be misunderstanding you. Your db files are very private, and I would not recommend trying to expose it to a public network like IPFS. But if you want to put your media files, your jpegs and so on, in IPFS, then hydrus has limited support for advanced users. I would shy from calling it a backup, though. Try adding an IPFS service under services->manage services and getting it to talk to your daemon. Then you should be able to pin files using thumbnail right-click->manage->remote services.

Be warned that the new IPFS 5.0 was broken for hydrus until I updated my code in the recent 397. It may have patchy support, and has always been experimental.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab No.14303

>>14284

You might want to check network->data->review session cookies. Clear everything from there for pixiv.net and try resyncing from Hydrus Companion. Maybe there is an old cookie in there messing you up for search preferences.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab No.14306

>>14285

Hmm, this is an unusual situation. I assume you don't have your trash set to empty by itself? Otherwise you could let the trashed files clear out naturally. And I assume you have files in trash you still want to hang on to, otherwise you could clear it from services->review services.

I was thinking maybe you can do something like with 'system:num file relationships', but I am not sure how you would limit the search space to trashed files but also capture potential dupes. At the moment, the thumbnail menu only allows you to show potential dupes for single files, which is a limitation of the current media storage system.

I don't think I have a clever solution for you. I don't want to expand the thumbnail menu to have 'show all potentials for all selected' until I have that media storage update done. I'll keep thinking about this. I may be adding some shortcuts to show file relationships in a new page soon, that may be able to do it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d07709 No.14307

Is there a way to send multiple files to an external program? Ie. select a few webms with audio and play them all in mpv as if you passed it multiple arguments on the command line?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6bc377 No.14308

>>14306

too bad, I really though I was missing something obvious…

Just to be clear, I can delete files in the trash, I disabled the automatic deletion because I wanted to "hang on" to those trash files until I can get rid of them AND their duplicates.

I purposefully avoided de-duplicating things I would like, because otherwise *all* those files would get deleted too.

My workflow would be like this;

- search for artists I really do not want to keep, deleting all of their trash (into the bin).

- then, get all potential duplicates of files in the bin (hamming of 2 or less, so that I don't get any false positives)

- move either all of the duplicates in the trash too or delete them on the spot.

I guess I'll just make my own solution using sqlite, python and the API (so that hydrus actually does the deletion). If anyone is interested I am happy to provide my script once I am finished.

I am not quite sure this is absolutely necessary to integrate this functionality into the client, since it is such a rare problem (you have to download manually!) to have.

Thanks!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

897aed No.14310

Good morning, friends. I'd like toa sk, is there a way to assign a shortcut to a specific sorting scheme? Like, say, quickly alternating between sorting by time of importing, or number of tags, or by rating?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5568e No.14313

>>14307

Also, is it normal that Hydrus fails to play audio on Linux in the internal viewer?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c4b652 No.14316

File: 595001480956e1e⋯.png (4.97 KB,211x149,211:149,client_18_05_20_00h_13m_11….png)

Recently i noticed that every imported file from yande.re were returning these "-" tags and i wondered if this is just me or if it is happening to anyone else. I don't know if this is related but i had to recreate my db not long ago and i don't think i have messed with the downloaders settings or anything

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3e2d4e No.14322

>>14296

Gmail rejected the files as potential threats, but you should have received a download link now.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8941a1 No.14329

hydrus website says it's primarily for windows and it works well in wine.

Is that just old information? Is there any feature missing from newer linux releases?

Also for dualbooting and accessing the same db, should I bother with the server or is that only really useful for remote access?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3e2d4e No.14333

Is it possible to add support for WAV files?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4934b0 No.14334

>>14329

I'm using the Linux version and it works for me.

>>12641

How do I download from exhentai? I tried adding the cuddlebear downloader (2019-08-15 version) and using Hydrus Companion but it doesn't do anything I can see. I also tried that downloader on its own, gets nothing.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7d4e2d No.14335

>>14300

It worked perfectly! Thank you.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8941a1 No.14336

>>14334

>I'm using the Linux version and it works for me.

Are you able to get audio working in the base file viewer? When I select mpv viewing on my debian install it launches a new mpv window. When I select hydrus viewer audio doesn't play.

Otherwise everything seems to work like windows but seemless webm audio playback is essential for me.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2ba267 No.14339

>>14336

I'm >>14307 and >>14313

I have the same problem with audio not working. The only other bug I've encountered on Linux is that Hydrus stops detecting the scroll wheel if I unplug my mouse and plug it back in without restarting Hydrus. Apart from that I haven't had any issues.

Not sure why the documentation even mentions Wine considering the native version works well enough.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

69fefa No.14346

>>14307

Not yet. I'd like to add 'playlist' support, but hydrus has never been great at generally stitching multiple files together, and I just haven't put the time in yet. It will likely come in the form of an expansion to options->external programs, where you set a format of how to launch multiple files for that particular exe.

If your player has a bit of playlist UI, you may have luck launching the first file in it and then drag and dropping the rest of your selection onto the playlist. I think thumbnail order is preserved through filename drag and drop export.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

69fefa No.14347

>>14310

Not yet. I hope to add this as I continue moving old shortcut tech to the new system under file->shortcuts. There is still a bunch of hardcoded stuff I want to clean up before I add too much new stuff.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

69fefa No.14348

>>14313

>>14336

>>14339

Yeah, unfortunately the native player can't do audio. It is all my old custom code, essentially a software renderer that does some duct-taped pipeline right from ffmpeg executable, which started as a gif renderer and slowly expanded to do webms and so on. I never wrote an audio pipeline for it, and while I was planning to experiment and learn how to do that, the Qt switch happened and suddenly an mpv player became a real thing with really nice hardware rendering and instant good audio support, and the native renderer got put on the back burner. If you can get mpv working on your Linux machines, I strongly recommend it. If you are running from source, you need libmpv, not mpv, which can be a pain. There is limited hydrus help on this, but I will be getting info from users and writing a guide for the 'running from source' help as it all matures.

>>14336

Can you talk more about mpv launching a new window? Does it actually launch the mpv external program, or does it seem to be doing an embed window (with no overlay), but it is detached from the hydrus media viewer? If you hit help->about, what does it say about mpv there?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

69fefa No.14349

>>14316

Thanks, I'll check that out. I bet there is some weird rating tag being parsed by accident.

>>14322

Thanks, got them. When I have a week with a bit of time, I will work the various 'file type' UI to better support larger numbers of mimes and roll in several new types.

>>14333

Good thought. Should be possible, I'll see if I can. It may have to wait a bit for same reason as I just said to >>14322 , but maybe it will fit ok.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

69fefa No.14350

>>14329

Thanks, yeah that is some old info, from when it ran on wx and was not very stable in non-Windows builds. Things are not perfect in Qt, but much better. All my platform releases use the same code, but a couple of library differences in my build environments mean some macOS things don't work as well, and mpv can be difficult to get working in non-Windows generally. I'll update the help.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

95ebeb No.14351

What are some recommended/best practices when using Hydrus?

E.g: Tips, shortcuts etc..

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

80681f No.14353

>>14348

I think both the other anon and myself are referring to mpv as our external video player of choice, not the embedded one.

If I understood correctly you're embedding libmpv into hydrus for the integrated file viewer and preview window, but the compiled Linux build doesn't ship with it and is still using the old native player? Or is there a option for it in the options menu?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

eb9d53 No.14354

File: 9746990340f23d4⋯.webm (10.51 MB,1204x992,301:248,simplescreenrecorder_2020….webm)

>>14348

It's launching the external program

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5b632f No.14355

Is there a way to reset the duplicate filter? As in, to search all duplicates from scratch again.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

eb9d53 No.14356

>>14354

This guy again, not sure exactly sure what I did to fix(break?) it, but it's fixed. MPV is embedding correctly and playing audio.

Thanks for all you do Hydev!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

50ba66 No.14362

File: 9a838a22531f8ac⋯.jpg (58.39 KB,640x596,160:149,Cursed_591ee6_7565731.jpg)

Hello again guys ! Still a newbie but I will finally be able to play around with hyrdus a bit more. I have another quick question that doesn't require a thread: I've just tried the migrating db function from one hard drive to another, on the second hard drive I have extracted the latest version of hydrus (v.398). But the primary drive was not up to date. Now the migration went well and I can still access my files through >Hydrus Network >db >client_files but when I load the program nothing appears.

I understand that the migration removed everything from the primary drive but nothing happens either on the secondary one (where the client files are now stored).

I would just like to know if there was a manipulation that I've forgot to make ? I've thought about the help section "informing the software that the database is not in the default location" but technically, since I have extracted a new installation on a new drive, isn't it its "new" default location ?

Also, if I manually copy the client_files and put them back on the primary drive, will it work as it used to ? I'm tempted to try it but maybe after I've fixed the main issue.

Thanks you guys for your help so far ! Stay awesome !

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7e73de No.14363

>>14348

>>14356

Other guy here. I figured it out too. Didn't realize all it took was to install libmpv and change the player choice in options->media.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

75d496 No.14366

File: 2445611d578c4f2⋯.png (55.14 KB,472x417,472:417,ClipboardImage.png)

How do I delete child-parents pairs?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

995a6f No.14367

File: 9128a2659a5c0e4⋯.png (14.09 KB,363x527,363:527,vinpj4U.png)

>>14362

Are you launching the client through a shortcut with this argument to tell it where your db is found now?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

de92b6 No.14371

>>14367

I've tried but it doesn't work. All the contents are still in the client_files so I can just import them back but I guess I've lost the tags…

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64 No.14373

Is there a way to delete files without leaving the "deleted" record that will affect the future imports? I'm doing lots of experiments and I don't want them to backfire in the future like that.

I know I can disable skipping the deleted files when importing, but it's not the same.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a85ac6 No.14376

>>14373

Yes, under files and trash section in settings, select to use the advanced file deletion dialog. There is an option there to not save a deletion record.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3 No.14378

Could MPV be added as an option for viewing jpg, pngs and gifs? I've already got the scripts and config setup for image viewing and it'd make it a great alternative to hydrus's native viewer due to MPV's customizability.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

17afc1 No.14379

>>13779

I came across this looking for help with why Twitter isn't parsing anything right now, but how would I add this to the Twitter parser?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

143392 No.14384

I am syncing recent updates from the PTR on an hdd and getting 2 rows/s. I know that its supposed to be slow, but is this unusually so for an hdd?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d3358e No.14394

is removing child tags broken in version 399?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

53d208 No.14397

Is there a companion system tag to system:limit that shows the next page/block of that many files? So that way a general search can be browsed page-by-page like on a booru?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7905a9 No.14399

In the tag management dialogs, why can't I select "all known files" and "all tags" at the same time? I need both all files from PTR to select the most commonly used tag among the alternatives, and also consult my own tags that won't be in the PTR.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

296377 No.14402

How do I view all my different tags?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9 No.14404

File: 15447e6d9ba8c06⋯.jpg (374.06 KB,647x1000,647:1000,15447e6d9ba8c06e11dfba4407….jpg)

I have a some files that got deleted somehow (probably HDD corruption) but the thumbnails are still intact and I want to make a backup.

Is there a way to export all thumbnails of files that are missing?

Also, can I tell export folders to ignore errors in the DB? I'm trying to export a TB of files, but it keeps stopping every few GBs due to those missing files.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

05d26d No.14405

how do we improve vid performanc? i have tryed mpv and native hydrus viewer, but both are so slow when it comes against just using external video player. any imrpovement i can do? thx

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d80e1a No.14411

I'm noticing gallery downloader pages stop prematurely with a status of "done" when there's still way more image to fetch. What causes this and how can I keep the search from stoping to soon?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e31a08 No.14412

I've been importing all my media files into Hydrus, and since Tixati (https://tixati.com/) supports renaming and moving files while still allowing me to seed, I've renamed and moved the files to their Hydrus copies, so that I can keep only one copy.

For many of them, Tixati checks and finds that 100% of the file is already downloaded, but others are recognized as 97-99% instead. I don't know if this changes the actual file, but I know you're not supposed to edit files once they're imported.

My question is: is this safe?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e31a08 No.14413

>>14412

I should mention that when it recognizes the file as less than 100% downloaded, it then downloads the rest of it, which is what I'm worried will break something.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c4b652 No.14416

How do i set default settings for new subscriptions?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ed6436 No.14418

>>14416

options > importing > default file import options

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5202b0 No.14420

>>14411

I forgot to mention this happens with sankaku channel parser if that thats relevant at all.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5202b0 No.14421

>>14420

Triple forgot to mention I'm at 11,948/100k+ when the done status appears. I'm logged in as well

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6d446e No.14422

>>14420

Sankaku has a very low daily bandwidth limit, if I recall. You can just remove that one. (Leave the others, they're there for a reason.)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2eef4a No.14423

>>14422

Thanks, I added a global rule to override sankaku's daily bandwidth cap. However I tried setting options to query in my download gallery, but I cna't get the query to resume. Do I need to delete it and start a new query all over again? I was quite deep into this query at 11k images.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a4761f No.14424

File: 599f3613f4d69ce⋯.jpg (14.76 KB,294x301,42:43,1558494002750.jpg)

I have dumped a ton of memes over years and now I can't easily find a specific one among them, so I'm looking for a way to find out if the image files are already tagged by other people and use their tags like a booru.

However, Hydrus Network seems like overkill. Is it possible to ask the program to only download tags for the files I have based on their SHA256/512s? Because I'm on HDD and the public tag repo is huge.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3b2ba5 No.14425

>>14423

Actully the global daily bandwidth increase to 100gb did nothing. New query only got to only a few hundred more items in the 'items' col before stopping with a status of 'done'. Soo wierd

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3b2ba5 No.14426

>>14422

To be honest I dont see a daily bandwith cap in the review bandwith window let alone for sankak.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e7c3f8 No.14427

Logged on while at work for first time in forever and they got a new plus subscription for $10 a month, otherwise you only get 50 pages with a free account. which sucks cuz they have got 10x the amount of results for my query compared to the next closest site. Id pay but they want your real cc so I said f that

It doesnt add up though because each page has 25 images x 50 pages only equals 1250 results when I had 8x that amount of results before getitng hit with 'done' status

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7905a9 No.14430

Is it possible to auto-remove existing tags an image has within a certain namespace when a new tag with the same namespace is being added?

My parser stores certain numeric values like image score in a tag, which would get updated when the same image is visited during next subscription check or manually. Now if the value changed it'll spam the namespace:value tags rather than keeping only the recent one.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c31d11 No.14454

Triaging unanswered posts, please forgive lateness and brief replies.

>>14355

There isn't really a way to do this yet. With help->advanced mode on, the thumb right-click lets you do some advanced 'dissolve' actions that undo dupe relations. With some clever use of the 'system:file relationships' search predicate, you can do some big searches followed by ctrl+a->dissolve to manually do it yourself.

>>14371

I am sorry you have had trouble. Please feel free to email me (hydrus.admin@gmail.com) or grab me on discord if you use that, and we can work on this one on one.

>>14366

Once you have them deleted the first time, so they have that '(-)' status on the left, apply that dialog. They should apply to your 'my tags' instantly once the dialog OKs.

>>14378

I haven't tried this properly yet. The answer I think is yes, and I am not opposed, but there may be some funny business in how I handle big zooms.

>>14379

Twitter killed their old API Jun 1st. If you are still having trouble with this, please grab 400 or 401 tomorrow. Should have a fixed-up nitter parser.

>>14384

I am afraid I no longer recommend syncing to the PTR on a physical HDD. It only really works for SSDs now. It is just a latency thing–8ms latency of an HDD doesn't mix with random access on a 500+ million row, 10GB+ database.

Pause your PTR for now. Read access is still going to be ok. When you eventually move to an SSD, turn it back on. Check here when you do, to see how to put your db on the SSD and files on the HDD: https://hydrusnetwork.github.io/hydrus/help/database_migration.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c31d11 No.14455

>>14394

I have not heard of it–can you describe the problem more exactly?

>>14397

Not yet. I vacillate on this idea. I may end up doing this, but the client has been build around getting everything at once, so actually getting the smaller pages would require me to build webserver-like page caching, and likely wouldn't be any faster to access except on the most common and yet also unchanging queries. For now, it would actually be a good bit of work to get it done, so I am waiting for more thoughts on it and support for it. Maybe there's a different solution we haven't thought of.

>>14399

This all/all domain is a magic one. It used to be selectable, but as the client grew and I needed caches to fetch this data in human time (some autocomplete text searches used to take 30 seconds or more, even with the smaller dbs!), and my caches need a kind of 'pivot' point to reduce the search space. I can quickly do you all of the blah on a tag domain, or all of the blah on a file domain, but not all blah that exists anywhere–not unless you want to wait ten minutes or double the size of your db, anyway!

I may get a strike of inspiration and think of a way to do this better, but for now, it isn't available. I think your best bet is to start with all/ptr, and then move to my files/all tags as your files get populated with the tags you like.

If you are feeling adventurous, you could try creating a new local tag domain and using tag migration to populate it with tags from my tags and the ptr, but this would be dangerous unless you limited it to something like 'all ptr series: tags' or similar, otherwise you are hitting the problem of doubling the size of your db with a ptr copy again.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c31d11 No.14456

>>14402

There is not good support for this yet, and no nice 'tag cloud' or anything else to make it easy to navigate, but if you are a mad lad, you can turn on '*' raw wildcard searches under services->tag display and search. That'll let you just put in '*' on your 'my tags' or similar and get all your tags in the results list. Don't turn it on for the PTR, or you'll be fetching 17 million or something tags.

I'd like to make auto-generated 'favourite tags' tag cloud controls in future.

>>14404

There's no way to save thumbnails for now. They are deleted in the maintenance actions.

However, under database->maintenance->files->review jobs, there are a couple of jobs tuned for your situation. Try what should be the default–'if file is missing and has url, try to redownload'. Queue that up for 'system:everything', or a smaller more targeted search to do a test or if you have a huge db (which it sounds like, with a TB), and it should make a new download page for those files and just get them for you again.

If many of these files are from hard drive imports, not downloaded with a url, I am afraid there is no easy solution now. I think your best bet is just to do the 'if missing, remove record' job once your url redownload job is complete. You'll get a big list of sha256 hashes exported to your db dir, which you can hang on to, which will at least be a record.

I have resisted keeping thumbnails of missing files as for most situations, they are garbage quality and picking through 1,000+ thumbs to search the internet for 'nice' versions or something is too much practical human work, and the url redownload is a better path to pursue when available. I don't want to be a stubborn asshole though, so let me know if the redownload doesn't do nearly enough.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c31d11 No.14457

>>14405

The mpv player is hardware accelerated, so if you have a nice GPU, it should be as fine as an external player–so if it isn't, is there any chance that your Nvidia/AMD driver has somehow been disabled for client.exe? If you dive into the GPU driver settings, has it somehow set hydrus's client.exe to an integrated Intel GPU or something, thinking it is a desktop application and shouldn't have hardware acceleration or OpenCL or similar?

>>14425

Sank cap out logged out users to I think 500 results and logged in users to 1,000 results. It'll 404 at gallery page index 26 or 51 or whatever. You have to mix tags or experiment with advanced id:<200000 kind of tags to dive deeper into boorus that do this.

>>14412

Hmm, I don't know what is going on here. Hydrus doesn't change anything, so my best guess here is that the bittorrent client is somehow seeing 99% when a block spills over from the end of one file to another, and in the rename/move step, it is losisc track of the spare part of the block on the other file, and has to redownload it (somewhere?) or just redownload to check the block hash. In this case, it is probably harmless. If Tixati is putting a thousand 63-byte 46a24e65a4e6546ae246.blah files in your client_files to be the spare part of blocks, that may be a slight pain in the neck for advanced maintenance tasks, but no huge deal.

If you want to do a test, figure out a search that will find just one of these files ('system:hash might do it'), then load up database->maintenance->files->review jobs and add 'if file is incorrect, move out' job for that precise search. Then run that job in the 'scheduled work' tab. This will scan that file for its hash. If the hash is the same (i.e. the file has not changed), it will do nothing. If it is different, it will move that file to a new 'broke files' area in your db dir.

Let me know what you discover, and if I can help.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c31d11 No.14458

>>14424

Unfortunately this is not possible. The way the PTR works, it needs to build that big local cache to stay fast. Best to put this off until you move to an SSD.

>>14430

No, not automatically. I built most of the tag workflow for PTR-like solutions, and much like for files, tags in hydrus are currently more in the 'this is a good tag, keep it', than 'update this regularly'. Content deletion pipelines have always been a bit back-burner.

I had plans years ago to figure out a subjective rating repository that would do this, merging multiple users' ratings and sharing an updated composite store, but I abandoned it. I don't think there's a way to do it nicely in the hydrus way.

I'll likely be adding hydrus ratings to the parser in the next year or two, so the endpoint here is you might have a 1-100 or just 0+ integer score in a local ratings service, and the downloader will overwrite that, and you'll have nice system:rating: blahbooru rating>50 kind of searches supported.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

86842b No.14460

I'm trying to download a pool from Gelbooru. Pasting the id number to the pool downloader page doesn't work, nor does pasting the whole url into a url downloader. Am I doing something wrong or is it broken?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

915c47 No.14462

>>14457

I've done some testing. For probably 99% of the files, the missing bytes are magically added when I add a neighbouring file, so I think you're right.

I also didn't get any broken files folders.

I don't know what made Tixati not recognize some of the files, but I think it's only happened in one folder of one torrent. I found no differences between them, and Hydrus considered them the same file, so I think the problem must be Tixati's. I redownloaded them and merged Tixati and Hydrus's copies in the client_files folders. Now, there are no complaints from either of them.

Thinking about it now, though, those files were much smaller than most of the other ones I tried, so maybe if I'd added all of them in one go, the overlapping bytes would have made Tixati recognize them after all? The only file that was recognized in the same folder was as big as all the other ones put together. I might do a test later, but it's hard to find my files in Tixati now. :p

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7b64d0 No.14465

File: fc50ca40f5d74b1⋯.png (7.53 KB,292x250,146:125,ClipboardImage.png)

Is there a way to add a shortcut to copy the md5 hash instead of sha256?

I know there's plans to update the shortcut system, but I just want to make sure I didn't miss an option.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f2f582 No.14468

File: 3196542f65bcc44⋯.png (11.17 KB,306x321,102:107,ClipboardImage.png)

>>14455

What's the difference between "my files" and "all local files"?

On another note, is it possible to increase the vertical size of the suggestion results area in the tag editing window (see image)? It's far too small to see all the suggestions from PTR when choosing the right way to word your tag.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

19a375 No.14469

Situaton: updated from 394 to 401

Desire: All the tabs are too small to show the custom names and instead read as [ X…) ] where X is the first letter of the tab previously. How do I revert this?

Thanks for the help.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a17990 No.14479

>>14460

Yeah, the default gelbooru downloader only does tag search I think. I see a bigger one here:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/blob/master/Download%20System/All-in-Ones/Single-Sites/gelbooru-tags%2C-pools-and-favs-2019.11.05.png

But I am unfamiliar with it and Gelbooru so I don't know if it solves your exact problem. Download that png and then drag-and-drop it onto Lain under network->downloaders->import downloaders, and you should get either a new downloader listing, or just pool URL support when you paste into a regular 'urls' downloader page.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a17990 No.14480

>>14465

There isn't, but I'll add it!

>>14468

It is subtle. I discovered earlier today that my new autocomplete control is showing too many file domains there, so some stuff that should be hidden behind help->advanced mode is showing for all users.

The main difference is that 'all local files' is an umbrella that contains 'my files', 'trash', and the secret 'repository updates' domains. If a file is in 'all local files', it should be on your disk. It is mostly only useful for some maintenance and presentation contexts where I need to test if a file is in the file structure or not.

In future, when multiple 'my files' local file domains are allowed, 'all local files' will continue to be this nice catch-all for them all.

>>14469

Try the new options on options->gui pages. The bottom box. There is a bit of text about the situation as well.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

86842b No.14483

File: ed039370bb92039⋯.png (37.65 KB,269x611,269:611,client_PcH704TgxF.png)

>>14479

I think I already have that because this is how my list looks like. What happens is that it doesn't find anything and then stops.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f2f582 No.14485

>>14480

Sorry, I couldn't understand your response to my second question. Is there no way to make the results area taller?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

72b0bd No.14487

File: 421a35f8f7f5bac⋯.png (11.15 KB,1479x651,493:217,2020_06_22_21_03_06_galler….png)

I updated to 401 to fix the twitter parser but I still can't download and gives me this error.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64 No.14491

I assume, there's no way currently to host a tag server with a web interface (like a local booru) which would allow other users to add tags to my images?

A local booru apparenly only allows to give a read-only access to selected few images. A server, I think requires the users to have their instance of Hydrus connect to mine, right?

I was thinking about splitting the monotonous tagging work for a large image collection among several people.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3a367b No.14493

>>14487

Be sure to use the nitter gallery downloader now. The twitter one no longer works.

>>14491

No already made way that I know of but you can add/delete tags so you could use that. There are web clients for hydrus already made like hydrus-web but I don't believe that has the ability to add/delete tags at the moment.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d4d53d No.14498

I added the public tag repo and have synced all the tags several months ago.

I've now removed the public tag repo from 'manage services' but I didn't notice any HD space being cleared up. Is there a way to delete the downloaded tag data?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

84ef02 No.14503

>>14483

Ah, damn, it looks like pools use a slightly different html format than normal Gelb search results. I'll let the guys who write these parsers know this now no longer works. With luck, the github repo will update in a week or so.

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/Download%20System/All-in-Ones/Single-Sites

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

84ef02 No.14504

>>14485

Shit, sorry, I missed answering it completely. You know, I think I hardcode that thing's height–I can just add an option for it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

84ef02 No.14505

>>14491

>>14493

Yeah, it is coming, but not ready yet. My first booru was not great, and I never found the time to make it nice, so now I am focusing on the Client API so other lads can make nice web/phone interfaces.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

84ef02 No.14506

>>14498

Run database->maintenance->vacuum->full to clear out most of client.mappings.db. Should be about 18GB freed up. It'll take some time, and you will need about 10GB temporary free space on your system drive and hydrus drive. Maybe a few minutes on SSD, much more on HDD. There is another ~6GB of client.master.db and 3GB of separate json files that can't be cleared out yet, but I will be writing maintenance routines to do this properly in future.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64 No.14509

>>14506

Does the mappings.db contain both local and PTR data? Why not separate PTR into a different file, so you could save space when backing it up, since backing up PTR makes little sense, as it can just be redownloaded?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

feb1cb No.14512

Is this safe? Will it look at my data etc?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7b64d0 No.14514

>>14512

By default there's no network communication. It's open source if you're paranoid.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

feb1cb No.14515

>>14514

Nah, I'm not that paranoid. Stupid spyware the school made me download probably knows my shit anyway, I just like the ease of mind. Thanks btw.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

84ef02 No.14519

>>14509

I have considered variations on this, and I may eventually do it, but I think my limit on simultaneous access is actually pretty small, maybe six or seven db files at once, so I can't go too nuts here before my access system has to connect and disconnect dynamically or something. For now I am just keeping things technically simple and splitting agnostically by content type and assuming db integrity in backup and on boot.

>>14515

Yeah, of course you can't trust my word, but I'm just an Anon making software he wants to use too. I don't want my private shit going anywhere, and I don't want to do it to you either. You have full control, and I have no idea what you do with the program.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6105e4 No.14520

what tab is the option to increase number of favorites? i remember seeing this but cant find it now. wanting to favorite tags for scraping later, but noticing it only holds so much

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ab8957 No.14522

If I have a gallery downloader with 30 queries, how can I restart them all to check for new files?

The best I found is selecting each query -> gallery import log -> try again.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9 No.14523

>>14522

"Select all -> copy queries -> paste" is faster, but it'll redownload files you already have.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

756c08 No.14524

Is it possible to modify an image after its been imported into the database? E.g., navigating to the file and re-exporting/replacing the file with GIMP to reduce the filesize. The original filename and resolution are still the same. Can the database still see the file? Does it know the new filesize?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6c2860 No.14525

>>14523

It doesnt download the same files though? I believe it just rechecks the same pages.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9edf2a No.14529

>>14520

never mind i was mistaken. thought new stuff was not being added to my favorites. altho i will say while here that a button to convert a tag into a partialy done query could be good

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64 No.14530

How do I search for files that have a particular amount of tags within a certain namespace?

For example, to find all files that have two characters.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c77192 No.14531

Can I use a VPN in combination with running a share or a server so that I don't have to give out my external IP address in a link?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ed6436 No.14533

>>14530

Not possible right now afaik

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2b8465 No.14534

>>14529

actually i was right the very first time. idk the exact number it happens at, but once i add so many favorites it kicks out old favorites and i lose them. is there a way to stop this happening or a way to bookmark tags without opening pages for each one

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

13433f No.14536

>>12740

If you want to import tags from a booru then following these instructions is the best way to import the tags: https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/wiki/2.-How-to-fetch-tags-in-bulk-with-md5-hash

You'll need to have advanced mode enabled to do this.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5202b0 No.14538

lovely program, but have some questions. I run more gallery querys than subscriptions, and often forget what querys Ive run and end up running them again. Is there a plan to store old querys somewhere out of sight in case we need them? I could leave the gallery tab open to "save" querys, but it might get acidentally closed and go poof

Also wandering on the tecnical reasons why theres downloaders+subscriptions? why not just a unified subscription that grabs all old files and then syncs new files it hasnt seen before?

ty

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

606fd4 No.14541

File: d8443113661de24⋯.mp4 (2.94 MB,476x476,1:1,source_engine.mp4)

File: 787490cb5bea32b⋯.jpg (40.71 KB,600x405,40:27,this_is_going_to_be_fine.jpg)

File: cfca7e332d1785f⋯.jpg (420.19 KB,2000x2449,2000:2449,honorary_aryans.jpg)

This is a fantastic piece of software and now it's here I'm amazed it wasn't thought of it years ago.

I have a few gripes with it however:

1. My imageboard media folder alone is about 10,000 images. Manually tagging every image to the extent that it becomes easier to find the image I want in the majority of scenarios by using hydrus, rather than by simply going into my folder and remembering the file name or roughly where the file sits in an A-Z ordering, seems like it is going to take weeks if not months of time spent tagging. I can't imagine there's a way to speed up this process so perhaps this is just an unavoidable hurdle to get over for new users, but because of the second and third issue I have doubts as to whether it's actually worth moving over to hydrus in general.

2. The lack of file names is an issue, particular if part of the humour of an image or webm is the file name itself. For most of my webms, for example, I know what they are called or at least know one of the words in the file name, so the fastest way to get to them is simply to scroll to the right place in my folder or search by file name. I don't think the tag system is ever going to be faster than this for a good chunk of my media. Look at these files I've posted for example - assuming I'm unlikely to forget their filenames, is it really likely they'll ever be more convenient to access via Hydrus than an old-fashioned folder?

3. The other advantage to browsing by file names is that you can give a file a name which very quickly sums up what it is without needing to actually view the media itself, in a way that tags either can't do or at least not nearly as conveniently. Infographics and screencaps of long posts are good examples, as you often can't tell what their content is by a thumbnail alone, but a simple title like 'why A causes B,' 'the problem with X,' 'essential guide to Y,' etc., instantly tell you what all the text, graphs or tables in the image show. Tags can get you to the related topics at best, but not the actual purpose/conclusion of the infographic or screencap. If you created a tag for each conclusion, then you're basically creating non-browsable file names so you might as well just go back to a folder.

That said, I have huge numbers of relatively simple images which I just saved with their default 4chan/8chan numerical filenames, and I think Hydrus is probably going become my main method of browsing those. The rest of my media collection seems best off browsed by file name.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5ac2d6 No.14542

>>14541

You can import them with their filenames (e.g. filename:honorary aryans) to get the best of both worlds.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

05ce1e No.14546

File: 4440b62c80bbe61⋯.png (1.86 MB,1400x916,350:229,ClipboardImage.png)

>>14525

>It doesnt download the same files though?

It does. but it'll still download the files instead of just skipping the URL.

>>14538

Move it to a page of pages if it's not already, then right click the pages tab and "save to a session".

>>14541

>1.

>>14542 plus you can autotag depending on the folder.

Besides that you can import tags from other places, like the PTR (a huge general tag repository) or boorus. That said files are not guaranteed to have tags, so I wouldn't count on that for every file.

>2.

Hydrus has wildcard searches that are usually faster than windows search even for hundreds of thousands of files. If you want to be able to sort by "files that have this word/files that start with this word/files that have these words/files that have these words in this order" then hydrus is faster, even ignoring tags. If you know the full name and only use the keyboard to go through folders then it's probably not for you.

>3.

Hydrus allows you to set a sort of name for the file that shows up on the thumbnail. By default it's a bunch of tags in a row like "title - series - volume - chapter" (so it looks kind of cluttered in pic related), but you can change it to whatever you like. You can just have it display the filename and nothing else, and use that as a quick description.

You can also group files by tags when browsing, like I've done with the titles, which is pretty useful depending on what you use it for.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

875a11 No.14549

>install Hydrus Companion

>set it up, make sure it works (Pixiv does)

when I try to run it on Nijie, I get "Could not find a file or post URL to download!". Nijie artist gallery search doesn't work either. I already sent my cookies from Nijie to Hydrus via Companion.

Anyone know how to use Hydrus with Nijie? Am I an retard, or did something break?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6105e4 No.14550

>>14538

still curious why gallery downloads and subscriptions aren't merged into one functionality? i can see 3 benefits:

1. less UI menus makes it simpler

2. gallery download page could be replaced with subscription page and pinned permanently. makes it quicker with no sub menus

3. old queries could be deactivated like subscriptions instead of being trashed. if you type in a query that was previously used, it pops up your old one so you cn rerun it

Also a functionality to rightclick tag>save as query for later would be kewl. Wouldn't need to favorite artist tags or keep a separate notepad of tags for later when u decide to fetch new content

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

48c1d2 No.14559

>>14520

>>14529

>>14534

I don't think there is a limit on favourites. Do you mean 'recent' tags, the ones that are added to a list in the 'manage tags' dialog automatically when you add them to files? I'll specify the three things I can think of:

Search favourites: managed under options->tags, also from tag right-click menu. Appears in tab beside page search results. No limit.

Manage tags favourites: managed under options->tag suggestions->favourites. No limit.

Manage tags recent: managed under options->tag suggestions->recent. Yes there is a limit, set there.

Can you say which of those systems you were using, and how you were adding tags to it? If it is 'recent', then yeah check the options page to boost the memory limit.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

48c1d2 No.14560

>>14522

Your best bet is the subscriptions system: https://hydrusnetwork.github.io/hydrus/help/getting_started_subscriptions.html

>>14523

>>14525

Most of the time, hydrus does not redownload the page or the file of URLs it has downloaded from before. There are a couple of exceptions like pixiv and twitter, where it will get the 'page' (html or json) due to the logic around these sites providing multiple files per page. But boorus and the like will skim along, skipping through previously-seen URLs at about 50 URLs a second, with no bandwidth spent.

You can override the default 'skip url' logic under file import options. If your client redownloads files, make sure your file import options aren't screwed. I only ever recommend this mode for fixing weird small problems, like if a site itself fixes some URL/file on their end.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

48c1d2 No.14561

>>14524

Hydrus does not support this. An assumption of the database is that files are read-only. You can of course change a file in place, but hydrus's understanding of that file is then incorrect, which may lead you into trouble down the line if you try to share that file with other hydrus software or a database maintenance routine ever runs on that file in future.

Hydrus is meant as an archive for 'done' files, so if you want to run optimisers to reduce filesize, I recommend fitting that other software into your import workflow, so download files to one folder, have your optimising step save those files optimised to another folder, and then have hydrus import from there.

If you want to optimise files already in the client, the 'nicest' solution is to do mass_export->optimise->reimport. The duplicate system can merge content back, but it isn't great at this specific problem (it'll probably think the bigger original file is good).

For the moment, this general class of problem has not been high priority for me. As well as it being useful that different users share the exact same files for hashing communication purposes, the space savings of most optimisers are usually in the 10% range, often with imperfect results, and of the various bottlenecks in hydrus's work, actual hard drive space is pretty cheap these days, and still growing, so rather than using gimp to work through ten thousand files to save a few hundred megabytes, it is usually easier just to spend $20 to advance the schedule of your next hard drive upgrade. It sounds stupid, but if you are thinking of doing this resizing by hand, consider your man-hours vs space saved and compare it to the price of just getting more storage. If your man-hours work out to 5 cents an hour, the decision is easy.

That said, hydrus will have 'process this file with this external executable' support (for things like waifu2x) hopefully within the next year, which will have automated tag+rating merge support. Might be easier for you to wait until that is ready.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

48c1d2 No.14562

>>14530

>>14533

I'd like to do that this week. If the db logic is not crazy with current caches, should be good to go.

>>14531

I'm not an expert at this, but I think to do this you need a special VPN for hosting servers, which generally costs more dosh. Basically it is a VPN that either gives you the whole IP or one or more fixed ports on an IP that can accept incoming connections. You then map that to your local machine, and you have a VPNned server. Your standard consumer VPN for $3 a month will not do this (although the way they do torrents does something like this, but I assume that is a specific solution they implement for p2p protocols).

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

48c1d2 No.14563

>>14538

>>14550

For figuring out subscriptions, I recommend just using something as simple as a sticky note application or a .txt document with a list of queries you want to run, and ones you like that you want to subscribe to.

I recently rewrote subscriptions to load fast. Now I have quick access to them, I would like to add new functionality like 'right-click on downloader page query->add to blah sub' and 'if I enter a query for something already subbed, throw up a yes/no confirmation window' kind of thing to improve productivity here.

As for why not merging the two, that sounds like an interesting idea. The general answer to any 'why isn't this thing more useful?' is I am always short on time. There are thousands of things I want to get done, but I can only push on a dozen or so a week. As for why it wasn't designed to be unified, everything in hydrus is pretty much ad-hoc. I just throw a beta together and iterate on feedback. I had download pages early, and subs only came later, once we figured out it would be useful to have a rss-feed-like thing.

I can see the advantages of your idea, and several users have wanted subscriptions that can look back in time and do full syncs, so I think there will be something for that in future. But I'll likely keep downloader pages regardless–many users like the instant feedback of seeing if they like a new artist just by getting twenty files or so, and some users do not use subscriptions at all, preferring to have entirely manual import pipelines.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

48c1d2 No.14564

>>14541

I am glad you like it!

1) Yeah, tagging takes a long time. Any process to automate it is useful. Most booru-style files are well tagged on the Public Tag Repository, but imageboard memes less so. Most files you download from galleries online are well tagged, and hydrus can read pretty much all of those nice tags.

Note that when you import files, you can click 'add tags before import' and add the filename and folders as tags, so you may be able to keep a good bit of your current workflow and retain filename data for later tagging if you wish. You can even set up sorting schemes for a 'filename' namespace, so you might not lose all that much at all.

2) I still don't have a solution for filename threads' files. You can import with all filenames as above, but the funny filenames are lost a little in the wash of 'image1' filenames.

3) I agree. Filenames and folders are fast and work well for small collections, while tags are slow but work for large collections.

As I say in the help, best thing to do is try hydrus with a thousand files or so, without deleting the originals, and see what works for you. You may find the balance is to use hydrus for booru files and keep your memes in folders. Or import some of your imageboard folders with broad additional tags per folder like 'reaction images' and 'thread screencaps' so on.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

48c1d2 No.14565

>>14549

I don't know anything about it, as I don't use the site myself. It could be the downloader is broken. I'll check in with the downloader guys to see if there is a new one available that I can roll into an update.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1aa5cb No.14568

>>14563

Totaly understand the time constrants, just figured it might have been an intentioned design choice. If u ever do consider unified sub+downloader page, may be add a pause/play button next to subs for ones your testing the waters with. Could also add a "limit to # files in total" box or sometng

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1aa5cb No.14569

>>14559

it was the first 2 options. its happened twice now where i add around 100-200 favorite tags (idk exact number) and then all the favorites disappear. im using v 392 right now

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

48c1d2 No.14571

>>14569

Thanks. This is not intentional, it is just supposed to be a list of any length it reads in and out, so I will check it out.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3a1873 No.14574

>>14571

>>14571

upgraded from 393 to 404 after I noticed all the tags I favorited yesterday weren't even added, but think I messed up when transferring the DB folder to the new version while the client was running.

The next version has all the files xfered now and showing my tabs from my last session, but all the pages are stuck on loading and getting "file identifier was missing" and "no such colum: file_notes.name_id" errors. I think the file that got messed up during transfer was the client.db since I remember some message a bout it. any ideas?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

48c1d2 No.14575

>>14574

Shit, this does not sound good. When you transferred, did you 'move' or 'copy'? If it was a copy, then the original db files is likely still, but any running files are hot, so a move may end up in corruption, or perhaps depending on the exact order of how you copied files here, some weird mismash of new and older db files. Do you have a backup?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

64cdff No.14578

>>14575

np i have a backup that's a few days old. i'm lazy though and wanta restore more recently if possible. the missing file identifier mentions a repository reset. is there a button for this or does it mean start over with no files and tags?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

64cdff No.14579

>>14578

nevermind, everything is to mixed to tell what goes where. going to restore and then place the db folder in hydrus 404 with the client running this time. should be just everything in the 393 db folder that i need to place into the 404 folder?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2fc20c No.14584

>>14579

Fantastic, you had a backup!

You can do it two ways:

Extract 404

Copy old db folder into 404's db folder

-or-

Extract 404 onto your existing install

I generally recommend the latter, since it is one quick operation and the path stays the same, but you figure out what you are comfortable with. For some people, extracting onto existing install makes their testicles shrink into their kidneys.

There's more here: https://hydrusnetwork.github.io/hydrus/help/getting_started_installing.html

As always, just make a backup beforehand and you are good.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

122e4c No.14588

I've been having issues with e621 subscriptions for a while now, specifically ever since they introduced breaking changes to their API, so I'm wondering if perhaps I could still have something broken somewhere despite the updates.

Whenever a normal watched query download is performed, I only seem to get one or two new posts for a given tag (using fav:<username> specifically here to watch my own favorites), however when I do a query reset, I get all of the missing posts, so something must be going on with which ones Hydrus is set to ignore. I've already got login credentials set, although I don't really have anything on the default blacklist and thus requiring a login.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

60d61b No.14589

>>14584

update on my update to 405 in the attempt to fix favorite tags being lost: unpause network traffic seems to cause memory to jump 2gb at around 2.5 minutes later, causing linux to kill the program due to too much memory. working on tweaking options to fix and haven't tested filling up favorites again yet

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

86b0fb No.14590

>>14589

disabled "run mantenance jobs" during idle time under options-maintenance and processing. seems to be ok for now when i unpause all network traffic. checked and decompression bombs werent being downloaded, so thats not it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f3cd46 No.14593

Is there a way to rename a whole namespace? I currently have namespace artist: and would like to rename it to creator: so I can use the collection function

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7b5586 No.14594

>>14590

was to the point i'd crash whether subs were paused or unpaused…. i closed my session cause it has a weight of 80,000 (assuming thats not a good thing). didnt fix anything, but then i tried clearing thumbnail cache, clear image rendering cache, and run fast memory memory maintenance in debugging options which seems to have fixed it and made going through images in the media viewer mucho snappier.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a61ac4 No.14596

File: fc7d3b470335013⋯.png (65.64 KB,1200x1200,1:1,1200px_Wayland_Logo_svg.png)

I don't know if or how to get Hydrus working on Wayland Linux.

When I start the client it… starts but nothing really shows up. It is apparently running though.

Here is what I get when I run it within a console - speaking of which, I usually can't just cllick on the client executeable, I have to let it run in a console.

hydrus client started

booting controller…

>Failed to load client buffer integration: "wayland-egl"

Available client buffer integrations: ()

No shell integration named "xdg-shell" found

No shell integration named "xdg-shell-v6" found

No shell integration named "wl-shell" found

No shell integration named "ivi-shell" found

Loading shell integration failed.

Attempted to load the following shells ("xdg-shell", "xdg-shell-v6", "wl-shell", "ivi-shell")

client already running

Well, the greentexted error is probably my issue, but I don't know anything really.

Thanks for reading, Anon.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

604561 No.14599

How do I create like-dislike rathings? The instructions in the help section appear to be for an earlier version of the interface. Is this feature still available?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8d95ab No.14605

>>14588

Thank you for this report. Is it possible that the missed favourites you are adding on the site are older than many other previous favourites (i.e. they have smaller numerical ids)? I wonder if you are getting this sort of situation:

current fav ids: 1000, 1001, 1002

you add: 100, 2000

The sub runs, and somewhere in that process the ids are sorted in reverse order. It gets 2000 as new, then sees enough 'already seen' ids in 1xxx, that it never gets to the unseen 100. I don't think hydrus logic would miss this as it normally goes through a whole page of results for new stuff, but maybe I am wrong and it does, or maybe e621 delivers API favourites in that sort of id order rather than 'favourited' date order, and hydrus is not looking at further pages since it treats all download sources like normal date-ordered booru searches.

If you run this 'favourites' search in a normal page, are the URLs that come back in id order, or time-favourited order?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8d95ab No.14606

>>14589

>>14590

>>14594

Shit, I am sorry for the trouble here. It is only subs that do this? The new subs system is supposed to be much better about lighter memory use. Can you estimate how many subs you have? Is it a dozen queries total, or a thousand? Are they all merged into a handful of subs or spread out across many?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8d95ab No.14607

>>14593

Not yet, but I have wanted this a long time. v408 will bring a new 'siblings cache' at the database level that makes it easier to do. I hope to have it in the near future.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8d95ab No.14608

>>14596

Hmm, I am sorry, I just don't know enough about Linux to talk cleverly. Different flavours of Linux, and their Window Managers in particular, have different success with hydrus. I build in Ubuntu, so if your flavour is way different, it may just not be happy with my .so files.

Although it takes a bit of work, the best solution other users in your position have found is to run from source. There is a full guide here, let me know if you decide to try this but have trouble:

https://hydrusnetwork.github.io/hydrus/help/running_from_source.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8d95ab No.14609

>>14599

Sorry, the help on this is bad. I keep meaning to start new clients with some default ratings, but it slips.

Hit services->manage services and click add->local like/dislike rating service. You can set shape and colour.

SHIT, I just realised when I did multi-column list rewrite the other week, I switched the name/type header labels for manage services list.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f7783 No.14615

>>14606

not your fault, likely my messup when migrating db to new version. I have 1 sub with several querys right now so it probably isnt that, and seems my sesion weigt is right back up to 80k. Things have been fixed and actually even faster than before ever since I cleared thumbnail + image cache–there were several times in the past where I had missing thumnail messages, so maybe those were never resolved until I cleared the cache.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7b64d0 No.14616

>>14614

1. yes, launch with -d or db_dir arg. you can create a shortcut to make it easier. also open a command prompt / terminal and try help for other options

2. no, but you can add tags based on folder path during import

3. I don't understand the question, but the point of hydrus is you shouldn't care about file structure since everything is searchable by tag

4. yes, all imported files are renamed to its sha256 hash. you can also right click on a thumbnail, and look at share > copy > hash

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2d304a No.14621

File: 4b7e5e835489de7⋯.png (5.67 KB,401x142,401:142,Annotation_2020_08_06_0129….png)

Every. Fucking. Time.

It's stuck indefinitely but task manager still shows disk activity.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ed6436 No.14622

Is there a way to, when importing files, get the space separated tags from the filename?

Some boorus name files in this format:

[file number] - [space separated tags].jpg

Can I get those tags into hydrus somehow, when importing the files?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5fab5b No.14625

>>14622

regex

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

14fee0 No.14626

Is there a way to clear Hydrus's memory of deletion records? I'd like to be able to clear that so I can start fresh now that I know how to use the advanced deletion dialog. That way I can ensure that only deletion records are recorded for pictures that I have individually reviewed and decided to delete.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ed6436 No.14627

>>14625

Can you post the regex? I can't figure out how to do it

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ed6436 No.14628

>>14626

I think you can do it in the "migrate tags" menu

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

14fee0 No.14629

>>14628

I just checked and couldn't find anything in the tag migration menus related to deletion records, just tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ed6436 No.14630

>>14629

You didn't look hard enough then.

filter: deleted

action: clear deletion record

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ed6436 No.14631

>>14629

>>14630

Never mind that is for deleted tags, my bad

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ed6436 No.14632

>>14629

Alright I found it, I knew it was there somewhere.

It's in services > review services

local > files > all local files

clear deleted files record

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8d95ab No.14634

>>14614

>>14616

You can't run multiple databases from the same process, but yes you can use the same exe to run different databases at once in different processes. They can't talk to each other yet. Note the switch is exactly:

-d "full/db/path"
-or
--db_dir "full/db/path"

d has one hyphen, db_dir has two. One day, I'll add the ability to have multiple local file domains, which will be a bit like hard drive partitions and allow a kind of multiple databases in one client, or at least an easy way to split sfw and nsfw kind of thing.

For 3, I know what you mean, and you cannot. My full thoughts are here: https://hydrusnetwork.github.io/hydrus/help/faq.html#external_files

For 4, it also tracks md5, sha1, and sha512, and when a site provides that info, hydrus downloaders usually pull that info, and the client can then decide to skip the download and save you time and bandwidth if it knows it already has the file or if it was already deleted before.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8d95ab No.14635

>>14621

Typically, the end of an update holds the siblings and parents. That's those final few rows. Parents atm require all the implied parent tags to be written to disk, so a single 'row' of parents could actually mean 20k tags, which takes time, and the average rows/s speed drops below 1. If there is disk activity, it is doing work. Best just to leave this to idle time.

Also, when I do this 'virtual' parents change in the coming weeks, I hope this process will be much faster.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8d95ab No.14636

>>14627

Every situation needs a little wiggling, but try something like:


For a file
E:\downloads\comic_name\12345 - skirt blue_eyes samus_aran.jpg

Try:

id -> (?<=comic_name\\)\d+
tags -> (?<=\s).+(?=[\s\.])

Basically, for id, 'all the numbers after "comic_name\", and for tags "all characters between space and space or period".

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8d95ab No.14637

>>14636

Ah shit, try:

(?<=\s).+?(?=[\s\.])

instead. That ? makes the .+ non-greedy so it pulls each word as its own tag, rather than the whole string of tags as one.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

14fee0 No.14639

>>14632

Thanks!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ed6436 No.14640

>>14637

>>14636

Thanks, regex on this level is a mystery to me, but I kind of understand how yours work. I just wouldn't be able to create it myself.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f7783 No.14642

>>14637

yo, i'm the poster that had problems with favorite tags resetting to 0 after 200 favorites. givin an update to say after i recovered my db in v405 and cleaned the image/thumbnail and ran memory maintenance i now have 500 favorites with no reset to 0 so far.

now that i got the favorites in order, i wanna see about blacklisting some stuff. it look like u gotta go through several menus to blacklist a tag. any thoughts on adding a rightclick tag option to blacklist once other actual important stuff is done?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b09a4e No.14643

Would it be possible to have Hydrus automatically search for an exact URL if I paste one in the search field, instead of having to select the URL option and then pasting it in the exact URL field?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b09a4e No.14644

>>14643

Or at least an option to set a shortcut to get to the "known url" window, because the "system:" menu has a lot of options. Maybe shortcuts to each item in the "system:" menu are in the works already?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5c8cee No.14647

Hello.

I updated from version 388 to version 407 today and noticed that when I am adding tags through the 'manage tags' window, if I hover over the media viewer window, the focus completely shifts to that window. Then I have to manually re-select the tag textbox to type in more tags.

I used to be able to scroll to the next image with my mouse and type in the page number in the meantime.

Was this change intentional, or is this just a bug on my side?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

66fadc No.14648

>>14593

file > options > sort/collect > add > type in artist

alternatively, type in something like artist-title-page or whatever if you want to incorporate the artist namespace into whatever your preferred sorting schemes are. dropdowns near the top let you set defaults

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ed6436 No.14649

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5dae8d No.14650

>>14642

This is for blacklisting in the downloader, saying 'don't download the file if it has "guro"' etc…? Yeah, there isn't a way to spam this with tags. Since you only usually set it up once, and most users only have a handful of no-nos, I haven't expanded it.

I'll add a paste button to the main input, since that is easy. See if making a notepad of bad tags and then pasting them in one go works out, and let me know if it is still too awkward.

If this is for just hiding tags, yeah atm it only works when you right-click one tag. I think that could also do with improvement.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5dae8d No.14651

>>14643

>>14644

Hmm, that sounds reasonable. I do want to get completely editable system predicates in the near future, so if you are doing lots of individual URL lookups, one after the other, you'll be able to quickly edit the one currently in the search and just put in a diff URL.

Also, one easy thing I can do is let you paste multiple URLs in the search pred, and it'll do an OR search over all of them. Would that make your situation easier, before I get the complex side done?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

340098 No.14662

I added a subscription (nitter retweets) to an account with a long archive, but I really only want to get the new files from this point on. I set the first run files number to 100, but every time it runs it goes looking for progressively older files instead of grabbing the new ones. Is there any setting I can change to make it stop doing that?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e44521 No.14663

File: f7f1996c6b55370⋯.png (4.93 KB,644x50,322:25,Capture.PNG)

is it normal to write a terabyte to my ssd to add the tag repository?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6cb978 No.14668

I setup hydrus-dd and api-thingy. How can it help me tag my files, do I just batch the client_files into it and it just werks? lel

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ab8957 No.14671

Is there an existing/planned support for symlinks/hardlinks instead of "copy," particularly on winblows?

Primarily to save space for large synchronized folders.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9a8486 No.14673

Why Hydrus doesn't support *.jp2 (jpg200)? It's so much better compared to jpg in terms of compression.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e44521 No.14674

File: c9089cfe25511c7⋯.png (382.56 KB,707x849,707:849,Capture.PNG)

god damn the public repository is a clusterfuck

is there a way to hide certain tags but not actually block the images with them?

is there a way to look at all the tags in the public repository?

i was hoping i'd just drop it in and autotag all my shit but i guess not

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

757120 No.14675

Was the in-client backup option removed?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d3358e No.14676

can i import/export the list of previously deleted files?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cda1e1 No.14678

>>14662

Hmm, I am not sure. It sounds like you have it set up correct, so perhaps the way nitter is delivering files is unusual. Is there any chance the files are coming in with a different 'sort' than usual (i.e. not 'newest first'), or it is just a normal username search?

If you go into the subscription and drill down to the precise query, going even further into its edit window, there should be the 'file import status' button which will show you the actual URLs it has been working on, and also the 'gallery log'. These will have the full data, and may shed light on whether it is getting things in the wrong order, or maybe how it is making decisions on fetching new gallery pages. Are the URLs in ascending id (10000, 10001, 10002) order, or are older ids appearing later, after that initial sync? Is the gallery search doing more than one page per subsequent check (it should have info about how it worked in the 'notes' column of the gallery log)? Normally the gallery will be like 'found 5 new urls, not fetching any more pages'.

Normally 100 initial files, and 100 per-check files works well for most sites, nitter included. Does this artist have a lot of retweets of other artists? Do their tweets tend to be single images, or multi-image galleries?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cda1e1 No.14679

>>14663

I am not sure how much it is in total, but it will be a decent multiple of the total end database size, which I think is now about 30-40GB. It makes regular checkpoints to stay safe and not lose data, which involves saving to disk.

Catching up on the PTR is essentially doing about eight years of processing work at once. I generally recommend to leave it work in idle time over a few weeks, but if it is set to do lots of work in one go, it'll add up. If you are completely synced up now, it should settle down to a little bit every day or so.

Depending on how that data is exactly counted in your Windows version, that number may be slightly misleading. I know some versions count 'reading output from an executable' as disk I/O, so when hydrus reads file metadata and generates thumnails from ffmpeg during file import, it can add a casual 400MB when it is really just reading from memory. So if you were doing file imports, or using something like my old video renderer, it may be slightly off. I don't know if those read bytes are legit or include pulling from the disk cache in your ram.

In any case, the PTR involves periods of heavy CPU and disk activity. It is something like 880 million mappings now, and a mechanical HDD can't keep up any more.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cda1e1 No.14680

>>14668

I believe there are several ways to use it.

I think the simple method for already-imported files is:

https://gitgud.io/koto/hydrus-dd#lookup-script

under network->downloader components->SEMI-LEGACY: file lookup scripts, and then turn on 'file lookup scripts' under options->tag suggestions. Make sure the dd server is running. Then, your manage tags dialog for single files will have a new 'file lookup' column with the 'dd' entry, which lets you query the dd server from hydrus with the file and get tag suggestions back.

With the Client API connection, things are more complicated. If you have an external script that can generate your imported files' hash names, I think you can make that script call dd to tag through the client api, like so:

https://gitgud.io/koto/hydrus-dd#for-api-intergration

I don't have personal experience with it. Koto is in the discord, if you are comfortable with that, and some other users who do have experience with it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cda1e1 No.14681

>>14671

I've played with it a bit, mostly Linux experiments, and some stuff to help IPFS access the same files. I am not completely opposed to the idea, but I will first have to extend my file storage system to deal with more error states. At the moment, it assumes that all files are there and actually there. My next plan for file storage is to allow multiple potential storage locations for a single file and to have graceful 'this file storage isn't available atm' mode, which will include multiple 'missing/weird location' error handling.

I understand you can hardlink folder locations now and the file storage system is generally ok with it. Some lads have to do that to get some NAS systems working right, I think.

I'll caveat all this with my thoughts on preserving original folder structures, though:

https://hydrusnetwork.github.io/hydrus/help/faq.html#external_files

If a media collection is large enough that space is a worry, it may not be worth trying to manage it anywhere with a friendly folder structure. I recommend 2GB movies keep their nice filenames and are kept out of hydrus, but all the tens of thousands of webms are just too difficult to keep track of. If you want a small collection of favourite files somewhere easily accessible, then an 'export folder' in hydrus may suit you well.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cda1e1 No.14682

>>14673

Is that JPEG2000? Do you have any example files I can look at and try to get working my end?

>>14676

I don't think so. If you are ok with a bit of SQL, you can pull it right out of the database. Would something like a list of hex sha256 hashes work? Do you want to migrate this list from one client to another?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d3358e No.14684

>>14682

>Do you want to migrate this list from one client to another?

pretty much

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1fb2ed No.14686

>>14651

That would make it easier for sure! It would be great to have all of those features, but beggars can't be choosers!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

14fee0 No.14693

Is there a way to change the default tab names? For example if I create a new gallery tab I would like for the tab name to simply be G. Files tab would be F. Etc.

I often have enough tabs open that I need to rename them to more condensed names. Otherwise they take up so much space that the navigation arrows appear and the Qt glitch that causes the current tab to be on the far right makes it hell to navigate through the tabs. I am aware of the page of pages workaround but dislike that way of doing things.

Alternatively is there a way to enter the next/previous tab using keyboard shortcuts? I looked in the shortcut command options but couldn't find a command that would do that.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7ef9bb No.14694

Would it be possible to integrate YouTube-DL into Hydrus?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

79e6e1 No.14695

File: 0ef0d9d4fdc5704⋯.png (35.5 KB,1365x915,91:61,hydrus_nothing_happening.PNG)

I have tried various tags and sources, but the gallery downloader does not seem to produce anything. Have I overlooked something obvious? Thanks in advance.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9147af No.14700

File: 6394842e4cdc002⋯.png (1.07 KB,368x25,368:25,custom.png)

File: 15c62fc9186529e⋯.png (100.3 KB,708x468,59:39,no_alternates.png)

I wanted to be able to select multiple pics and use a shortcut to set them as alternates, but I could only do it through the custom shortcuts which don't work on the main view.

Did I do something wrong or is there no way to do that currently?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

79e6e1 No.14701

>>14695

Nevermind, I reinstalled it and it works now, but I have no idea what I did differently though.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4a2bd7 No.14702

>>14671

Bumping because it got mixed with the wrong replies:

>Is there an existing/planned support for symlinks/hardlinks instead of "copy," particularly on winblows?

>Primarily to save space for large synchronized folders.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ecde00 No.14703

Any chance of multiple local file domains in the same installation happening within the next few years or it still a pipe dream?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3831ac No.14709

I've been using hydrus for a while now, and it's been wonderful for my image collection.

Right now, I'm trying to do something similar for the zip archives I have (10k+ archives mostly from sadpanda). Is there a tag repository for things like this? I've tried the PTR, but it doesn't seem like the tags for archives is well developed. Another problem I have is the thumbnails. Without having to do intensive stuff like change the code to read into the zip file to get the thumbnail, is it possible to change the thumbnail or find the thumbnail in the filesystem and replace it with the desired thumbnail?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9147af No.14712

>>14709

>is it possible to change the thumbnail or find the thumbnail in the filesystem and replace it with the desired thumbnail?

Yes, but you'll have to do it yourself.

If the file is under db/t[number]/[fille].something then the thumbnail is under db/f[number]/[file].thumbnail

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3831ac No.14713

File: 526b7d0c8156adc⋯.png (221.16 KB,586x320,293:160,previews.png)

>>14712

I don't think this is true for archive files, from what I've seen archive files always uses a static zip image for all files.

I've tried testing this by importing a zip file.

It does generate a regular db/f[number] file, but it doesn't generate a db/t[number].thumbnail file like it does with images.

Placing a dummy thumbnail image in db/t[number].thumbnail (I just took an existing thumbnail image and renamed it to the zips hash) to "trick" it, didn't work, it still uses the standard zip image. So the client must see its a zip file, and only use the zip image.

But, if you try retrieving the thumbnail through the api (http://127.0.0.1:45869/get_files/thumbnail?hash=[number]) it does return that fake thumbnail. So I suppose, the api must have a different logic for retrieving thumbnails. Also, don't do what I just did with "tricking" hydrus by creating a thumbnail for the zip file, and then successfully retrieving it via the api. Once you retrieve it via the api, hydrus stops recognizing that zip file and no longer be searchable.

I'm mostly trying to see if I can create a gallery within hydrus, where the you can preview the archive, by having it show the cover.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ea52ce No.14715

Tried posting this once but I don't think it worked becaues captcha

I can create a new database on a Samba share but I can't move an existing database to the Samba share through any method. I'm on macOS, so I've tried both the built in database migration tool and manually copying ~/Library/Hydrus. Nothing worked. If I manually copy ~/Library/Hydrus to a USB drive, I can start Hydrus from that database successfully.

The error and client.log just say "serious error" and dump out a call stack trace. Here is the trace:


2020/09/11 12:28:11: Traceback (most recent call last):
File "Hydrus/hydrus/client/ClientController.py", line 1798, in THREADBootEverything
File "Hydrus/hydrus/client/ClientController.py", line 803, in InitModel
File "Hydrus/hydrus/core/HydrusController.py", line 498, in InitModel
File "Hydrus/hydrus/client/ClientController.py", line 189, in _InitDB
File "Hydrus/hydrus/client/ClientDB.py", line 299, in __init__
File "Hydrus/hydrus/core/HydrusDB.py", line 185, in __init__
File "Hydrus/hydrus/core/HydrusDB.py", line 454, in _InitDB
File "Hydrus/hydrus/core/HydrusDB.py", line 494, in _InitDBCursor

(don't know if code tags work here, never posted on 8kun)

and the offending line:


def _InitDBCursor( self ):

self._CloseDBCursor()

db_path = os.path.join( self._db_dir, self._db_filenames[ 'main' ] )

db_just_created = not os.path.exists( db_path )

self._db = sqlite3.connect( db_path, isolation_level = None, detect_types = sqlite3.PARSE_DECLTYPES )

self._connection_timestamp = HydrusData.GetNow()

self._c = self._db.cursor()

if HG.no_db_temp_files:

self._c.execute( 'PRAGMA temp_store = 2;' ) # use memory for temp store exclusively


self._c.execute( 'ATTACH ":memory:" AS mem;' ) # ERROR HERE

I can't figure out what the problem is. My first thought was some kind of permission problem, but the file permissions are the same whether Hydrus creates a DB on the Samba share or whether I manually copy or use the built in migration to move an existing DB there. The permissions match the ones of a DB on my local machine (rw-rr on files, rwxr–r-xr-x on directories), and they appear as owned by me and I can read/write/delete them no problem. I'm hoping someone with experience with Hydrus internals can at least point me to things to try, because I am lost.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

fc44d8 No.14716

hi this is my first time posting here

I use manjaro and I downloaded Hydrus and wanted to get a server running, but when I launch the server executable it doesn't seem to do anything. I then tried running it by using "python 3 server.py and got this error in the hydrus_crash.log


Traceback (most recent call last):
File "hydrus/server.py", line 120, in <module>
File "hydrus/hydrus/server/ServerController.py", line 32, in ProcessStartingAction
EOFError: EOF when reading a line

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9229cc No.14720

I don't seem to be scraping translations on gelbooru.

Is there some parser that will grab these ("notes" / page_notes on gelbooru)?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ea52ce No.14721

>>14715

Found my problem. I just ran a text diff on client.db created on my local machine vs one created on the Samba, and a few bytes different in the header led to me discovering that the database had journal_mode=WAL on my local machine but the "legacy" journal mode when created on the Samba share. SQlite docs say WAL-mode does not work with DBs on network storage. I also found the Hydrus option no_wal. Running Hydrus normally, creating a new WAL DB, then running Hydrus no_wal to change it to legacy journaling, and then finally copying it over to Samba and running Hydrus again worked. Going to try it with my actual DB now. I don't have SQL experience so please let me know if what I am doing will lead to breaking things in the future. It seems to work for now.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.14722

>>14684

I am sorry I have been procrastinating on this question. When I looked at the SQL, it is actually a pain in the ass to insert back in manually. Since this has been asked a couple times before, I will just write a way in the client to do import/export, and then it will be easily sorted.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.14723

>>14693

There's no keyboard shortcut yet, but I want this easy navigation.

I will think about adding options to edit the default names. Although I can see your problem, I also encourage slender gui sessions as much as possible. If a client has dozens and dozens of pages open, a better solution is often to figure out a neater workflow, perhaps with temporary processing tags on 'my tags' and saved searches to quickly bring those files back. I will think about it though.

I collpased my mess of static 'sort this shit out' pages down to a single 'processing' page when I added favourite/saved searches, and it has been a godsend.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.14724

>>14694

I would like to. My current plan is to write a 'external executables that (optionally) eat files and spit out files' system that we can enter any exe into, and then plug that into the downloader or general file conversion. youtube-dl, ffmpeg, imagemagick, waifu2x, anything like that.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.14725

>>14695

>>14701

I am sorry for the trouble. I am not sure what was going on there, it looks like the downloader jobs did not start. Please let me know if you get this again.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.14726

>>14700

Do you have help->advanced mode turned on? I think I hide some of those clever commands behind that.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.14727

>>14702

Ah, do you mean for 'export folders'? I experimented with this with manual export (I think you need help->advanced mode turned on), and I did have a plan to try this out for export folders. I'll bump this up and take a proper look.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.14728

>>14703

It is not a pipe dream, but it is such a big job that it'll have to win a 'big job' poll. I expect to get through three or four big jobs a year once I am back to normal schedule, and the next poll should be up by Christmas. Let's see how it does there, and we'll see when we can expect it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.14729

>>14709

>>14712

>>14713

I plan to have nice support for traditional zip/cbz/cbr files in future. It is sufficiently popular that I expect it to win a 'big job' poll pretty soon. I'll have proper thumbs, browsing of internal pages, and support for conversion back and forth between individual files and compiled 'chapters' or 'volumes' or whatever. This will be a part of my attempt to solve the problem of handling awkward 'page' tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2 No.14730

>>14715

>>14721

Thank you for this information. I will attempt to detect WAL and networking gubbins in future. I generally do not recommend running the db itself across a network share due to latency issues, but I hope I can at least detect the situation and throw up a warning, and ideally recover and set WAL off.

NEW THREAD

>>14731

>>14731

>>14731

>>14731

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
Post last edited at



[Return][Go to top][Catalog][Nerve Center][Random][Post a Reply]
Delete Post [ ]
[]
[ / / / / / / / / / / / / / ] [ dir / random / 93 / biohzrd / hkacade / hkpnd / tct / utd / uy / yebalnia ]