[ / / / / / / / / / / / / / ] [ dir / random / abcu / cutebois / hentai / k / loomis / mai / pdfs / rzabczan ]

/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.
Learn more about the EARN IT Act, the latest attempt to gut Section 230
/fringe/ has been migrated.
Name
Email
Subject
Comment *
File
Password (Randomized for file and post deletion; you may also set your own.)
Archive
* = required field[▶ Show post options & limits]
Confused? See the FAQ.
Embed
(replaces files and can be used instead)
Voice recorder Show voice recorder

(the Stop button will be clickable 5 seconds after you press Record)
Options

Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf
Max filesize is 16 MB.
Max image dimensions are 15000 x 15000.
You may upload 5 per post.


New user? Start here ---> http://hydrusnetwork.github.io/hydrus/

Experienced user with a bit of cash who wants to help out? ---> Patreon

Current to-do list has: 2,017 items

Current big job: Catching up on Qt, MPV, tag work, and small jobs. New poll once things have calmed down.


File: b28babaa13b7dbf⋯.png (765.02 KB, 1278x720, 71:40, b28babaa13b7dbfb92d271ee0a….png)

2f807c  No.12641[Last 50 Posts]

Here you can ask questions so that the board is not clogged with small threads.

Old thread >>9327

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8dac8a  No.12643

Is there a good place to discuss tagging standards for the PTR? There has been plenty of talk on what tags and namespaces we should use, and I think there should be more. Maybe it's worth its own thread.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

71077a  No.12644

>>12638

Siblings are useful, but being able to right-click a tag in the manage tags window and select "replace tag with…" which lets you type in a tag to replace it with would be great. This would only affect the selected files that have that tag. I might still want to keep that tag on other files I haven't selected, so a sibling isn't helpful. I could manually remove the tag from the selection, but then I can only add the new tag to all of them.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

17913a  No.12646

i'm working on tagging approx. 18k files right now with rating tags, i.e. rating:explicit, rating:questionable, rating:safe. i'm trying to figure out if there's an effective way to do this automatically.

I've set up obvious tags as parents for rating:explicit, so explicit is applied automatically with practically no problem. is there a way to say to the client "apply tag rating;questionable IF file does not have rating:explicit AND file has tags (tag1 OR tag2 OR tag3)"

the reason I want to do this is because if i set a parent relationship for let's say tag breasts->rating:questionable, then i'm going to end up with a load of files with both rating:explicit and rating:questionable since the parent relationship does not check for the rating:explicit tag.

also, mr dev man, i love your program. thanks a ton

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a49c88  No.12647

>>12632

>sounds like a very subjective description of namespaces to me.

Tagging is entirely subjective to begin with.

Programmatically, namespaces are used for disambiguation to prevent naming collisions. What this means for tagging is that in practice you should have namespace_A:same_character_name and namespace_B:same_character_name to disambiguate two characters. People do shit like "character:Yuno (Mirai Nikki)" and "character:Yuno (Black Clover)", cluttering the tags with two variants of Yuno instead of providing a namespace for each of them (eg: "black clover:yuno" "mirai nikki:yuno"). This is probably done to avoid cluttering namespaces with is often its own namespace (series: or copyright:). In practice, almost nobody uses namespaces solely for disambiguation. They use it for meta information like "series:" or "artist:".

>not sure what the difference to you is between having two namespaced tags "hair:short","hair:long" and having two regular tags "short hair","long hair". Either way you have two tags.

I explained the difference. Information about the image (meta) vs information that describes what the image is (descriptors). This is also how every booru I know of works and how I choose to tag. Boorus will use the tags "black panties" and "black dress" and not namespace them under "black:" like "black:panties" and "black:dress". Mixing descriptors with meta information is messy as many things can be swapped with their namespace and still make sense. For example, which is more valid: "dress:black"/"dress color:black" or "black:dress"? Why would you namespace one and not the other? Would you use both namespaces?

"hair:ponytail hair:ribbon hair:black hair:short" is clutter and even if you break each "hair:" down into more specialized namespaces like "hairstyle, hairaccessory, haircolor, hairlength" you only worsen the clutter.

Compare: "ponytail ribbon black_hair short_hair" vs "hair_style:ponytail hair_accessory:ribbon hair_color:black hairlength:short"

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

146181  No.12650

>>12647

Character: Yuno Gasai is different to Character: Yuno, the Meal-Saving Prince. Clutter is a buzzword to excuse minute and less important tags from being tagged on images. A dress being black doesn't mean it's not in the style of Dior's iconic Little Black Dress, and so a Dress:Black and Black:Dress can be applied to the same image for the same garment and mean two completely different things. If a user believes that Hairstyle:, HairLength: or HairColour: tags are too specific, they can simply not use them. Hydrus allows certain namespaces that the user deems unimportant or uninteresting to be left out when viewing a certain tag or groups of tags. It's called Censoring. Hydrus does NOT allow for tags to appear on images where there are none. Therefore, it is the duty of any user of the PTR to overtag images and allow the user to pick and choose which tags or namespaces they see fit to read or discard according to their preferences.

All of this relies on people actually using namespaces, though, which people still don't.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a49c88  No.12652

>>12650

>Character: Yuno Gasai is different to Character: Yuno, the Meal-Saving Prince.

No shit, that's the point of differentiating them. I excluded last names for the sake of example but had a feeling you'd be insufferable enough to ignore the point. So here's a more solid example of two people with the same first and family names.

Michelle Williams (All the Money in the World) and Michelle Williams (Destiny's Child)

Both would be tagged "Michelle Williams". This is what namespaces are *meant* to resolve and where the name "namespace" comes from as a programming term. You'd provide a namespace for each Michelle Williams as a way to disambiguate between two otherwise identical names.

>Clutter is a buzzword to excuse minute and less important tags from being tagged on images.

If you noticed, I didn't remove any "less important tags". I simply didn't give them redundant namespaces that provide no additional value over a plain tag and only serve to add clutter the search terminology.

>Therefore, it is the duty of any user of the PTR to overtag images and allow the user to pick and choose which tags or namespaces they see fit to read or discard according to their preferences.

I don't use the PTR and am not speaking about public choose-your-own-tags-and-censor-stupid-tags group tagging. Most people are retarded when it comes to tagging things which is why once boorus grow past a point their tag quality goes downhill fast with very poorly or even misapplied tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8dac8a  No.12658

>>12652

>which is more valid: "dress:black"/"dress color:black" or "black:dress"?

If I wanted to namespace this tag (I don't), "dress:black" makes the most sense. "dress color" is unnecessarily specific, leading to more namespaces than needed.

However, if I cared for searching by/adding/removing dress colours often, then I could more easily find all dress colours in a long list of tags if there was a "dress" namespace. This applies to a list of tags on an image, as well as search results for tags having entered only part of the query (e.g. just the namespace, or just the colour).

This philosophy works well with Hydrus, and does not increase the number of tags. More namespaces are introduced, but only for what you are interested in.

The only problem is that one has to add local siblings for all tags they wish to namespace, and possibly add the reverse siblings to the PTR if they want to easily mass upload tags.

>had a feeling you'd be insufferable

Now, now

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a49c88  No.12661

>>12658

>If I wanted to namespace this tag (I don't), "dress:black" makes the most sense. "dress color" is unnecessarily specific, leading to more namespaces than needed.

Not saying you wanted to use these namespaces - but I was pointing out that black:dress and dress:black are both equally valid ways of tagging the same thing using namespaces. "dress color:black" was just used as a more verbose "dress:black", the comparison was more between "dress:black" and "black:dress". Further examples of these namespaces, you might have "dress:wedding" and "black:shoes".

>However, if I cared for searching by/adding/removing dress colours often, then I could more easily find all dress colours in a long list of tags if there was a "dress" namespace.

I don't believe this is a common enough scenario or a significant enough increase in quality of life from scanning an alphabetical listing of tags for "color_dress". It takes maybe an additional second, if even that, to see if an image is tagged "red_dress blue_dress black_dress" instead of "dress:black dress:blue dress:red" even on my images with ~70 tags. It provides that small amount of QOL with the tradeoff of either increasing the complexity of the tagging system (rules about what belongs in the "dress:" namespace) or increased verbosity or redundancy when searching.

Complexity: "hair:long black_hair ponytail ribbon (Complexity is defined as having to clarify "Which values belongs in the 'hair:' namespace?")

Redundancy/Clutter: "hair:long hair:black hair:ponytail hair:ribbon" (Reduced complexity, "All hair related values belong in the hair namespace")

Verbosity: "hairlength:long haircolor:black hairstyle:ponytail accessory:ribbon" (Removed redundancy and reduced complexity by being more verbose)

No complexity, no redundancy, no verbosity: "long_hair black_hair ponytail ribbon"

>…as well as search results for tags having entered only part of the query (e.g. just the namespace, or just the colour).

"hair:" returns "hair:long" and "hair:black" (and all the other length/color in the hair: namespace) and "*_hair" returns "long_hair" and "black_hair" (and all the other length/colors tagged as adjective_hair).

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

923d52  No.12663

>>12643

>Maybe it's worth its own thread.

Sounds good to me, it's a bit messy to mix it in with Q&A

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

69bc83  No.12666

How do you get Hydrus to download Explicit tags on rule34hentai where you need an account to see? The Hydrus login doesn't do anything and Hydrus still skips these when downloading from that site.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

69bc83  No.12667

How do do you undo an imported downloader from an png? I tried importing a downloader from an png and I don't really like how its getting tags so I want to go back to its default settings.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7fa1bd  No.12668

>>12667

Is namespaces the only way to sort tags?

Is it possible to add them to sorting list to have them presented in a way that looks logical to humans?

If it was possible and if coloring a tag was moved from namespaces to these list, I don't think we would need so many namespace wich essentally multiply the tag in the database.

dog and species dog both exist.

This may be stupid, though, I'm no programmer.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a49c88  No.12671

File: ba02e151b04e6f6⋯.png (6.15 KB, 436x147, 436:147, 1558575951.png)

>>12668

These are the tag sort options. Currently there are no ways to manually give sequencing or priority to individual tags. Also only namespaces can be colored. There is not currently a way to override "unnamespaced tags" to be colored on a tag level, to my knowledge, but it is probably something that is capable of being added.

May I ask why you feel that dog and species:dog are redundant tags? Personally I use "dog" for any actual dogs and "species:dog" for anthropomorphic characters. For example, I don't have a "catgirl" tag, I use "species:cat" while a pet cat would simply be "cat".

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bc2b78  No.12673

Has the Linux version of Hydrus gotten any more stable? How do I migrate my DB from windon't to linux?

Will it ever be as stable, or will this always be a primarily windows-focused application?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cc18da  No.12674

>>12673

Im running it from source, so far only once hydrus crashed.

>How do I migrate my DB from windon't to linux?

Same as in Windows.

>Will it ever be as stable, or will this always be a primarily windows-focused application?

idk seems pretty stable to me.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2ede7d  No.12676

My database sees only 21 images out of few 1000. Even though they are there. Everything else is considered orphans. Am I fucked?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c9523f  No.12684

>>12674

That is nice to hear, last time I tried it on ubuntu it would crash constantly.

Maybe there was an error in my setup.

I'll try it soon!

Do you think there is any way to easily remove the more smutty compnents of the program (explicit references to hentaihaven/gelbooru etc.) becuase I want to use it in a way where it'll be relevant to cataloguing photos of family and friends? (i.e.: father, mother, brothers will see the program).

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

eed651  No.12687

How come the installer fails everytime I try to download it?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7fa1bd  No.12688

>>12671

I used that example because the tag "dog" is aliased to "species:dog" in the ptr.

I too would prefer it not to be a defacto alias.

I wager you either don't use the ptr or found a way to blacklist alias locally?

I wonder if the way we use namespace is necessitated by boorus compatibility rather than being the most efficient way to sort tags.

Using a list system would however requires more user involvement though, which doesn't seems a good idea for such numbers of files.

In the same vein I wonder if affix and suffix are possible in lieu of those qualifier, by which I means linking up to tags togethers.

"character name" is affixed to "series it come from".

It's 100% not doable now but may be technically possible, perhaps.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

21e517  No.12689

I'm getting a filenotfound error even though the file exists on my system, not sure why this is happening? db diagnostics say nothing is wrong

I created the db on MacOS and have since imported to Windows, exact same db works on mac but fails on windows… did I miss something during my migration? (db was fully portable)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a49c88  No.12690

>>12688

I don't use the PTR because people tag like retards, so you're right. :)

>I wonder if the way we use namespace is necessitated by boorus compatibility rather than being the most efficient way to sort tags

As I explained somewhere, namespaces are used to differentiate two things that have the same name. It comes from programming, especially object orientated programming, where you'll have many objects that may share the same name but aren't the same thing. For example "second" and "second". "time:second" and "position:second" provides disambiguation. ( See: https://stackoverflow.com/questions/4211827/why-and-how-should-i-use-namespaces-in-c )

People don't use namespaces properly and end up with "(clarifications)" in their tags which, in my opinion, is quite ugly. Because disambiguation is rarely needed, it's boorus often use it for *meta* information like who drew the image or what the image is a drawing of rather than what it contains.

>"character name" is affixed to "series it come from".

This is already double by making use of child-parent relationships and if you don't tag like a retard. Unless I'm misunderstanding you. :)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1f9246  No.12696

I'm still kinda new to all this, but im wondering why theres a image limit on the twitter downloader? It only seems to stop at a point. Also will it ever be possible to download gifs and videos off from twitter?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8dac8a  No.12700

Has anyone ever noticed booru subscriptions consistently failing to pull certain tags? I am suspecting that any tags which are replaced by a local sibling are not getting added to the image at all.

For me, this is a Sankaku subscription. I'll look into it a bit more.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

06eef5  No.12721

Where can I find other downloaders? Or are all of them already installed?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9bf8dc  No.12727

>>12684

>Do you think there is any way to easily remove the more smutty compnents of the program

At the very least you can hide them from view from network -> downloaders -> manage downloader and url display. I think you may be able to entirely remove references to them through network -> downloader definitions and clicking on each submenu and deleting everything in there.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c74f94  No.12731

File: f7ebb1b57225a86⋯.png (208.86 KB, 927x794, 927:794, 2019-05-27-111720_2560x144….png)

Hydrus has every feature I need, but the major deal breaker was that it wants to copy everything to it's own folder, when I rather it, like music libraries + players, just to monitor a list of directories.

Is this still the case? If so, can there be changes where copying "imported" files to its own directory is entirely optional?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

38f133  No.12732

>>12731

>If so, can there be changes where copying "imported" files to its own directory is entirely optional?

Yes, but that would fuck with the entire program. Hydrus depends on files being in it's own folders in order for half of its features to work.

There might be some workaround in the future but no plans yet AFAIK.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c74f94  No.12733

>>12732

The only thing I really want is some aggregate, search, catalog, tagging features. Should I be looking for another (simpler) program?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

af4ce4  No.12734

Not quite sure how to ask this, but is there a way to purge all PTR data from my db?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6f816a  No.12736

>>12684

>That is nice to hear, last time I tried it on ubuntu it would crash constantly

Sorry for late reply, DO NOT USE THE PORTABLE VERSION, RUN IT FROM SOURCE.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.12737

I could swear I asked this already but maybe it didn't send as I can't find it.

Following scenario:

One PC acts as a local server, nine others use that local connection to access files on the server. Would it be possible to put the DB on the server and access it from all other PCs?

What happens if several clients try to change things?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8dac8a  No.12739

>>12734

have you deleted the ptr from the services menu?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

edfbea  No.12740

File: 56150d5bf7aac98⋯.jpg (133.86 KB, 850x963, 850:963, rainbow_vomit.jpg)

New to Hydrus, feeling like a proper retard. The public tag repo's account referenced here: https://hydrusnetwork.github.io/hydrus/help/access_keys.html seems to be maxed out.

An alternative way to get tags imported seems to be to import a tag archive, BUT it looks that functionality has been removed in the current version?

So how do I get tags? I've searched all around and I can't find a resource explaining how to get tags with the current version.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5b4ce7  No.12742

How should I tag variations/revisions of a illustration?

I found using a combination of page and chapter (what should the chapter be even called?) with collections produces the desired effect, but it seems like a terrible solution.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ad3314  No.12743

>>12740

Just going to have to wait it out. The public tag repo is unaccessable until the bandwidth limit resets at start of next month. In the meantime check out https://cuddlebear92.github.io/Quicksync/#two and use the services -> import repository update files menu to get a head start on updates.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

af4ce4  No.12744

>>12739

That was the first thing I tried, but it didn't seem to do anything. I somehow got client.mappings.db from 15GB to 3.5MB (I think I exported my local tags, deleted client.mappings.db, and re-imported my tags), but client.master.db only dropped from 4GB to 3.5GB after vacuuming

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

38f133  No.12745

File: de9f0f1a4e298a0⋯.jpg (26.61 KB, 347x450, 347:450, de9f0f1a4e298a01561b7f9354….jpg)

>>12740

Shit's fucked. Like >>12743 said, you can only wait.

>tfw the ever growing tags to petition only makes me more afraid to do one big fuck up like accidentally petitioning all tags to the PTR again and having to erase it all

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b459c1  No.12746

>>12740

Use >>1914 or do the individual file lookup. The problem with that is if the hash doesn't match then you'll get nothing. That's why I run everything through IQDB and use the script to collect it all for import

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9bf8dc  No.12750

>>12737

>What happens if several clients try to change things?

I'm not sure which thread it was in (it was sometime within the last month or two) but I'm pretty sure Hydrus Dev said not to do this. Something about SQL not being able to handle multiple programs trying to lock/unlock the db.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12794

>>12644

Nice idea, thank you. I know what you are getting at with the current workflow not being able to do this for a multi-file selection. I think I can do this, but I can't promise it super quickly.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12795

>>12646

Great, I am glad you like it!

Best way to do that for now is with a search page that you hit F5, ctrl+a, F3, ctrl+v, enter, enter, ctrl+r on every week. A search like:

[ (semi-lewd tags), -rating:explicit, -rating:questionable ]

Then hit F5 to refresh and see all the files that apply. ctrl+a, f3 to bring up manage tags. ctrl+v, enter to add 'rating:questionable', then enter to close manage tags and ctrl+r to remove all selected files from view.

There isn't a way to do kind of 'logical' tag siblings/parents that rely on more complicated tag states and implications. I have thought about it several times in a 'wouldn't it be nice to add tag x if both tag y and z were present' way, but that is a lot of power and I can see how it could go extremely wrong if I did not wrap it behind two advanced modes. So I am a little wary of it atm and am ok that human eyes go over complicated jobs like this for now.

rating is a very subjective namespace. I agree that explicit is going to be easier to set up than questionable or safe.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12796

>>12666

I have heard the R34H login script can be finicky, just failing sometimes. If you are absolutely certain you have the credentials set up correct in manage logins dialog, and that the login script is active, try hitting the 'reset login' and 'do login now' buttons on the same dialog, just to retry.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12797

>>12667

Hit the manage dialogs under network->downloader definitions menu and delete the new objects. You are probably talking one GUG, a couple of url classes, and a couple of parsers. If they overwrote old objects, those dialogs should have 'load from default' buttons or similar. Once you are all set up, hit the manage url class links dialog and make sure the 'url classes' you want are connected to correct-sounding parsers. You can click the 'try to link…' button on that dialog to try to auto-fix the links here.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12798

>>12668

I would like future versions of the tagging system to support metadata about tags to permit custom sort orders. Future tag siblings will allow you to better control your various dog/species:dog preferences locally just to your client.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12799

>>12673

If I could wave a magic wand, I'd love for the Linux version to be more stable. I'll be putting some time into improving multiplat stability in about three or four months. Unfortunately I am a Windows guy so just have less experience and time spent on different Linux flavours and window managers.

The stability problems on Linux are mostly to do with my shit UI code interacting with stricter/different window managers that want things in slightly more proper ways. I'll keep pushing on the problem, but I can't promise excellence for all systems in quick time. That said, some Linux users experience no problems at all with source or the built release. I recommend you try it, see if your situation is good enough for you, and plan to be ok with staying on Windows or whatever.

To migrate your db, the basic rule is just to install on Linux and then copy your install_dir/db folder across. Don't do this if you are uncertain–feel free to ask me for more details if you need help. Make a backup beforehand. Background reading here:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12800

>>12676

This sounds like a serious problem. Please check the 'help my db is broke.txt' document in install_dir/db. If the files are legit still in your client_files folder, but just seem to have disappeared from your client's records, there are ways to recover. I am happy to help with all of this, but first you need to make sure your hard drive is healthy, which that document will walk you through. Feel free to send me an email if you want to work more one-on-one with this.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12801

>>12684

>>12727

Yeah, delete the stuff from downloader definitions. Bear in mind this is imageboard-centric software. Anyone who googles 'hydrus network' is going to end up here pretty quick, so I recommend you sit in the cockpit and just drive some slideshows if you don't want to be asked questions about the INTERNET HATE MACHINE.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12802

>>12687

Have you got an anti-virus that doesn't like something in it? This has happened a couple of times before. The 'upnpc' executable file in install_dir/bin are common culprits for false positive detections, and I know some scanners can see inside the archive of the installer exe.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12803

>>12689

Can you say a bit more about your migration situation? If you open up database->db migration, does everything appear to be in the right place there? What's the first two hexadecimal characters of the file's hash, and is it in the right 'fXX' subfolder for that? If the client gives you path details in those FileNotFound errors, do they line up with what you are looking at, or are they old/incorrect to what you expect?

Also, if this is a video, is the actual error about not being able to find ffmpeg, which hydrus uses to render video? Do any videos work? What is ffmpeg's apparent version under help->about?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12804

>>12696

There are two limits with twitter–one is a ~2,800 tweet limit caused by their API, and another is if the user has a series of 20 or so tweets without media content, the downloader bails early.

Twitter videos is tough. twitter deliver videos as streamed mp4s, so no easy URLs for hydrus to latch on to. I know one user is working on this problem. Apparently twitter can give you direct vid URLs from the API, but it takes an API key.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12805

>>12700

Can you check your options->tags for 'Apply all siblings to all services'? If it is checked, would unchecking it help? I expect to extend this system in future so you can control which services can apply to which others, but for now it is all or nothing.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12806

>>12721

Lots here:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/Download%20System

Check the 'all-in-ones' folder for easy-import pngs. Bear in mind the quality of these downloaders varies a lot. Test them a bit to make sure you only get the tags you want, and don't import a thousand all at once or you'll fuck up your UI with gigantic lists of shit you don't care about.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12808

>>12731

Here are my current feelings on 'external file' management:

https://hydrusnetwork.github.io/hydrus/help/faq.html#external_files

It is technically possible, and if it were a simpler problem all around I'd do it, but I just don't think I can put the maintenance time into this sort of system. If you only want easy-external access to a few thousand files, you may be able to cobble together what you want with an 'export folder' that maintains a synced folder of a search and gives the files nice filenames based on tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4  No.12809

File: 25d07823678a708⋯.jpg (125.19 KB, 626x850, 313:425, 25d07823678a708de5fa6cd342….jpg)

>>12806

Nice.

Is there a list of all boorus that allow md5 or some other type of md5 search?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12810

>>12733

ACDSee is good. If you only want to catalogue a few thousand files and external management is critical for you, that might be a better fit.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12811

>>12734

>>12739

>>12744

Unfortunately, my current 'clear orphans' tech doesn't do master definition recycling. I plan to add this, so it will go down eventually, but we just aren't there yet.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12812

>>12737

>>12750

Yeah, there aren't great solutions yet for a single client accessed from multiple locations. I do not recommend you run a client.db from a network location, as the locks the network drive provides are usually fake and not good enough for what SQLite needs to be safe.

Running multiple clients at once from multiple network locations will cause your db damage.

The Client API is likely to open things up here. I know users who are working on phone/tablet/web browser wrappers that present you booru-like interfaces.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12813

>>12740

Yeah, sorry, PTR bandwidth is tight atm. It is now free again but will likely block again for the last week of the month. I have a plan to fix this in 2-3 months. You can just leave it to sync and it will catch up when it can.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12814

>>12809

I don't know. Most boorus share the same software, so if you know Gelbooru 0.2.0 supports it in some way, any booru that runs that should be good. Doing a search for '(booru) advanced search' or '(booru) cheatsheet' can usually get you to a nice page like this:

https://gelbooru.com/index.php?page=help&topic=cheatsheet

https://danbooru.donmai.us/wiki_pages/43049

https://e621.net/help/show/cheatsheet

https://konachan.com/help/cheatsheet

So looks like gelb 0.2.5, danb, e621, and moebooru are good with a flat 'md5:hash' search. I'd bet Gelb 0.2.0 can do it. I am not sure about Gelb 0.1.1.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9bf8dc  No.12816

>>12809

Check out the last few posts from the Hydrus Scripts thread:

>>12604

>>12631

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5552df  No.12822

Does gallery downloaders check for newly added posts periodically or do I have to create a new query to fetch new posts?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12827

>>12822

The downloader page just does one job once. If you were to manually create a search for the same query you put in before, it would start a new queue starting at the current new beginning. Obviously you do not want to do this yourself every week or so, so if you want to have a downloader that periodically re-checks and keeps up with a particular query for a site, check out 'subscriptions':

https://hydrusnetwork.github.io/hydrus/help/getting_started_subscriptions.html

PROTIP: Don't go fucking nuts with this. Start slow and figure out a workflow that won't nuke your available bandwidth and hard drive space.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e38a70  No.12829

File: 3afc2de946b8f08⋯.jpg (46.53 KB, 420x377, 420:377, Scr100.jpg)

>>12641

I suppose this happened because of a disconnect, but what do these "lg"s mean?

Some files got corrupted on download? Can I check this somehow or should I do something else to remove "lg"s?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e38a70  No.12830

>>12829

Forgot to mention, It's a danbooru/gelbooru subscription.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4  No.12832

>>12814

Nice, thanks.

>>12816

I already use all of those. I want more tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a11e90  No.12833

>>12829

Ig means ignored, so 27Ig means 27 ignored URLs.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c6c972  No.12834

>>12833

Got it, thanks.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5552df  No.12835

File: c8c97e5452c9da7⋯.jpg (2.04 MB, 1288x1822, 644:911, c8c97e5452c9da7656de4548c9….jpg)

Is there a way to search only images with certain namespaces other than putting a wildcard like "character:*"? It's very slow to search with wildcards. Using only "character:" does not work.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12836

>>12834

Ignored tends to be either 404 or tag blacklisting (say, the file had 'scat', and that is in your blacklist). If you zoom into the individual sub query and order its big file import list by status, you can review the ignored reason in the 'notes' column.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12837

>>12835

Type in 'character:' in the autocomplete entry and wait a moment, and a new special tag should appear in the dropdown list called 'character:*anything*'. This is a special tag you cannot type in manually that uses cleverer db code to speed up the namespace search.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5552df  No.12839

>>12837

Nice! Any reason why "character:" doesn't default to this?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12844

>>12839

That's a good idea, thank you. I think it is probably reasonable to say that 'text:' is not a tag people will typically want to attach to anything and search for regularly, so I'll swap in the 'text:*anything*' version whenever this is added.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

21e517  No.12852

>>12803

database -> db migration said everything was in the right place, all the files existed on my system and were in the correct subfolders. Client wouldn't cite a path in the errors, not sure why. ffmpeg is also installed locally + system wide.

I fixed the issue by importing a new file; not sure what this changes but after that I didn't get any more errors. not really sure how to reproduce this, sorry.

for posterity: what i did was move a fully portable ~45gb database from mac to windows, using resilio sync (home version)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

50e20e  No.12853

Is there a way to search for files that have mutiple tags in the same namespace?

Main scenario i am thinking is searching for files that have mutiple 'series:' tags for crossover images but dont have a 'crossover' tag.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

973c0a  No.12854

i have two files that i want them to appear as a group when i select sort by collection

what do i do?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.12855

Will using socks5 with your application result in the DNS lookup also being routed through the proxy? E.g. querying safebooru will be resolved through the proxy setup in the settings?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.12856

Is there a pane with tags that I can click to add to an image? I hate having to right click>manage>file's tags>type tag name in just to add tags to images in a page. It isn't much better in the media viewer either from what I can tell. It's super inconvenient.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b459c1  No.12857

How would I go about recording a file's original creation/modified time as a tag? Maybe a script to record it in a text file? Like the ones the booru tag parser makes.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.12859

>>12855

Potentially answering my own question:

Looking into the source code on page

https://github.com/hydrusnetwork/hydrus/blob/master/include/ClientNetworkingSessions.py

line 126, it appears you merely set the proxies to be the necessary value. Presuming a sufficiently up-to-date version of Python is installed with requests[socks], just making sure to use socks5h in the manage_options->connections should do it. In short, the user does have power over this.

Consequently, one can easily run hydrus over TOR. I did notice socks5 being quicker than 5a, presumably because naming resolution can be slow. In addition, since you'd be using the proxy's DNS, you can't really cache results. It'd be interesting if you could use your cached DNS results from the host machine, but for new lookups use the proxy. However, that's a separate thing all together.

tl;dr you can easily proxy over tor with hydrus.

If you could confirm BO, I'd be grateful.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.12869

Quick question, should I be seeing saved webms as zips of images in Hydrus? Sort of like how you can unzip an epub and get a bunch of HTML pages. Or, should the saved webms be *.webms and viewable in Hydrus? This has probably been answered at some point or another, so, if so, sorry about that.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

62d3c6  No.12875

This could either go in the bugs or this thread: Does the Hentai Foundry downloader support fetching tags? I can't get it to work with subscriptions, nor with download tabs.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12881

>>12852

Hmm, I am not sure what this was. I wonder perhaps if the OS X legacy db info had slashes in some path instead of backslashes, and one part of the files manager was unhappy with that? It is possible that by looking at your current paths and ok-ing the dialog put those unusual paths through a normalisation routine that cleaned them up for Windows, but it could also be something else.

If you go to install_dir/db and check the relevant 'client - 2019-xx.log' file and scroll to the error time (I think you'll be able to ctrl+f for 'missing' or similar to find it as well) and check the exact path that was logged in the error, is there anything funny about it? Maybe like 'E:\my files/ab\abcde….jpg'? If you have the whole error traceback, that may be helpful to me, but if so make sure you either email it to me directly or redact the path and file hash if you post it publicly.

I'll keep this problem in mind. If you aren't getting it any more, even on the same file, it sounds like it fixed itself. Please let me know if you start getting it again.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12882

>>12853

Not yet. I'd like to add an option namespace component to system:num_tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12883

>>12854

Collection works by namespaces right now. If the files have a shared 'title:' tag, you can collect by 'title' namespace and that'll put them together. If they have page numbers, you can sort by page to get them in the right order.

I overall do not like how paged-content is handled in the client. Page tags are clunky to work with, and grouping many sorts of comic and mini-comic is a pain. Adding cbz support and the associated hydrus database backend to deal with file groups/pools with indices is probably the silver bullet here.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12884

>>12855

>>12859

I know very little about proxies, so I cannot comment very intelligently. I just pass whatever you put in those input fields to requests, and it handles the rest. A bit of searching lead me to this:

https://stackoverflow.com/questions/12601316/how-to-make-python-requests-work-via-socks-proxy

and this:

https://github.com/urllib3/urllib3/blob/a019c5e9e8ac461111552fdf9aa97899977cccbf/urllib3/contrib/socks.py#L141-L166

So yeah, it seems like using 'socks5h://' does DNS through the proxy, but 'socks5://' does not. I'll update the UI on that panel to suggest 'socks4a' and 'socks5h' as well.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12885

>>12856

There are a few tag 'suggestion' columns you can turn on for the manage tags dialog under options->tag suggestions. Favourites/related/recent can be useful depending on your workflow and can be double-clicked (and in the 355 I just put, you can keyboard-shortcut focus to these) rather than having to type every time.

The manage tags dialog that opens from the media viewer is actually a window not a dialog (i.e. it does not prohibit interaction with the parent window while it is open), so you don't have to close and open it again to edit other media. Also, by default, hitting page up/down on an empty tag autocomplete input should send a previous/next command to the media viewer parent, so you don't even have to change focus to tag and move on.

That said, I would like to add some quicker 'remove'-style actions to the normal taglists when looking at thumbs or the media viewer. Something like right-click->remove this tag, which could spawn a little yes/no dialog that would handle the more complicated back-end for you without having to make the whole manage tags dialog.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12886

>>12857

I have a long-time job to cache this information and make it searchable and sortable. I know another user wants this very much, so it will eventually happen.

If you do not want to wait, yeah I think a script would do the job best. Do a stat call on each file and then output to a (hash).jpg.txt file that you could reimport with 'parse neighbouring .txt files' checked. You'd have to reimport these beside your files, which might make for a pain in the ass workflow though. You might be able to wangle it by reimporting from your actual client_files directory, with the .txt files embedded beside, but that is probably asking for trouble. If your collection is relatively small, maybe something like:

system:everything

export (copy) everything to a separate folder

generate your .txts

reimport that folder, setting to delete original files.

It'll crunch that import pretty quick, since it already has all those files.

The files will get your 'creation-date:2015-02-01' whatever.

But that would also be a pain to keep up with. I'll add this eventually (another thing the new file mainteance manager will be good to retroactively fill in) and handle it all automatically, so if you can wait, I'd advise that.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12887

>>12869

Are those from danbooru? They are probably ugoiras imported from Pixiv, which danbooru offers both in native zip format and a converted webm. If you would rather the webm, please hit network->downloader definitions->manage url class links and switch the 'danbooru file page' entry to 'danbooru file page parser - get webm ugoira'.

If you want to reimport the zips you already have as webm, do a search for 'system:has url:danbooru file page' and 'filetype:zip', and then ctrl+a and right-click copy danbooru file page urls, and then queue those up in a url download page with the 'file import options' set to 'do not skip downloading because of known urls'.

Native ugoira support remains a long-term hydrus goal.

If you are getting weird zips from somewhere else, please let me know. If the zip contains a list of jpegs or pngs, it is probably an ugoira.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12888

>>12875

It only does 'creator' and 'title' tags. The first HF parser I made grabbed the artist tags, but they ultimately proved too low quality to keep. Too much #i_feel_good_today #lewdtuesday and #the #party #is #in #my #pants bullshit unfortunately.

Alternately, are you not getting the creator and title tags? I just did a test here and it seemed fine, but let me know if you aren't getting anything at all.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.12893

>>12884

Yeah, I just wanted someone else besides me to try it out. I believe it works for me, and based off of function definitions and descriptions, I'm pretty sure it's acting as expected. However, it's not like I've done some absolute test like using Wireshark to see that no DNS lookups are being done locally, but, rather, are all being done through port my proxy lives on.

>>12887

The website is ATFbooru. I created my own parser/gallary downloaders ect. because I didn't see, at a glance, an existing one that suited my needs. It works perfectly for images–including gifs–however, despite the data being in .webm format when I go to the page in my browser, the download is a .zip. As you said, the zip contains a list of of images, therefore, it's probably that thing you called an "ugoiras".

I can look into the 'danbooru file page parser - get webm ugoira' downloader and add it to my own script.

However, I do want to be certain, the webm conversion wont lose quality right? They're equivalent? I ask as a comparison between differing video compressions (e.g. Xvid yields a loss in quality when compared to H264).

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12894

>>12893

That's the tricky thing with ugoiras–they are a pretty weird format, a bunch of images rendered by javascript according to variable framerate. There is no excellent lossless format to convert them to. The sites you are getting webm conversions from may be using some clever encoding options to preserve lossless and frame timings, but they may not. Webm has a lossless mode, but this is not typical–the filesizes can get pretty gigantic. The problem is aggravated by most artists putting jpegs in their Ugoiras rather than nice pngs, which can look beautiful in an ugoira when presented right, so you are already taking the compression hit from jpeg.

Ultimately, if I can write a native hydrus ugoira detector and renderer (although we'll have to figure out how to attach the JSON frame timings, which aren't guaranteed to be in the zip), then we can just keep the original files. Or figure out the right params for lossless variable frame rate conversion using ffmpeg and do that webm/whatever job 'properly' ourselves. Whole thing is a bit of a shitshow with no ideal solution at current.

If you want to copy the danbooru parser, I think it basically detects the webm embed with a higher URL priority than the normal file download link. Then, when the page provides zip on the download link and webm embed, it goes for the embed, and when there is a jpeg or whatever in both, it won't find the video embed and falls back to the non-sample-guaranteed download link.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b459c1  No.12896

>>12886

Well it's not really that tedious….if I knew how to script.

I thought once Hydrus imported the files, it gave them a new creation date matching their import time though

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.12897

>>12894

>filesizes pretty gigantic

not too important.

>sites use clever encodes ect.

>artists putting jps in their Ugoiras

This is where my lack of knowledge of pixiv comes into play, but what is it that exists on the artist's computer that is uploaded? Does the artist upload a zip of images and then pixiv determines how to render it? If so, then the zip is the "loseless" original since that's exactly what was on the artist's PC.

As a completely separate aside it's a tad sad when you notice something, but it doesn't work out. E.g. there was an attribute field which held all the tags for the image, delimited by spaces. Consequently, if you HTML parse your way into that object and get the attribute, proceed to split (via regex option in Hydrus) you can get a list of all the attributes. However, although one will have a list of tags, they wont be counted as distinct results. I guess it's where optimality and usability are traded.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12900

>>12896

Ah shit, yeah, sorry: I had forgotten how the copystat worked. It copies modified date, but a new creation date is made. I use this, if you are further interested: https://docs.python.org/3/library/shutil.html#shutil.copy2 Some permission bit stuff is copied as well.

This was not true in early versions of the program, I think maybe four or five years ago, so files imported before then will have modified dates the same as creation dates.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2f807c  No.12901

>>12897

I don't use Pixiv, so I don't know how you make an Ugoira. The uploader sets the framerate I think. I presume they have an upload form for all the images and you add ms timings beside each file, and then they create the zip and the accompanying JSON or whatever to deliver the timings to your browser. But maybe they have a program that generates that info clientside and you upload it all together.

Yeah, I'm sorry about the inability to split strings at that last stage. When I next put a good amount of time into the downloader, untangling the rats' nest of the edit parser ui will be a top priority along with fleshing out some needed parsing functionality like 'split this one string by whitespace and return multiple strings'. iirc, the current pipeline can't deal with n strings appearing from that space atm, so I can't just add a checkbox or a 'split this by xxx'. That might be wrong though, so I'll make a note to look at it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.12902

>>12885

Thanks for the detailed response. I don't understand what you meant by this because no matter what I do, I have to switch focus from the tag input window to the media viewer to go to the next image:

>Also, by default, hitting page up/down on an empty tag autocomplete input should send a previous/next command to the media viewer parent, so you don't even have to change focus to tag and move on.

Btw, would you be averse to fleshing out the hover tag pane on the left of the media viewer with more functionality? I picture a possible workflow inside the media viewer being:

1. text input gets captured automatically by a new search box added atop the hover tag pane when pane has focus.

2. The hover tag pane shows tags already on the image when the search box is blank. When search box contains text, it's populated with autocompleted tag suggestions, and any autocomplete suggested tags that are already on the image have a highlighted gold background.

3. If there's no autocomplete tag suggestions, then the tag doesn't exist and therefore the Enter key should create and append the typed tag to the image. Idk what the performance implications of this would be, so maybe namespacing can be used to cut down the tag pool so it's less laggy?

3. up/down arrows can navigate autocompleted tag suggestions without needing to use mouse input to change focus from the pane. Enter key appends tag to image, delete key removes tag from image. Shift/ctrl click to select multiple tags perhaps?

4. Left/right arrows can navigate to the next image without needing to change focus via mouse click.

5. right click>mange>file tags for more advanced tagging

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

69bc83  No.12922

>>12796

Still doesn't work. The rule34hentai downloader still ignores Explicit rated images on rule34hentai by saying it couldn't find file or post URL to download.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e63191  No.12923

Is it possible to download a pixiv tag starting from the oldest images through the gallery > pixiv tag search function or do I need to go through page by page starting from the first/oldest page in url download instead?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e63191  No.12924

>>12923

Another bug I've discovered: if I go via the individual page links, after page 10/11 it keeps giving me the same set of images no matter which page I go to afterwards. Very odd!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

deb25c  No.12928

>>12897

To better explain, if I open up the media viewer and then hit F3 to open up a manage tags window on top of it, I start with the keyboard focus in the autocomplete tag input text box, so if I were to start typing a tag, results would load in the list below. If from there, instead of typing regular text, I instead press page up/down, the underlying media viewer will browse previous/next. Do you not get that?

If you have multiple tag domains, like 'local tags' and 'public tag repository', then pressing up/down arrow in the same situation–focus on the tag input text box, and no other content in that box–should switch between them. Do you get that at all?

I am open to allowing easier tag editing in the tag list on the left, but the simple view it presents makes it tricky. It presents the 'combined' tags from all tag services in the client, so if you add a new tag, should it go to 'local tags' or 'public tag repo' or something else? I suspect my next step there is to add simple right-click menu options for 'remove tag' and then refactor the tag processing code, which is currently all bundled up into the manage tags dialog, into a shared space, so that taglist can be clever enough to say either 'remove this tag from x yes/no?' to 'this tag is in two services, remove it from x or y?' and so on. Once I have that code refactored, I am open to playing with more quick workflow here.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

deb25c  No.12929

>>12922

Hmm, I am not sure then. If you open network->downloader definitions->manage login scripts, go into the 'shimmie login' and run a test on the domain with your credentials, it should list the results of each page hit. You should be able to copy the result body the server returned. You could try to copy and paste those pages into some new .txt files and see if there is a 'bad login' or cloudflare-related error that the login system isn't picking up? I am not sure if I wrote the shimmie login script or someone else, so I do not feel very confident talking about it. I understand it does work for some users.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

deb25c  No.12930

>>12924

>>12923

Yeah, the basic hydrus gallery downloader works on the same principle of starting at page 0/1 and walking through each 'next' subsequent page, adding results, just like you might browse. Since most boorus present images in descending upload order, you will be getting newest first. I know some users have played around with adding a booru-specific param to the url to change the sort order, but this has mixed success.

Can you say more about the same results after page 10/11 issue? Is this for pixiv, and is it in your browser or hydrus? Can you give an example URL that does this? Several boorus cap their results to 500 or whatever, even if they suggest something big like 152 pages, and after page 25 or so they'll deliver a blank page, 404, 403, 'please login to see more', or the same last page results again. Does this sound like that?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e63191  No.12932

>>12930

The pages on pixiv in the browser are fine, they continue as normal. But no matter how many times I refreshed the client, reloaded page 12 in the browser and copied it again, it gave me the previous pages' images over and over, which obviously didn't import because they're already in the db.

Here's the URL I was copying:

https://www.pixiv.net/search.php?word=%E3%81%A9%E3%81%86%E3%81%B6%E3%81%A4%E3%81%AE%E6%A3%AE&s_mode=s_tag_full&order=date&p=12

The previous URLs had exactly the same layout except the number was just lower. 11 is the last one to load properly, after that it just returns page 11 irrespective of the page number.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

69bc83  No.12933

File: 79a1eb2c07b16ec⋯.jpg (34.62 KB, 832x289, 832:289, 3464634.jpg)

>>12929

Actually forget all that, I'm an idiot. I didn't set the login to shimmie, this probably should be a default setting for rule34hentai.net if it isn't already.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

17dfbe  No.12934

File: 5d5a0908308f8ca⋯.png (9.62 KB, 367x233, 367:233, Capture.PNG)

Why do I keep getting this?

How do I stop getting this?

I've only got three pages open.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e63191  No.12935

>>12932

I'm dumb and I figured it out. It needed a login to complete more than 11 pages it seems.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f92c9c  No.12946

Just started using Hydrus, and I'm trying to get it to autotag an existing collection, but I'm a bit confused.

>Tags are rich, cpu-intensive metadata. My repository has hundreds of millions of mappings, and your client will eventually download and index them all.

Seems way overkill that the user should have to download all that information if they only need tags for the specific files that are part of their own collection. Isn't there a more efficient method?

I'm aware also of the tag archives, but it seems that's also a broad and inefficient approach containing data for the complete file set from a booru/gallery website as well. Is there no other way except to maintain a massive database of all the tag associations of all files ever seen? If downloaders could be used to query downloading tags only for the user's existing files (it may be possible, but I am ignorant), that would be very useful and eliminate the need to waste HDD space/ processing power.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b459c1  No.12947

>>12946

Only if the hash of your file exactly matches the hash used by the booru of choice. Protip: they usually don't unless you got it straight from that booru.

Use >>4475. And read the thread. It'll take some light editing to get it working due to a recent hosting issue.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0ffbbc  No.12948

>>12946

As I understand it, main reason is privacy.

Under this system, you have full access to all those tags without having to tell a tag server anything about which files you do or do not have.

And you can just sync once in a blue moon, instead of having to phone home every time you add a new file.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f92c9c  No.12950

>>12948

That's a very good point I hadn't considered.

>>12947

Having a real bitch of a time messing with scripts. I got a few from that furfag dr guy. Got one of them to work for one image that I manually selected, and then everything took a shit and the scripts just hang at the fetching stage now.

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/wiki/2.-How-to-fetch-tags-in-bulk-with-md5-hash

It seems this method would be the absolute easiest but there is no longer anyway to get the md5 file hash in the options menu, and the default file names are the same as the sha56 hash. I think I'm going to just throw in the towel and try the public tag repository.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

24e08e  No.12952

Not sure has this been asked before: How to minimize the active subscription windows? They are blocking the thumbnails and are very annoying.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.12953

I ctrl+f'ed the threads, so hopefully this hasn't already been asked.

I created a default sankaku channel tag subscription with a test query of "tifa lockhart" (without quotes), but I'm getting a network error.

Specifically, bad gateway 502 along with some html shows up at the top of the edit screen for the subscription. When I first performed the search, it asked me for a user agent string I believe which contained firefox and some other stuff. I did a similar subscription for gelbooru which worked fine. I'm using tor but can access the site just fine in my browser.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

458202  No.12961

>>12932

>>12935

Thank you for the update. Pixiv seem to change their access rules every few weeks, it is a constant battle.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

458202  No.12962

>>12933

Ah, thank you for this update. I think it is supposed to be default for new users–I think the shimmie login script has an example URL that does the link-up automatically on client first-boot initialisation–but I wonder if you got caught up in some combination of version updates that meant this did not apply? I apologise if so. The login system, as I am sure you now know, it pretty a scrappy v1.0, barely out of prototype.

I do remember other users telling me R34H had funny login rules anyway, so I recommend you keep an eye on this in case it starts giving you more trouble. A future version of this whole system will be able to detect if you get suddenly logged out for some reason and will try to recover and give human-friendly errors and so on.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

458202  No.12963

>>12934

I apologise, I fucked up the clipboard watcher! It is spamming those messages false-positively on certain kinds of URL that are not matched. It is fixed today, please let me know if you have any more trouble with it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

458202  No.12964

>>12946

>>12947

>>12948

>>12950

Yeah, privacy and low server CPU are the top reasons. I prefer that your personal CPU cycles happen on computers you control, so the tag repo is really just a clearinghouse of data. The PTR is getting huge now though, 580 million tag mappings, so after this duplicates work I am going to put some time into figuring out better solutions here. One top priority is adding a tag filter to the incoming stream, so you'll be able to say "I don't want filename: or title: tags" and so on.

This is very much on my mind. The PTR has been a great success, but it was always a crazy prototype and we are hitting some limits.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

458202  No.12965

>>12952

No excellent way yet. I think each sub has an option in its edit panel to not show a popup, but I generally advise against this as you often then don't notice it at all, even when it is having problems. This is on my mind, and I have a fairly high priority job to figure out an up/down arrow button or something so you can collapse and expand them. For now, best solution is to hit network->pause->subs and then just unpause them when you are happy for them to run.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

458202  No.12966

>>12953

502 is a server-side error. It could be that Sank was overloaded at that time and could not respond to your request, or it could be CloudFlare giving the 'we tried talking to the site but it timed out' similar message, or they may have blocked Tor. Is the browser Tor as well, or just hydrus?

If you run a test download and check the gallery log button (the little icon button on the gallery search section of the downloader panel), it should bring up a list with the first URL it tried to hit with more error info in the 'notes' column. Do right-click->copy notes and put it in a text viewer–that should be the html page that was returned, which should, if you scroll down a bit, have some more info. It might be like 'Tor is blocked m8' or something else. I'd be interested in knowing what you find, but be careful just pastebinning that html and linking it here, as it may have personal info.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a7d23b  No.12973

I just clicked here from /tech/, shithole that it is. I have no idea what this project is. Could we have a sticky?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b459c1  No.12975

>>12973

>New user? Start here -→ http://hydrusnetwork.github.io/hydrus/

lrn2read

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.12985

>>12966

Thanks, seems to be working now.

>>12966

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.12986

What do you guys think a good max allowed bandwidth would be? The default 64mb for sankaku is too little.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

24e08e  No.12992

>>12965

That'd be nice. Or make each of them a tab would be nice too.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f92c9c  No.12995

I noticed some of the tags from the PTR are just flat out wrong, such as pictures in which a girl is standing and facing the viewer so that her ass isn't even visible being tagged ass, and so on. Is there a way to deem certain public tags as less important and hide them, or a way to ignore public tags and copy only a portion of them to local tags?

I suppose I could always just add a local alias and search by the alias instead of the public tag.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

054854  No.13002

>>12986

I added this after being contacted by a top person at Sank who was concerned about their bandwidth costs, which I sympathised with. After some discussion, we compromised that I would set a restrictive bandwidth limit by default. You can of course edit your client's rules to be whatever you want. If you want to do them a favour, you might like to try to find what content you can on other sites.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

054854  No.13003

>>12995

There are not excellent ways yet. The PTR has plenty of weird and incorrect stuff, and there are so many new tags every day that it is completely beyond me to moderate effectively (I am also not a natural janitor or site-manager). I will be doing substantial work on the client-repository relationship after this current duplicates work. You will have more control over what tags you get and where they go. I expect to add multiple 'local tags' services and easier ways to migrate your tags around so you can better handle all this stuff.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5f52a4  No.13014

Anyone else having problems with the deviant art downloader, first the login script stopped working, now it can't seem to download any page content to parse correctly.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e7d1d7  No.13016

File: 1480f17fa211481⋯.png (1.48 MB, 1921x1050, 1921:1050, Capture.PNG)

File: 1e3b8fc7807581f⋯.png (5.05 KB, 512x112, 32:7, imgur-2019.03.07.png)

File: 2d6a9abd4847968⋯.png (6.03 KB, 512x109, 512:109, reddit-2018.09.21.png)

>>11887

>>11791 here, sorry for the late reply

I tried it again on v355, still happens

I'm using the AIO reddit and imgur parsers (or the imgur one that comes with the hydrus client, not sure) from the "Hydrus Presets and Scripts" repo

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.13017

I created a gallery download page and it stopped after 19 images with a message about 502 bad gateway. I figured the booru was having issues, but no, I can access it just fine. In fact, a new gallery with the same exact query is downloading right now just fine, making the message below confusing:

expires in 50 minutes: 502: <html>

<head><title>502 Bad gateway</title></head>

<body bgcolor="white">

<center><h1>502 Bad Gateway</h1></center>

<hr><center>nginx</center>

</body>

</html>

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c247f  No.13018

Is it possible to have two clients share a database? I keep my data on external drives and want to be able to work in both Windows and Linux.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4a3c97  No.13027

>>13018

I think the answer is yes, they just can't be open at the same time.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.13029

>>11122

Sankaku gives you at most 1000 results from any query, but you can circumvent that to some extent if your initial query was at most 3 tags by looking up query + order:filesize and query + order:filesize_asc.

Is there a better solution to this? Finding a custom query to get as close to 1000 images without going over and then having to do this every 1000 images is a pain. Even 2000-3000 images would be a massive improvement.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.13031

>>13029

Nm, I happened to be checking the bugs thread and noticed someone made a parser that solves this supposedly. Just out of idle curiosity, why are parsers shared as png files and not text files? Also could a parser be malicious?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5b6c03  No.13032

Databases work cross-platform, right?

I want to try out hydrus through wine because of all the linux version's issues. How do I go about making the switch? Do I simply copy and paste the linux hydrus network over the windows one like I was simply updating? Or do I restore the (linux) backup I have through windows?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

308209  No.13033

just started using this a few days ago, really liking it for sorting my giant folders of booru rips, but one thing I can't find is parent child picture relations, like danbooru has.

I want to be able to set one picture as a parent, and then another as a child picture, for alt versions and small doujins/image sets. Is there a way to do this or am i fucked?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

03b279  No.13034

>>13031

I wrote that parser. It's a png because Hydrus lets you export them as .png files for easy importing. If you're that worried just copy and paste this:

[58, "sankaku gallery page parser fixed", 2, ["sankaku gallery page parser fixed", "b29f9b48f7c6374d12e7d4cbb527f16fd3c725946dd6479c0684e5e0e1dea15f", [55, 1, [[], "example string"]], [], [26, 1, [[30, 4, ["Next page from rel link", 7, [27, 6, [[26, 1, [[62, 2, [0, "div", {}, 22, null, false, [51, 1, [3, "", null, null, "example string"]]]]]], 0, "next-page-url", [51, 1, [3, "", null, null, ""]], [55, 1, [[[1, 7]], "parsed information"]]]], 0, false, [6, 50]]], [30, 4, ["Thumbnail URLs", 7, [27, 6, [[26, 1, [[62, 2, [0, null, {"class": "content"}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]], [62, 2, [0, null, {"class": "thumb"}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]], [62, 2, [0, "a", {}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]]]], 0, "href", [51, 1, [3, "", null, null, "example string"]], [55, 1, [[], "parsed information"]]]], 0, false, [7, 50]]]]], ["https://chan.sankakucomplex.com/?next=3794306&tags=krystal"], {"url": "https://chan.sankakucomplex.com/posts/index.php?id=123456"}]]

And I suppose a parser could be designed to work in such a way it makes a shitload of network traffic to get you IP banned from a booru, honestly, mine might because sankakuchan hates people using their bandwidth yet has the most expansive collection of images imo. Also it's not too hard to make a image file into a malicious script, that used to/might still be an issue on Linux, it just requires you to execute the file which isn't really a concern with Hydrus.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6aca6e  No.13036

>>13033

You can set images as alternates, and you can click an images and find all of its alts, but not much more. It is something the dev is working on currently.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

422735  No.13037

Is there any way to print a list of tags and their occurances? It'd be really helpful to weed out the typos and stupid non-tags artists insist on using in their galleries

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8dbe79  No.13040

>>13033

This is a highly requested feature, which I suspect may win the 'next big job' poll that the dev does every few months. IIRC, the duplicate system overhaul he's doing right now is basically a precursor to being able to overhaul the 'alternates' system.

For now, I'd suggest setting those images as alternates of each other.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c31583  No.13043

I imported the latest Quicksync update files (not the full db), and it's taking forever to process now, like 1 entry per 10 minutes, then cancels out after around 15 minutes. Is there something i'm doing wrong or do I have to let this run for a year?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

afd4aa  No.13050

Is there a way to set multiple tags with one shortcut?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.13051

>>13034

But doesn't hydrus execute the code from the png?

Also the downloader still finishes at only 1k even though I've set it to stop after 2000 and changed the bandwidth restrictions. Is sankaku channel tag search the right downloader to choose?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

481666  No.13058

Is there a simple way to set up a blacklist for subscriptions?

I have a subscription setup for sankakucomplex with a few hundred artist queries in it, and while I do want Hydrus to download anything tagged with those artists, I don't want it download anything tagged with, for example, "holding_hands". I know could add "-holding_hands" to each query, but I don't want to have to do that for every single query.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b  No.13062

>>13014

It seems ok to me here. I just did a login test and download and things came through ok. Are you getting any errors, or are the URLs just not appearing when you do a gallery search? If they have any errors, can you look at the list of file import statuses (hit the little icon button on the downloader page) and right-click->copy notes on the bad results, so we can see more of what went wrong? I wonder if they turned on CloudFlare protection or something briefly.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b  No.13063

>>13016

Thank you for this follow-up. After a bit of playing around, I was able to reproduce this problem. There is a certain unusual parsing type that was kicking in here that was not sending tags to the next stage down. This is fixed for 358. Please let me know if you have any more trouble here.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b  No.13064

>>13017

If you turn on help->debug->report modes->network report mode while this works, does that give you any more information? Is there perhaps a url class that is redirecting the URLs to a different URL? I've had that sort of problem before, where I was accidentally hitting the wrong URL despite the URLs in the file list looking correct.

It could also be some CloudFlare-style protection that is blocking anything that doesn't look like a browser with a BS 502 error. Usually you get more info than that brief html document though.

If you have some example URLs, I am happy to try from my end in browser and hydrus.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b  No.13065

>>13018

>>13027

Yep, the db is platform neutral so you can have two installs point at the same db and they should be fine. Only rule is only one client install can be open on a db at a time. Normally this is impossible to violate since the client will detect if it is already open on the same computer and complain, but if you have a funky network setup this check can be skirted.

If you just want to have a db on a usb drive and move it back and forth between two machines or boots, Linux and Windows, you should be completely fine. I recommend you extract two portable installs and put the db proper in its own directory, something like this:

E:\hydrus_db (db and client_files folders go here)

E:\hydrus_linux (linux install extracted here)

E:\hydrus_windows (window install extracted here)

And then launch Windows/Linux with shortcuts with '-d="..\hydrus_db"' or '-d="../hydrus_db"' as appropriate. But you know your situation better. Here is some more help about all this:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

Let me know if you would like any more help.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b  No.13066

>>13032

I'll just point you up to my post above: >>13065

Basically there is no trouble migrating the db around–it is completely portable and will work on any install from any location, so you are good to copy or move it around to test in other locations. As long as you can see the main db dir with your hydrus client exe or script, you should be good to go and if everything is working, it'll launch.

I still recommend making backups and so on before you try anything too fun, just in case.

The help file here talks about the different components of the hydrus install:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

I think in your case you'll be wanting to move around your original install_dir/db folder. Extract hydrus to your wine folder structure and then copy that db dir straight into the install. I am happy to help if you run into trouble.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b  No.13067

>>13051

I am no cyber-hacker-3000, so I cannot promise anything iron-clad, but my serialised png system shouldn't execute any external code. The data embedded in the png is zipped up JSON, stored as grey bytes at the bottom of the png itself, which I read and load. It unzips to the same as in >>13034 's post. There's a couple of data bytes in the top-left of the png to say where the data starts and how long it is. If what is read and unzip parses as a hydrus parser object, it is good to import, otherwise it will throw an error. At no point are arbitrary lines of python code intentionally executed.

If someone has an attack vector for https://docs.python.org/3/library/zlib.html#zlib.decompress or https://docs.python.org/3/library/json.html#json.loads, they could make up a malicious png. My Windows dev machine here says the current zlib version I am on is 1.2.11, which the docs suggest is currently OK.

If you are very concerned about your privacy, I recommend you only import by copy/pasting the json text and giving it a sanity pass with your eyes just to make sure it isn't trying to api-url-redirect your gallery queries to "NSAHydrusUsers.ru.cx" or anything.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b  No.13068

>>13037

Not at the moment. Many clients have way too many unique tags to show in one place in any kind of reasonable wait time, so figuring out show-all UI has been tricky. There's a 'big job' to figure out a tag cloud control which would likely have this tech.

If you only have, say, 2,000 unique tags, you might have some luck messing around with wildcard searches in the autocomplete tag input. I have some blocks to stop simple '*' queries, but someone was telling me the other day that '*:*' works and will show basically everything you have. Be warned it may take some time for such a query to return all results and may be hyper laggy.

Otherwise doing a search for a whole bunch of random files and then sorting the results' tags by incidence is a decent way to find (1) and done stupid tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b  No.13069

>>13043

Try forcing an analyze under database->maintain->analyze->full. It will take a few minutes, but if your problem here is due to too much data at once, this should fix it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b  No.13070

>>13050

Not yet. Future shortcut system should be able to do this, but I will likely not work on that until I have caught all the old code up to what is current.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

913f5b  No.13071

>>13058

Sure. You do this through tag import options of any downloader context. You don't want to set this for every individual subscription, so I recommend you go network->downloaders->manage default tag import options and then set them either for the global default or per domain. If you haven't overridden this info in your subs, your subs will inherit from these defaults at runtime.

Under your tag import options edit panel, check the button for 'set file blacklist'. Add your scat/gore/whatever tags that you don't want, and any downloader using those tag import options that finds a page with one of those tags will skip the file download and give an 'ignored' status.

It is worth pausing your subs while you do this and testing this with a manual download page, just to make sure you have it set up how you expect.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b459c1  No.13072

What was the procedure for manually copying login cookies into the client?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.13073

>>13067

Thanks for the breakdown, and I'm not too concerned about it. It was an idle thought that arose from thinking how this png system weirdly obfuscates the downloader code, so I wondered what the security implications were.

>>13034

I looked and I didn't see any new downloaders added, so I assume this was added onto the sankaku channel tag search downloader. Is this intended for use with the gallery downloader? Maybe that's why it halts after 1k downloads.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7b9286  No.13075

>>13072

This should be explained in >>11696.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6b1ae4  No.13077

>>13072

>>13075

I have a lot going on at the moment, but in the 'to-do fairly soon' list is adding cookie access to the Client API, which will make this whole thing a lot simpler. The guy who writes the browser plugin is keen to add a button that'll just transfer login cookies to hydrus in one click.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6b1ae4  No.13078

>>13073

I don't know exactly how or what you imported in your situation here, but if you added this, you likely also need to update the 'links' to make sure it is applied to the right 'url class'. Check network->downloader definitions->manage url class links. Find the sank gallery url class and make sure it is matched up to the new parser you imported.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

fdc241  No.13083

Anyone else having issues with pixiv? I keep getting an error that "The parser found nothing in the document!" Only started happening within the past 48 hours.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9687ab  No.13084

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.13085

>>13078

I imported the png version of >>13034 via network>downloaders>import downloaders. It's supposed to allow the sankaku gallery page to exceed 1k items.

Interestingly, after I did network>downloader definitions>manage url class links and applied `sankaku gallery page parser fixed` (which I assume is from the png I imported) it consistently stops after only 24 items.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1d8061  No.13086

>>13085

Hmm. That is the correct procedure for switching over. I expect you'll want to switch back until this gets figured out.

I am afraid I do not know anything about the 'fixed' parser other than it is supposed to pull a different gallery page with some 'startswith' id or something, so I can't talk cleverly about debugging it. I assume you are logged into sank if you were originally getting 1k results, otherwise I would say 'try logging in'.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0ced6e  No.13107

Is there a way to use Hydrus with sites that make you click "Agree" or "Enter" before you can use them, e.g. giantessbooru.com?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3b3d66  No.13108

File: ef5654866847151⋯.png (22.22 KB, 358x439, 358:439, error.PNG)

Whenever i try to download tags from the repo i get this error. It's always the same keyerror. I tried looking in the DB files for they key in case it was conncected to a faulty path but i can not find it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0ced6e  No.13111

Hydrus doesn't download anything from DeviantArt, no matter if I log in or not. I understand that you're supposed to have the client download something without logging in before you can do it with an account, but how?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4  No.13114

Gelbooru added some gay default tag blacklist you have to modify to access certain tags. Any way to remove them from hydrus?

Here's an example.

https://gelbooru.com/index.php?page=post&s=list&tags=loli

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4  No.13115

>>13114

Also, when you go into the options you can enable all site content and it gives you a special key you should be able to throw in with searches to get the full results like "&api_key=anonymous&user_id=9455", but I couldn't get it to work with anything.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1db026  No.13117

Speaking of tagging systems, what are idealistic searches you wish could be usable on boorus?

>eg. "on-model"

>eg. "girly/femboys but no masculine males in the image" (possible solution would be having negating tags for like "non-femboy")

>eg. "horsecock but only on horses"

As a side note, I found an imageboard which used anime girls for an image captcha. I wonder if that could be utilized for tagging boorus, like how Google uses theirs for AI training.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1093d  No.13118

File: 65b6599d7235be7⋯.jpg (73.93 KB, 499x454, 499:454, 1445ea260a25cf25d2b5ab5709….jpg)

is there a way to exclude a tag from a 'everything' search?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0ced6e  No.13121

>>13111

Never mind, I figured it out. I needed to clear cookies first, through the login menu, then try again. Now that I have downloaded some pictures without logging in, I should be good to go.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4  No.13122

>>13107

Yeah, should be able to. In the new downloader system, that process would count as a 'login'. You can create one of these yourself, although it is not trivial to do. You also need very basic web development knowledge, such as what a cookie is.

The help for all this starts here:

https://hydrusnetwork.github.io/hydrus/help/downloader_intro.html

There is supposed to be some login help here:

https://hydrusnetwork.github.io/hydrus/help/downloader_login.html

But I just discovered that I never wrote any! If you would like to write your own login script, skim some of the downloader help to see what my approach is and then check out network->downloader definitions->manage login scripts. Your client should have 'hentai foundry guest login click-through' as one of its defaults, which basically does the same as what you want here: if it doesn't see a session cookie in the HF session, it simulates hitting the main page and clicking-through.

I think if you duplicated the HF login and then edited it to your other site you might have some luck. That particular HF script also posts an HF content form to unhide spicy content, which you can remove. I am happy to help you if you run into trouble, and if you figure something out, please post it so other users can benefit!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4  No.13123

>>13108

Thank you for this report. Please pause the repo if you have not already. Can you then go to your log (install_dir/db/client - 2019-7.log) and scroll down to this traceback and copy the last bit? (If you still have the popup, you can do the same thing with the 'copy traceback' button). I think I know what is causing that error, but I'd like to check the whole error be certain.

It looks like your files manager is/was unable to find one of its subdirectories. Have you ever moved your database around with database->migrate db? If you have, please go to your moved 'client_files' dir. If you have not, please go to the default at install_dir/db/client_files. Are there 512 directories in there? There should be 256 in the format fxx and 256 with txx (files and thumbnails, respectively), with the xx counting in hex from 00 to ff. Do you have the 'f4c' directory? I suspect you do, because if it were missing, hydrus would complain on boot. If so, it sounds like your db has somehow lost its internal store of db locations.

If you do have the f4c folder, please make sure your client is closed and then jump up to your base db dir, normally install_dir/db, and run the sqlite3 executable. Copy paste these lines into the terminal that pops up exactly:

.open client.db
SELECT COUNT( * ) FROM client_files_locations;
SELECT COUNT( * ) FROM client_files_locations WHERE prefix LIKE "f%";
SELECT COUNT( * ) FROM client_files_locations WHERE prefix LIKE "t%";
.exit

You should get 512, 256, 256, but I assume some will be less. Please let me know what you get. We can fix it manually if only f4c is missing (i.e. if you get 255), but if there are more missing I won't walk you through manually fixing this issue, since it is a pain in the ass going through 256 entries to figure out what is missing. If this is your issue, I will write a new repair routine on boot that will catch this unusual problem and try to fix it automatically. I am 95% certain I can have this in for 359.

You should not import any files to your client in the meantime. That missing directory issue will affect any file imports. The cause for this issue is also a concern. I cannot think of any way that one of those directory entries would be removed. Anyway, let me know what numbers you get, and the full traceback, and we'll get further into this.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4  No.13124

>>13111

>>13121

Thank you for this report. I have a report from another user that suggests the DA login script is broken, and I am due to give it a look this week. I am not sure what your cookie issue was, but please keep me updated if you learn anything new.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4  No.13125

>>13114

>>13115

I don't have a gelb account so I can't talk too cleverly, but I think your best bet here is to have hydrus login to gelb for you. I recommend you use a new throwaway account. Am I right in saying that gelb has a blacklist tied to your user account? I assume you can set your throwaway account's blacklist how you want, and then hydrus will inherit that when it logs in.

To add that api_key to searches in hydrus, you'd have to alter both the gelb GUG and URL Class in hydrus. It might be a bit of a pain if you aren't used to the system, so I recommend trying the login first. Let me know if that works for you. Related help at the bottom here:

https://hydrusnetwork.github.io/hydrus/help/getting_started_downloading.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0ced6e  No.13126

>>13121

>>13124

OK, so I can download stuff if I'm not logged into DeviantArt, but once I do log in (and it does say I'm logged in just fine), it doesn't download anything. The login says "valid", and I get no error messages; it just doesn't import any files.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0ced6e  No.13127

>>13122

Thanks! I'll look into it and see if I can make it work.

I actually sought help from that downloader_login page before posting, but it wasn't too helpful, as you can imagine. :p

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4  No.13128

>>13117

I don't tag on boorus, but for hydrus I am all for longer/combined tags to clarify these situations. I think tag parents in hydrus can help tag quicker too. For instance:

horse penis on horse

could have all these parents, some being grandparents:

horse penis

equine penis

animal penis

penis

animal genitalia

species:horse

species:equine

whereas 'horse penis on futanari' or '-on human' or whatever other pseudo-suffix would have a different set of nice parents. It is neat as your brain can work on this higher level of recognition and it lets you add a bunch of valid tags in one action. Same with something like "girly/femboys but no masculine males in the image" or the various "small female big male" stuff you come across sometimes. With these specific tags, you can define precisely the core of what you are interested in in the image without being awkward or overbroad in the search. Also it lets you quickly and precisely dismiss what you don't want with a single negation tag like ['horse penis on horse', '-horse penis on human' ], rather than ['horse penis on horse', '-species:human'] or ['horse penis', 'species:horse', '-species human'], which are not the same thing.

I think you could easily go crazy trying to define these forever, however, so I will fall back to one of my main tagging rules: only tag what you are personally interested in. If you want to see horse penises only in their natural God-given state, then pursue this 'horsecock but only on horses' path, and perhaps the '-on intersex' companions you might want to exclude with a negation tag. But don't try to tag everything on everything on everything or you'll go nuts.

I have spent thousands of hours creating an engine that burns trillions of cycles of CPU to precisely categorise and search different types of bepis. Didn't expect I would be here ten years ago, jej. I am certain that within ten years we will be feeding this info into AI tech of our own with some fun results. :^)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4  No.13129

>>13118

Yeah, when you start typing a tag, put a hyphen "-" before the tag, so like "-footwear". This will discount any file with that tag from the search. I think all regular tags will work this way, including special tags like "creator:*anything*". If you have some tags in the 'selection tags' area on the left, you should also be able to right-click them and say something like 'exclude this tag from the current search' and some other options.

Note that 'system:everything' is secretly implied to any search that has >0 tags, so if you just enter for '-footwear' or whatever, you don't need to add the 'everything' yourself.

Also, when you do big searches like this, try hitting the 'searching immediately' button (ctrl+i shortcut as well, I think), which will pause the search-on-new-tag and lets you queue up a bunch of tags before actually firing off your big search.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4  No.13130

>>13126

Thank you for this info. No idea what's going on here–maybe the Eternal Cloudflare has struck again, or maybe once you log in they now deliver differently 'shaped' gallery page data or something. I'll look into it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4  No.13132

File: a2ccb844c47ccc4⋯.png (15.2 KB, 776x164, 194:41, 1.png)

File: 6cbce6fcb8c5015⋯.png (50.98 KB, 670x886, 335:443, 2.png)

File: 35dcad48dc090c7⋯.png (29.74 KB, 710x296, 355:148, 3.png)

File: 3fc92ddf5f9c710⋯.png (14.49 KB, 754x452, 377:226, 4.png)

File: 084662c329cf53c⋯.png (57.45 KB, 750x244, 375:122, 5.png)

>>13125

>Am I right in saying that gelb has a blacklist tied to your user account?

Yes, but it also has a default blacklist which you need to disable in your "account". Your account being any account including temporary anonymous ones.

Anyway, I tried it before but I forgot to modify the GUG before, so I tried it again and still nothing. The URL throws the extra information in properly, but still no results. (1-3)

I also tried creating a temporary account, it logged in properly, but still no results despite the cookies looking like they're working. Throwaway account and literally billions of possibilities for the remainder of the hash, so there's no problem with the fifth pic. (4,5)

I tried restarting hydrus just in case but still nothing.

The API thing doesn't work on the browser, so I guess that's Gelbooru's fuck up, but logging in with an account with the content enable does, so that might be some fuckery with how hydrus downloads stuff I'm guessing.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3b3d66  No.13136

File: f1c3809c8ac6ea6⋯.png (7.86 KB, 628x141, 628:141, query.PNG)

File: 07121d68648065d⋯.png (68.68 KB, 817x840, 817:840, traceback.PNG)

>>13123

I have 250 'f' folders and 256 't' folders. I did migrate my db from one folder to another at one point and suspect that something got messed up in the process. I have also noted that when i run the 'file integrity' tool other folder names appear, which i presume are other missing ones.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3b3d66  No.13137

File: 7f211373e6f9651⋯.png (65.26 KB, 702x889, 702:889, traceback2.PNG)

>>13136

Another example.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a4dd9b  No.13139

>>13085

That parser was faulty, I believe. I made it work for me some time ago, should be at >>13138 – please tell me if it works.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bf766a  No.13146

To hydrus dev specifically: how do you manage all the tasks involved with a project of this scale? I find it staggering that you're able to keep on top of it all so well, presumably as one person. Apologies if there's already an answer around.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ff8fee  No.13159

I asked some time ago about how to restore old downloader functionality(e.g. "get X more new files"), and I tried simply increasing the "stop after this many files" values(search and gallery parser) and resuming the search, but Hydrus doesn't do anything with it.

Is there anything else I can do or must I switch to searching with a high value initially and pausing when I'm satisfied?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

939696  No.13168

File: 04eda2644078404⋯.png (3.97 KB, 379x126, 379:126, display all site content.png)

>>13125

I've been messing around with it a little bit. Even though I was able to log in to Gelbooru successfully in Hydrus, it didn't allow me to download any blacklisted tags, neither in a download page nor subscription. However, following the instructions from the help page you linked and exporting a cookies.txt from browser into Hydrus did allow it to work. I suppose that means the relevant info is not tied to your account but is in a cookie.

I think this is the setting in Gelbooru's options that's causing the issue.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4  No.13170

File: 00fca99fb072ea5⋯.png (16.73 KB, 312x287, 312:287, ClipboardImage.png)

>>13168

I thought I had looked through the cookies before but I'm guessing I'm retarded.

Pic related is enough to get it working.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d9f4f4  No.13171

I've been working on creating my own parser recently, and have been successful at the parser bit, but it only downloads the first page of a gallery link. How do I make it download the entire gallery?

My gallery parser correctly returns a "gallery next page url", and I have a url class of type "gallery url", yet only the first page is being downloaded.

Am I missing something stupid?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

086196  No.13180

Whoahhhh so what the fuck am I looking at here? What exactly is hydrus? Can someone please give me a run down

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4  No.13182

File: 9ffbd02c9c77292⋯.png (83.15 KB, 752x786, 376:393, ClipboardImage.png)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4  No.13183

File: c904bcea9b3e1e9⋯.png (1.34 KB, 294x29, 294:29, ClipboardImage.png)

File: cf8817ce53633ae⋯.png (64.47 KB, 1136x238, 568:119, ClipboardImage.png)

>>13171

Try setting it to no limit and download a tag you know goes past one page, then look at the gallery log. The last URL should give you a note that'll explain what's fucking up.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

68ca54  No.13184

Is it possible to have the tag autocomplete enter existing namespaces when tagging with a word that isn't a non-namespaced tag? e.g. entering "splatoon" automatically tags as "series:splatoon" since I don't have a "splatoon" tag in my tag services. I know it's possible with tag siblings but adding an additional pair for each tag is a lot of extra work.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4  No.13185

File: 29c08084f92875a⋯.png (4.43 KB, 300x166, 150:83, ClipboardImage.png)

>>13184

It already autocompletes in the searches if that's what you mean. If you're talking about tagging it as "splatoon" and having the tag translate to "series:splatoon" then yeah, you need sibblings.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a  No.13191

>>13132

>>13170

Hey, I am sorry I was late to this. I think another user (sorry if it was you somewhere else, and I am forgetting) was telling me that this cookie is not tied to the account, but the session. I still have a job to add cookie export to the Client API. This will make this operation a one-click action for anyone with the browser add-on and clear the hassle from this shit.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a  No.13192

>>13137

>>13136

Hey, I am sorry getting back to you late here as well. I believe this now auto-heals on boot in 359, so have you had luck with that?

For some reason, six entries have been dropped here. You might like to check install_dir/db/help my db is broke.txt just as some background reading. It might be worth running a db integrity check on client.db at least, just to see if there are any other dropped rows in that file.

If the new version did not fix this automatically for you, you can fix it manually by using this line:

INSERT INTO client_files_locations ( ?, ? ) VALUES ( "fXX", "client_files" );

For each dropped fXX prefix. fb9, f4c, and four others that I imagine you can figure out by booting and running some more file integrity checks or similar. If your actual location for the files is not install_dir/db/client_files (as per the "client_files" parameter there), the client will ask you to relocate to the correct place on boot.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a  No.13193

File: 3aa3b38b6d702ab⋯.jpg (139.19 KB, 900x1200, 3:4, 3aa3b38b6d702ab341f50bcac3….jpg)

>>13146

I use ToDoList by AbstractSpoon to 'keep track' of everything, and I strongly recommend this software for any project tracking (I also use it for daily schedule management), but in truth I am always drowning and overwhelmed, ha ha.

I am just one lad, and once hydrus picked up a little, my todolist queues quickly turned infinite, almost always growing faster than I can process them. You can see the current running todo count at the top of this page–1,783 items atm jej. I sometimes cut that number by a handful after a good week of small work, but it typically grows by 15 or so items a week. The ride never ends.

I have found the trick, given the unrelenting queue, is to instead bottleneck my own schedule so I don't burn myself out. Marathon not a sprint, etc… Although I love working on hydrus, I get tired pretty easily. I usually do about 30-40 hours a week on hydrus, and while I could easily 'schedule' 100, I'd probably still not catch up on my to-do for another twenty years and I'd kill my soul pretty quick and quit the project. Strict schedules have been helpful.

Although I am fairly clever, I am not conscientious. I am not normal or professional, so I have some different ways of working compared to typical devs on other projects. I don't do many widely considered good practises and the back of my mind generally recoils at them, viewing them as glow in the dark. Coming to know what I am weak at–and then not try to do that–has been difficult but important. I know that I burn out and end up in drama-bombs when I work with others, so I have a strict solo-dev policy for hydrus, and that has worked really well for me. Weekly updates is another–I am a 'do homework the night before' kind of person, so whenever I have tried monthly updates, I end up playing an MMO for 2.5 weeks and then scrambling to catch up. This schedule keeps me focused and allows for quick prototyping iteration and bugfixing turnaround. Focusing on larger jobs has been difficult, but the 'big job' poll, giving me a day or so a week to push on a longer-term thing, has been great. And having the new 4-weekly rotation of small jobs, cleanup, small jobs, ongoing polish has been good at ensuring I don't get lost in the muck of some bullshit for six weeks and forget about other important things.

The most difficult thing that I still fail at is keeping up with messages. I am not social and don't enjoy stepping out of being Anon. It is real work for me to deal with posts and DMs and emails, and it isn't something I can avoid, so I mostly catch up when I have the motivation and guilt to get it done.

It helps having an Anon-centered userbase. I love imageboards and enjoy having tech-fluent users whose hand I do not have to hold. It means I can put out scrappy tech and make autistic but functional UI rather than needing to make things nice and easy and pretty, which I am not good at, and still get great feedback on complicated topics.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a  No.13194

>>13159

Yes, unfortunately there is no way at the moment to get that old queue-building method back.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a  No.13195

>>13168

>>13170

>>13191

I guess since this is divorced from needing an account, the cookie is not actually remembered account-wide. Thank you for this information.

If this is a tick-box with a typical html POST form to save it, I should think it would be possible to both write an account-less login script and extend the existing account-credentialled gelbooru login script to also do this step, if they user wanted. The Hentai Foundry scripts do something similar here, to do the initial home page 18+ screen click-through.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a  No.13196

>>13171

>>13183

If you are testing the URL in the normal URL downloader, this page doesn't have the pipeline tech to do a gallery walk, so it will only ever parse the single gallery page you drop/paste into it. If you want to test a full gallery walk with 'next page' stuff, you will need a GUG for it and do it in the gallery downloader.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a  No.13197

>>13184

>>13185

Yeah, my first siblings system proved only half the job here. I want to make sibs and parents much more portable so users can just grab like a 500KB text file from somewhere and just pull a whole bunch of decent 'series' siblings from the PTR to their local tags in one go. Doing it as one person by hand is a huge pain, and the current system does not resolve subjective disagreements well (e.g. is "bodypart:", though technically accurate, actually 'good'?).

The next step of tag sharing will be better en masse definition curation, and making it more shareable.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cc0406  No.13206

>>13193

Thank you so much for this fascinating insight. I've been struggling with organisation a lot recently, so I thought I would ask you how you do it, as you're clearly able to continually deliver despite all the challenges you mention. That's where I'd like to be, and I have every respect for you. That software looks like just what I've been looking for (and not finding) - thank you again.

I thought that keeping up with messages might be difficult, as you have so many scattered locations to check. I expect it to be a shortcoming that imageboard communication is naturally ephemeral, so you may feel pressured to follow up with people in a timely manner. Perhaps that helps you keep on track though. I expect I've taken quite a bit of your time at any rate, for which I apologise. Certainly don't feel like you should respond to me.

I have held off on suggestions for the software in respect of the fact that the userbase is something worth preserving, as you say. Any improvements to the UI, for example, would probably lower the barrier of entry and result in an influx of lower-skilled users, causing more hassle for you. I had wondered, though, if it was possible to write hydrus in such a way that frees you up to work entirely on the backend, leaving frontends to interested users. But I'm sure it's a lot of work for not much payoff.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cd5321  No.13209

Am I required to run hydrus as an administrator? I don't recall needing to several versions ago.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a  No.13211

>>13206

Thanks m8. I appreciate your comments. I like to insist I am just a lad though, no one special. I don't want to get a big head.

It has taken years for me to figure out a good schedule and attitude to doing this work, and there is no magic trick I have found except keep working every week and recovering over and over from failure. If you want to figure out your own system, my best advice is to keep at it, and if you are young, don't expect it to come as fast as you want. Stress is part of the whole package, like any other work, and another resource to be managed.

Stoicism works for me. It is a little cliche in some imageboard circles, but if you haven't read it, I recommend Marcus Aurelius:

https://archive.org/details/meditationsofmar00marc

Now that hydrus is functional and does a whole bunch of things, my overarching plan is to move myself and the hydrus code to be more platform-focused, providing a base that users can make things for, rather than pushing towards a complete package I have to continually tinker with. The downloader overhaul last year was the first really big stab at this–moving the whole thing to be completely user-configurable so regular users can populate and maintain it, rather my having to hardcode and update all download info myself. I now want to extend the Client API, maybe play with add-ons of some sort, and 'generalise' the code to allow more user-configurable behaviour. This lets me worry less about infinite specifics and spreads the workload without my sperging out at working in a team.

I'll be doing some PTR/tag repository work after this current duplicates stuff. Main push will be not dissimilar: giving clients more control over what they pull from the PTR, and starting work on improved local tag siblings management.

Completely decoupling the UI from the db is probably too big a job to consider at the moment, but I know some users are planning pseudo-booru wrappers using the Client API. We'll see how that goes and iterate on from there.

I am all for new suggestions, btw, so please don't be afraid to say something. I just have trouble keeping up with them a lot of the time. I'm not about to add a Facebook plugin, ha ha, but thoughts on improving and cleaning up workflows is always helpful. As the guy who understands the cogs behind everything, it is often difficult to picture what is confusing or unhelpful to users.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a  No.13212

>>13209

Shouldn't do, unless your db is in a protected location like 'C:\Program Files'. The client needs to write to its db dir, which is by default in install_dir/db, so if that is in a protected or otherwise read-only location, it might not work as a regular user.

What error messages are you getting, if any, that drove you to try 'run as admin'?

Here's some general info on the db and how to move it about, if that is helpful:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cd5321  No.13216

>>13212

Ah okay, the install directory got moved recently after discovering Chocolatey and installing hydrus through it. I'll see about moving it back to the root of C:\ where it was before.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cc0406  No.13217

>>13211

Yes, that makes a lot of sense. I agree about stoicism. I happened upon your recommendation of Marcus Aurelius to another guy, and it's good stuff. Some self-discipline is certainly in order, which cannot be provided by software.

I certainly intend to give some feedback after I've had a chance to organise my thoughts. I'm sure most of it, if not all, will have been said already though so hopefully it's not too tiresome reading duplicate suggestions. Can't really see a public to-do list (or the like) around to help prevent that, though.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3b3d66  No.13220

>>13192

Updating to the latest version seems to have solved the issue. When i started the client i got the prompt that the db entries were missing, and i could then point them to the folder where my files are stored. Will do some integrity checks now to see if anything else breaks.

Thanks for the help!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1d1a8c  No.13221

File: be3abb69baee671⋯.png (330.79 KB, 594x640, 297:320, 1557349478670.png)

Is there a way to force-download the whole tag set of the PTR? I increased the max bandwidth for Hydrus services but it's saying

>service has exceeded bandwidth - next request in 5 days 17 hours

I know there was a way to do it in a previous release but I can't find it now.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1d1a8c  No.13222

>>13221

Ah, never mind. I found the option to import.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

506ec4  No.13226

File: 251535f27a89464⋯.png (948.46 KB, 791x725, 791:725, 251535f27a894644416f8e13c6….png)

I already have the PTR downloaded, is there any way I can export it and import it on a separated hydrus?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

481666  No.13238

Did the dev ever add the ability to manage multiple databases? I know there was talk of it at one point, but I haven't kept up with development in a while. I like the idea of having separate databases for SFW and NSFW or some similar scheme.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a1195b  No.13241

Why is there still a search plugin for the Mishimmie? It's dead, it was never exported and Dreamhost has no interest in restoring it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516  No.13249

>>13217

Yeah, I'm afraid my to-do is too big to reasonably convert to a public system. It has personal info in it too, from user submissions and so on, so I'd have to sanitize rather than just copy/paste. I also haven't found an online issues system that doesn't make my head explode either–for a while, I had github issues activated, but for me it just resulted in dozens of long discussions about how potential new features might behave, and I lost more and more work time to it until I couldn't keep up and abandoned it. All that shit is driven towards boosting social media engagement, which is the very thing I hate, kek.

I know some users on the discord tried to make some sort of wiki to act as a 'frequently asked issues' or something, but I don't know how it ended up.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516  No.13250

>>13220

Great, thanks for letting me know.

>>13226

Yeah: turn on help->advanced mode and then check the repo under services->review services. Hit 'export updates' to pull them out, and then on the new (also advanced mode) client hit services->import repo update files to push them in. Then do one sync with the PTR on that new machine so it knows the updated files it wants, and it should recognise it already has them and be able to start processing right away.

Be very careful doing this. It is a lot of CPU work all at once, and I didn't design the processing pipeline to deal with it. I recommend you let it run for x minutes processing on a shutdown maintenance run (which will have strict timing rules and be able to cancel itself eventually even if it runs super slow), and then restart the client and run database->maintenance->analyze->full. This will make the db do some optimisation stuff on the new populated db tables so it can do the rest of the processing at decent speed.

I overall do not recommend hitting 'process now', if any user has told you about it, on a fresh sync. Some users have tried to do a full processing run in one go from nothing, which usually locks up the client due to mis-optimisation and slow work.

Let me know how you get on!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516  No.13251

>>13238

Not yet. It'll be a pretty huge job, unfortunately. I'd like to have multiple local tag services coming in after this duplicates work as part of a significant en masse tag management overhaul.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516  No.13252

>>13241

Thank you for this report. I'll take it out of the defaults.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

dae166  No.13258

Is there a way for me to filter by filetype? I know you can sort by filetype as an option, but say if I wanted to only see .jpg files of a certain tag, is there a way for me to do that besides manually adding filetype tags to every image?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1f5fbd  No.13263

Guys, how do you set up a Derpibooru login? I do not know what kind of login script it uses. My grabber just uses the API key.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6aca6e  No.13265

>>13258

yes you can filter by filetype same way as you apply other filters to a search

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7d6a76  No.13269

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d693b0  No.13273

is there a way to know the last time hydrus was open? I went to a repair center and I'm afraid the saw all the nasty shit I have.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6aca6e  No.13279

>>13273

Maybe the log files? Check last time modified, or if there are any dates written to them.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9403ab  No.13280

>>13279

fuck, they did.

fuck fuck fuck fuck

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9403ab  No.13281

File: 25f30059e6c85c3⋯.png (165.33 KB, 1366x768, 683:384, scrot_2019-07-23-23:33:59_….png)

>>13280

>>13279

fuck I'm not going back there never again.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6aca6e  No.13289

>>13281

Holy shit, there were looking at your stuff for like 7 hours? Maybe they want a copy

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

79c1de  No.13299

File: e52f2c611cfaf72⋯.jpg (522.28 KB, 1920x1080, 16:9, cunt destroyer.jpg)

If my filenames already contain character delimited tags like "dog, corgi.jpg" how can I have Hydrus import those words individually when I select "add tags from filename", rather than creating a single "dog, corgi" tag? I assume there's a regex trick.

Also any general tips for scrapping tags from an existing image library's filenames?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3f7141  No.13300

>>13281

F

This is why we use passwords

(And not repair centers)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3607e9  No.13301

Hopefully ther's an easy way to do this.

I have two seperate instance of Hydrus with different files, tags etc.

Is there a way for me to merge them into a singular Hydrus instance with all tags intact? Could I start the server on one port and connect to it from the other instance and grab everything?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f95d3  No.13302

File: 41ce6a73d3d4559⋯.jpg (180.82 KB, 750x750, 1:1, 1509677821_1392204711558.jpg)

The menu bar on Hydrus is missing. Not the sidebar, the one on top. Is there a way to get it back? I don't remember removing/disabling it.

On a semi-related note, is there a list of built-in keyboard shortcuts?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

623fd2  No.13307

File: 8aeab596c788f21⋯.png (406.58 KB, 1500x1000, 3:2, ClipboardImage.png)

File: fd1e0ddb7dacb5d⋯.png (1 MB, 1920x1080, 16:9, ClipboardImage.png)

I wanted to make a thread, but it would probably be useless if the BO/devs don't agree to the proposal, so I think I will just post here.

Due to the recent take down of ExHentai, I'm aware some anons use Hydrus, either as a database for their own pics or not, myself included, since I use it to tag hentai doujins I store for easy access so there's a chance that they have doujins databased on hydrus of their own.

What chance there is to convince the Hydrus dev of making something out of this? So anons makes a collective database of mangas and doujins so they can be easily shared there and to some other place?

It could work like p2p, so no extreme bandwidth use in Hydrus servers, hopefully, and if it works good, it'd work like a torrent client of sorts, but to display these doujins directly. If it goes well, big circles can host their doujins themselves there.

I might be extremely wrong, though. But just want to know if there's a chance.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4  No.13312

>>13307

Hey, I am afraid I have a ton of IRL shit going on today, so I can't catch up with my messages properly until tomorrow. I will try and check in again later this evening. Please feel free to make a fresh thread. There is certainly a lot to talk about on this subject.

If Anons can get use out of hydrus as an archive tool, I am happy. As I say a bit here >>13305 , to me this episode is a reminder that you can't rely on content, particularly spicy content, remaining in a location you do not control. Hydrus was in part designed to solve this problem by being a 'local' booru.

That said, hydrus is not currently well geared towards paged content like comics. Increasing support is planned, and I am in no way opposed to adding cbz/cbr import support and improved paged browsing, but at the moment the overall hydrus workflow works best for single images like you would find on most boorus and imageboard posts. Any Anons who would like to know more, please feel free to ask other users here or check my 'getting started' help here: https://hydrusnetwork.github.io/hydrus/

In regards to IPFS or using Hydrus File Repositories or something to effect better sharing on large scale, I think this will be increasingly possible over the next few years, but hydrus is not currently there. The IPFS integration (which also lays the groundwork for other small-file p2p support) is under active development now and is still mostly for advanced users who are already familiar with how hydrus works. There isn't an easy switch to throw and have doujins flow nice and easy again. In future, I'd love to do eventually add IPFS/hash/tag cross-referencing to be able to search IPFS or other file sharing domains using tags and file identifiers harvested from other sources.

I am open to more opinions and thoughts. I am not big into the doujin scene, so I am largely ignorant of the actual mechanics and the overall size and shape of the problem to solve. I saw some posts saying Sad Panda was about 50-200TB. If there is a db out there with all the file hashes and tags and so on, it wouldn't be super difficult to convert it to hydrus tags and import it to a client. Current PTR is 620 million tags covering 31 million files, so if those are the sorts of numbers we are talking about, I think the current hydrus db can handle some jobs related to this at least.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516  No.13326

>>13265

>>13258

Yeah, try system:filetype in a regular search page. You can mix it with any other tags or whatever.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516  No.13327

>>13263

I don't use derpibooru–I assume there is a derpi login script on the github that another user made? If you have imported it ok, go network->downloaders->manage logins and find the entry and click 'edit credentials'. Put in your username/password and hydrus will try and login as you the next time you make a request on the derpibooru domain.

Most of the downloaders in hydrus download as if you were browsing. Again, I don't know if the derpi login script (and accompanying downloader you have) works on an api key, but I imagine it just uses user/pass and simple gallery/file post page requests like most of the others.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516  No.13328

>>13269

What's the problem? Did some go missing?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7b9f2c  No.13330

>>13307

ipfs is the solution, and it's being developed, but the timeline is very long and not likely to come out any time soon.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516  No.13333

>>13281

If you like, close the client, go to install_dir/db, and enter this:

.open client.db
SELECT dump_name, datetime( timestamp, 'unixepoch' ) FROM json_dumps_named WHERE dump_type=13;
.exit

That'll give the timestamps of the x most recent 'last session' and 'exit session' timestamps, just in case any are left over from that 07/23 period. It probably won't add anything more useful than that log, but you never know. It depends on your idle settings, but since tag repo processing kicked in two and half hours after boot there, my guess is they opened the client and then for a period they didn't move the mouse, probably starting about 1pm or so, then moved it again at 1:30 to cancel processing, then left it alone again. I assume your client had caught up with the PTR by the 3:41 processing job since it stops there, or maybe they were continually moving the mouse until 18:40 looking through all your stuff, maybe, and then closing the client safely.

If your session does not seem to have changed in your boots since then, the worst the guy probably did is click through your open tabs. If the files (or empty pages) are the same as before, he didn't hit f5 on anything.

In an unrelated note, hydrus runs well from an encrypted partition. Ciphershed is an easy-to-use solution:

https://www.ciphershed.org/

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516  No.13334

>>13333

Shit, should have said, 'run the sqlite3 executable' in the db dir. That'll give you terminal that you can paste those lines into.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516  No.13335

>>13299

It is generally a pain in the ass every time, since it depends on what the exact file format is, and regex is not the most pleasant tool to work with, even if it is excellent.

For commas, something like this should work:

(?<=parentfoldername\\).*?(?=,)
(?<=, ).*?(?=,)
(?<=, ).*?(?=\.)

But I am a bit rusty.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516  No.13337

>>13301

No simple merge system yet. If you only want to move files and tags, it isn't super difficult–basically, export all the files and reimport them to the new db, and hit help->advanced mode and use the advanced service-wide update button on both clients' appropriate local tags panels on review services to export and import a Hydrus Tag Archive of your tags.

If you need to move ratings or complicated (i.e. other than 'everything is archived) archived status, there aren't easy solutions. For inbox/archived, you can split it into two jobs, but ratings have no import/export yet.

Let me know how you get on. I can figure out a hacky fix to transfer the data or maybe some simple import/export finally.

Subscriptions and several other objects should be exportable and importable from their dialogs, either to clipboard or png files.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

216516  No.13338

>>13302

The menu that has file | pages | database and so on? That should not be removeable in hydrus, but perhaps your OS or Window Manager allows you to clear/hide it? What OS are you on? Can you take a screenshot of what you see?

The shortcuts system is in a mess of several different systems. Some is hardcoded. The new system is available under file->shortcuts, which you will be able to check out once we get your menubar back!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5d1b82  No.13347

is there a requirement needed for ubuntu lts minimal to get the server to run? the client appears to run fine but getting an unhandled error on the server when i close it from the command line. end goal is to move the ever growing db off my pc and onto this box that has way more storage.

when its running i'm not able to connect from another computer and the firewall is disabled ubuntu.

Unhandled Error

Traceback (most recent call last):

File "threading.py", line 914, in _bootstrap_inner

File "threading.py", line 862, in run

File "site-packages/twisted/internet/base.py", line 1267, in run

File "site-packages/twisted/internet/base.py", line 1276, in mainLoop

— <exception caught here> —

File "site-packages/twisted/internet/base.py", line 902, in runUntilCurrent

File "include/ServerController.py", line 407, in TWISTEDDoIt

StartServices()

File "include/ServerController.py", line 333, in StartServices

HydrusData.Print( 'Starting services\u2026' )

File "include/HydrusData.py", line 970, in Print

print( repr( text ) )

File "include/HydrusLogger.py", line 159, in write

self._log_file.write( message )

builtins.ValueError: I/O operation on closed file.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0c232b  No.13350

Hi to all,

There are any bad side effects by installing hydrus (and it's archive) into a share program folder like mega or dropbox?

I wanna try to do this in order to access and update my archive from multiple computers

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d2c03a  No.13351

>>13312

Feeling sad about this, also https://8ch.net/animu/res/111544.html#q112288 (not using crosslinks to prevent bumplocks)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5d1b82  No.13352

>>13347

Threw a bunch of python modules that it was missing. still not run able.

Unhandled Error

Traceback (most recent call last):

File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner

self.run()

File "/usr/lib/python3.6/threading.py", line 864, in run

self._target(*self._args, **self._kwargs)

File "/home//.local/lib/python3.6/site-packages/twisted/internet/base.py", line 1272, in run

self.mainLoop()

File "/home//.local/lib/python3.6/site-packages/twisted/internet/base.py", line 1281, in mainLoop

self.runUntilCurrent()

— <exception caught here> —

File "/home//.local/lib/python3.6/site-packages/twisted/internet/base.py", line 902, in runUntilCurrent

call.func(*call.args, **call.kw)

File "/mnt/hydrus/hydrus network/include/ServerController.py", line 407, in TWISTEDDoIt

StartServices()

File "/mnt/hydrus/hydrus network/include/ServerController.py", line 380, in StartServices

HydrusData.Print( 'Services started' )

File "/mnt/hydrus/hydrus network/include/HydrusData.py", line 970, in Print

print( repr( text ) )

File "/mnt/hydrus/hydrus network/include/HydrusLogger.py", line 159, in write

self._log_file.write( message )

builtins.ValueError: I/O operation on closed file.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

623fd2  No.13353

File: 43371e74cb75ddd⋯.jpg (29.39 KB, 500x541, 500:541, 43371e74cb75ddd1dbe054d998….jpg)

>>13312

Alright, mate. I understand that Hydrus infrastructure is not quite ready for something like what I have in mind, but as I understand, Hydrus is being developed collectively, isn't it? So what I'm going to do is make a thread to rise awareness of this, so people can have it in mind and help with the development. All you said so far sounds correct, and I have the tags saved somewhere.

Thanks for your reply.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5d1b82  No.13357

>>13352

Nevermind i'm just going to move it to an external drive

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3607e9  No.13359

>>13337

Worked like a charm, thank you! I only needed the tags and the file associations so that was very easy.>>13337

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1d1a8c  No.13360

File: 5526854a8a58447⋯.png (658.92 KB, 1656x997, 1656:997, 2019-07-26-105215_1656x997….png)

>>13338

I'm on Debian, using 2bwm and bspwm mostly. I wasn't able to find a way to restore the menubar, but I was able to export all my files with their associated tags and re-import them into the new version of Hydrus, so all's well that's ends well. When I did have the menubar missing, it looked like this. It wasn't an environment thing because other GTK programs had the menubar present and when I tried to copy

db
into the new Hydrus release the menubar disappeared there, but before I did so (and now, after exporting, deleting the db files, and re-importing) the menubar was present.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a  No.13363

>>13352

>>13347

>>13357

Thank you for this report. Another user told me that 361 is having trouble starting services correctly–they only seem to start for him when he hits ctrl+c, which then quits the program. I did change this this past week, and it was all fine on my test machines, but something is messed up with the initial event queue for certain platforms. I am going to give it another go this week. Please let me know if 362 works for you, or, if it is convenient, see if 360 works ok.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a  No.13364

>>13350

Yeah, I generally recommend not putting the actual db on a remote system. SQLite likes to have some clever hard drive locks, and these remote locations tend to offer 'fake' versions of the lock commands if they even offer them at all. It might eventually cause malformed errors, or it might just not work for you at all.

It is also good to have the actual db on a local system for latency purposes.

You can try it if you like, and I'd be interested to know how it goes, but make sure you have backups of everything, and don't try to run two separate clients from two separate computers on the same db at the same time.

I know several users who enjoy putting just their media files on a NAS or even in the cloud. Keeping the db files and thumbnails locally but putting files on remote storage tends to work ok. Here's some related help talk about your options:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a  No.13365

>>13353

I'm afraid it is just me, one lad. I have trouble working in teams, so I have a strict policy of working on my own for hydrus. I am happy to help other Anons with their various situations, but I am always drowning in work, ha ha. I can never get things done as fast or well as I'd like.

That said, the overall push of the past couple of years has been to make hydrus more 'platform' based with the completely user-configurable downloader system and the new Client API. This lets other users fix their own problems without having to wait for me to hardcode-fix the danbooru downloader or whatever. I'd like to continue this trend going forward.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d5062a  No.13366

>>13360

Thanks. That's odd. On my end I just make a MenuBar object an attach it to the main frame, and then wxPython and your Window Manager are supposed to handle the rest according to native controls. Some Window Managers put it up on the main OS frame. My best guess is 2bwm was trying to put it into a location wxPython couldn't handle and it bombed out. Increasing compatibility for different Window Managers is a constant fight, so please let me know if you discover anything new here.

>>13359

Great, thanks for letting me know.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f1c4c7  No.13370

If I were, for purely cosmetic reasons, to change the current media destination's location drive letter (it's the same drive, but now E instead of L) how would I go on about that? I can't really migrate the database as it's the same drive. Just tell Hydrus where to find it again via adding a "new" location?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

623fd2  No.13374

File: 5e4571719738bbb⋯.png (1.08 MB, 1439x852, 1439:852, ClipboardImage.png)

I have a really old version of hydrus. If I update it will I lose all my tabs?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bc2c89  No.13375

Hey,

Thank you for your hard work.

This is an amazing tool.

My question: why not upload the tag database to something like megaupload/mediafire and lets us import it that way, instead of through chunks of 64mb/day?

Is there something preventing you/anyone from doing this? Of course it wouldn't be up to date but that would eliminate a large portion of the waiting from our side /bandwidth from your side.

(I also asked this through tumblr, feel free to ignore my question there)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ad3314  No.13376

>>13375

Already been done here:

https://cuddlebear92.github.io/Quicksync/index.html#two

In Hydrus use services -> import repository update files

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f1c4c7  No.13377

Oh, got another one:

Is there a faster way to get a single file out of hydrus than right click > share > copy > image (bitmap) if I want to post it somewhere?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d25ad4  No.13379

>>13374

Ideally no. Theoretically, it should be fine™. A bunch of stuff will change, like all those thread watchers will become multiple watcher pages each with one page, but nothing should break, hopefully. There's a max page limit of about 145 or so, now, and I am not sure what yours currently is, so pare down your session if it is larger than that.

The main thing is make a backup before you do it. If it all goes wrong, you can always roll back.

I think we are talking more than 100 versions. The client won't let you update more than 50 versions at once due to bitrot (I can't guarantee the update will do anything). I recommend trying 10 versions at a time. If you are on 250, try 260 and boot it. If no errors, try 270, 280, and so on. If it breaks, try 275 or whatever, half-way between your jump. Let me know if you run into trouble.

If you are a madman, you can try a couple of 50s or whatever, but whatever you do make sure you have a backup before you try. I know one user just did like 320->360 and he said it was ok, but this is a complicated subject and every client is different.

>>13375

>>13376

Yep: https://hydrus.tumblr.com/post/186658823864/hey-there-thank-you-for-your-amazing-work-i-was

Let me know if you have any problems.

>>13377

I think you can set up a shortcut for that, but it might only work in the media viewer. Try file->shortcuts->media and add a new shortcut for 'copy_bmp'.

However, if you are uploading to 8chan or any other normal site with an upload box, just drag and drop the thumbnail to your browser. It is a lot nicer to send the real file rather than a bmp/png copy. If you are trying to DnD to discord, try turning on the discord BUGFIX option under file->options->gui.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bc2c89  No.13383

>>13376

>>13379

I see, thanks!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ab698d  No.13400

File: 22dd349b8ad0e47⋯.png (927.71 KB, 1920x1080, 16:9, ClipboardImage.png)

Trying to add the public tag repository and I don't see a place to put an access key. Any idea what I'm doing wrong?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ab698d  No.13401

File: a032848cb0838f6⋯.png (805.51 KB, 1920x1080, 16:9, ClipboardImage.png)

>>13400

Scratch that last question I guess, figured it out, but now it doesn't want to connect. Turned off my firewall momentarily to see if that was the problem, but it's not.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7a7385  No.13402

For someone with a ton of images is there some way for me to limit my inbox search to, say 1000 images so everything doesn't slow way down trying load all the images in the inbox?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

98f3ac  No.13403

File: 78984882c0fd25f⋯.png (276.43 KB, 1200x1300, 12:13, ClipboardImage.png)

File: 4a727eab6a1ad73⋯.png (273.89 KB, 1034x1320, 47:60, ClipboardImage.png)

File: 19a126812845b4f⋯.png (89.25 KB, 1040x2028, 20:39, ClipboardImage.png)

From /v/'s draw thread.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

98f3ac  No.13404

>>13403

someone had requested a mascot for the software.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6aca6e  No.13405

>>13401

>>13400

just do the option on the help menu to set it up automatically for you

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ab698d  No.13407

>>13405

I did for both before coming here, and it still wasn't connecting.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6ab37a  No.13416

File: 220f1f2f49c75ae⋯.png (9.79 KB, 468x445, 468:445, a.png)

>>13402

System:limit is what you want. Once you click on it you can specify how many files to limit it to.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

12d396  No.13419

>>13407

I figured it out, I was using an older build. New version works just as advertised. Also seems to be downloading only 6000+ tags, will that number increase?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

12d396  No.13420

>>13419

Also, I have all of my images sorted into fairly general but still useful folders, is there a way to have Hydrus keep those images assigned to those folders while uploading?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

155fb9  No.13421

I am very slowly bumbling my way through learning the program and figuring out how to use it properly, and only seven hundred or so images in did I find the automated tag lookup script in the options menu. I noticed, however, that the script does not also pull the urls from the pages of the art, as the more roundabout way I was using does. Is there any quick and easy way to make it so the urls get pulled and attached to the image along with the tags as well?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ab698d  No.13422

>>13420

Nevermind, figured that out too, turns out it's an option in file>import folders.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

43c1dd  No.13430

File: 541653bd0d0baca⋯.png (132.3 KB, 833x360, 833:360, hydruschan.png)

File: 96cc024f2cc9807⋯.png (131.72 KB, 810x810, 1:1, hydruschan2.png)

File: 660e0ae7d387a62⋯.jpg (79.33 KB, 1280x959, 1280:959, hydrus-tan.jpg)

>>13403

Hey, this is great! Here's some other ones made by users from a while ago, mostly just having a giggle. When previously asked 'What should a hydrus-tan have?', I've generally said it should have an error somewhere, or long UI hangs, ha ha ha. Making her thick works with hydrus having a bloated install and db, I like it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

43c1dd  No.13431

>>13421

That's a good idea. That tag lookup script stuff was the prototype for my later downloader engine overhaul, and the way it fetches and parses its results is not great. I'd really like to rework it to use the newer parsing system, at which point it could definitely associate URLs in a sane way. That whole system is also pending an eventual throttled 'auto-check' system that you'd be able to queue up and have work in the background in some sane way that didn't end up eating bandwidth and CPU.

Best way to associate URLs right now is to run a gallery download. A bunch of the site parsers now pull md5 or similar hash and recognise when the client already has the file (and hence don't end up downloading it again, but will get tags etc…), so they work a bit like the lookup scripts anyway. You might like to do a test on a site you like for an artist you like and see if that solves your tag&url problem quicker. Help on this if you are new to it:

https://hydrusnetwork.github.io/hydrus/help/getting_started_downloading.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

155fb9  No.13432

>>13431

Thank you for your attention! That is what I have been doing, feeding the hashes of the existing files I'm importing from my own folders through the downloader to get their tags and urls, the only problem for me is that the process for that is being excruciatingly slow for me, taking upwards of two hours to get through about 150-ish files, meanwhile the tag lookup is nearly instant.

I should assume that is purely a connection problem on my end, yes? And also thank you for all your documentation, it has been an immense help so far.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5f34d9  No.13445

Is there a way to delete unused tag info? Hydrus's two .db files sum to 19.5gb filesize, while my actual images are only 4.4gb. It seems like it's storing a ton of info about tag associations for images I don't actually have. Is there a way to tell Hydrus to go through and prune all data that doesn't match to an image on disk?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4b2510  No.13446

File: c999422e0e0a97a⋯.png (8 KB, 287x134, 287:134, m4DW4X0[1].png)

Forgive me if I've missed any explicit support for it even, but is there a workable way to parse and download images from Pillowfort blogs, akin to Tumblr support? I ask because a few artists have moved there ever since Tumblr's porn crackdown.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d49cfa  No.13447

>>13432

Ha, sorry for insanely late reply. Since you see normal tag searches go fast, I think your connection is fine. There is a built-in limit for 'gallery' URL hits in hydrus, which might be what you are running up against. Normally, I strongly recommend keeping this limit in, as it makes regular booru searches work at a sensible pace, but if you change the setting under options->downloading, under 'gallery downloader' section, the 'additional fixed time…' setting, which is likely 15 or 30 seconds, take that down to 3, and these url queries, if they are indeed waiting on that, should go through quicker.

I wouldn't recommend taking it down to 1, but you can try if you want. Gallery hits tend to be CPU heavy for servers, so hydrus tries to also be polite by not hitting them too frequently. Hash lookups may be ultra-fast serverside, but if they have set up bandwidth banning based on frequent heavy requests to the gallery script, you might get hit without meaning to.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d49cfa  No.13448

>>13445

If you are synced with the PTR, unfortunately you need to get all the PTR definitions and data as well, which is about 20-odd gigs now. As you guessed, it gets all the mappings (~700 million at this point!) the server has, so all computation happens clientside and with complete privacy. This tends to be less of an issue if you have a 1TB+ total collection, but for your situation, it is not great.

If you wish, you can delete the PTR, which will clear most of that data after a vacuum call. It will not delete master records, which I have a planned solution for as per here, which I answered earlier today:

https://hydrus.tumblr.com/post/189534434774/ive-been-using-hydrus-and-trying-out-the

>Unfortunately, there is no way to clear out ‘orphan’ master definitions for hashes and tags yet. This is becoming a bigger problem with various larger dbs, so I expect to have it in in the next couple of months. I know what I need to do, I just have to write a decent maintenance routine that won’t gobble up too much CPU. The ideal is to have it work in the background, so you won’t have to do anything–it’ll just shrink your db naturally over time.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d49cfa  No.13449

>>13446

I haven't got one in by default. The main repository for user-made download scripts is here:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/Download%20System/All-in-Ones/Single-Sites

Unfortunately, I do not see one for Pillowfort there. I just checked out the site and saw it is in beta and needs a registration fee. Let me know if I missed a way to view as a guest. If you have a login, do you know if they have an API to talk to? Is their browsing experience similar to tumblr? I do not make downloaders myself any more, but if you could provide some example html/json/xml data for the Pillowfort equivalent of a booru's gallery tag/username search and post pages, perhaps the guys who do make downloaders now can figure something out.

If this is a bit technical for you, feel free to get into the discord and see if one of them will walk you through it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291  No.13450

So im getting disk I/O errors because my hdd is presumably dying on me and I neglected to make any backups of my database. I move the entirety of the db over to a new drive successfully except for client.mappings, I get errors whenever I try to move or copy it. Am I fucked?

Also the captcha on this chan is fucking cancer holy shit

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291  No.13451

>>13450

Also I have tried cloning it in sqlite, seems like its getting stuck at current_mappings_10.

The program also still works fine, no tags seem affected though I dont really know what client.mappings handles. The only reason I noticed something was wrong was because I suddenly couldn't process repository updates.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

df8eda  No.13455

My yande.re tag parser has disappeared for some reason. Can anyone post an import file for me?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d567bf  No.13459

Is there a way to re-import files from my collection at mass in order to change where their tags are stored? I made the mistake of saving all import tags to just the public repository, now im noticing now that a huge amount of images are getting tagged completely incorrectly by it so now everything is jumbled. Therefore I just want to make a new local tag system for specifically the imported tags but re-importing everything manually would take weeks.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d8ca57  No.13463

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d8ca57  No.13464

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d8ca57  No.13465

>>13459

I am not totally sure about your problem here. When you say to re-import the files to get their tags again, did you add tags to your files when you imported them? Was that from regex filename parsing, or neighbouring .txt files, or something else? Or were the files downloaded, and these tags are parsed from boorus? If you were to do this the slow way, what would you be doing?

services->tag migration may be of use here, but if things are too jumbled, maybe not.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d8ca57  No.13466

>>13464

>>13463

>>13465

d8ca57 is me, fuck this multiple imageboards with different BO trip systems, ha ha ha.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291  No.13467

>>13465

Parsed from boorus mostly. The slow way would be to manually download the images again using the urls that are saved with them, one at a time.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

980238  No.13471

>>13464

Thanks! This works when I do a test parse, but when it comes to actually importing a new image from yande.re it's not pulling any tags for some reason. I double checked that the url class link was correct, but still no luck.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

57a349  No.13475

anyone recommend a good external viewer for mac? don't really know whats good

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5fab5b  No.13479

How could I go about pulling "date taken" data from my pictures and get that converted to a tag?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

74f865  No.13480

I'm going to merge two different databases together with one being on an old hydrus version.

Is just exporting all the files them importing them with the tags enough or is there a better way to go about it?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5fab5b  No.13481

>>13480

that's probably the most straightforward way of doing it

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cc9b37  No.13483

>>13471

I got it. It turns out that my default tag import options had been reset at some point.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

74f865  No.13484

File: fe3e16eb82ae240⋯.png (5.63 MB, 2520x3564, 70:99, ClipboardImage.png)

Here's a bunch of questions since I just got a SSD and cleaned my HDD.

First I was thinking of migrating everything(hydrus, thumbnails, DB files) but the actual files to the SSD, but I'd rather not bloat it straight away since it's a really small SSD. Is there any way to check the size of everything before moving?

And about merging DBs, I remember reading something about having different sets of tags in the same DB, is that a thing I can do and I'm just stupid or has that not been implemented yet? Can I hide a set of tags by default? And is there a way to apply a certain tag to every file currently in the DB other than manually?

Thanks in advance.

8ch is shitting the bed as usual, pls don't doublepost

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5e009a  No.13485

I wonder if it would be difficult to create a thread image ripper like the ones for 4/8ch for other *chans running tinyboard + vichan or whatever 7chan is running? Could one just edit the 4chan one or is it a more involved process?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1d7680  No.13486

Anyone have a GUG for 4plebs? It seems similar to an imageboard setup but watcher, URL, and simple downloader all fail, yet the full images are there when I click manually and are just stored on its own CDN like on 4chan.

Or is there some workaround?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

679f38  No.13488

File: 37cca738f805a78⋯.jpg (81.32 KB, 1280x720, 16:9, 36475866324.jpg)

Is there a way for:

-automatically tagging like "page:1-page:n:" for downloading from booru images, not only imported from local files

-transferring part of the files to another hdd (for example large films)

-handling external subtitle for video

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9f76e1  No.13489

>>13467

Ah, yeah. Hydrus does not remember the origin of every tag, so you can't do a local-only retroactive mining of tags based on tag source. Best way is to set up a new url downloader page and feed the urls in again (doing a right-click on a big ctrl+a of the files you want should let you copy all urls that match a certain site, and then you can paste back into the url downloader). In order to get the tags again (normally hydrus will just skip urls it has seen before), you'll want to set the tag import options to 'get tags even if urls/hash already known'.

Do a test of a dozen files first here, to make sure you have it all set up how you want, and then do a bigger batch.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9f76e1  No.13490

>>13479

Hydrus does not read EXIF data at the moment. I plan to add this sometime and add search predicates and tag conversion for it.

I do not know when I will add this, so if you want to do it now, you will have to use an external program to pull it before import and then convert the tags you want to a newline-separated 'sidecar' 'filename.txt' file to import beside the file. I know some users who have done this, although I am afraid I have forgotten the name of the EXIF extracting software they used. I think they had a system where the software converted to xml files, and they used a script to convert that to newline-separated as hydrus understands for import. If you are not very technical, it may be a pain in the ass, but I am happy to help if you want to give it a go.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9f76e1  No.13491

>>13480

>>13481

Yeah. You might want to do it in two batches: one for inbox files, one for archive, so you can quickly regenerate that information in the destination. If both clients have services->tag migration, that's the best way to import/export tags atm, using Hydrus Tag Archive files.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9f76e1  No.13492

>>13484

database->migrate database has good estimates of db size, ok estimates of media size, and ok estimates of total thumb size. If the SSD is very tight, you might not want to move the db files–they need a drive with free space to work well. I recommend 1-2 times as much free space as the size of the largest of your four db files. This is no big deal if you don't sync with the PTR, but if you do, your client.mappings.db is probably 18GB or so.

You can now create multiple local 'my tags' services. Hit services->manage services to add more. The new services will appear in any place you can add/edit tags as separate panels or tabs.

services->manage tag display is a new-ish panel that lets you hide tags for single services or all services.

There is no way atm to apply a tag to all files in a file domain. I recommend you set up a page with:

system:limit=1024

-the_tag

And then do several rounds of f5-ctrl+a-f3-ctrl+v-enter-enter. You can push the system:limit a bit higher to do it in larger batches, but you'll take an inefficient CPU hit dealing with too many at once. If you have 100k+ files, this may be too much work.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9f76e1  No.13493

>>13485

I keep meaning to knock up some quick parsers and url classes for the bunkers, and then I do not find time for it. It would not be super difficult for anyone familiar with the download system. Most imageboards offer good APIs or nice html, and as you say, if you have a parser for one vichan board, you probably have support for 95+% of them running the same software.

8kun is difficult because of the DDoS stuff.

>>13486

I don't know anything about it, I am afraid. How do they fail, is it a 503? If they have aggressive CloudFlare protection, this is a common cause of hydrus not being about to get files when a browser can.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9f76e1  No.13494

>>13488

- Not yet. I would like to add this some time.

- Try database->migrate database to set up multiple storage locations. It allocates files across multiple locations randomly according to set location 'weights', so if you just have a couple of multi-GB files, you can't guarantee it will be smart about putting them in one place, but as your collection increases in size, it will be more accurate.

- No support for this, unfortunately. I am not sure there will be a neat solution in the near future. You are free to do what you want, but I recommend you not use hydrus for movies. I designed it for imageboard-style files, where you need to keep track of hundreds of thousands of smaller files, so most of the workflows and maintenance timers are built around that. Files that have sidecars just don't work well with the hydrus storage system.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

04beb1  No.13496

is it possible to delete the deleted files record for specific files, rather than the entire database? I know I can do the latter via `services > review services > local > files > all local files`, but I want to keep the whole deleted files history, except for maybe ~120 files worth.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

1dd1e8  No.13497

Alright I think I'm an idiot. I'm trying to download the images from an artist on pixiv, and despite having other pixiv downloads that work this one just doesn't and I can't figure out why.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

37bed4  No.13499

File: 1887535f707ca9b⋯.jpg (108.84 KB, 838x1064, 419:532, __shark_girl_(yukimoto_shu….jpg)

I want to use Hydrus to manage an existing collection of pictures downloaded from boorus, and nothing else. If I only want to get tags from those boorus instead of importing the entire PTR, is this https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/wiki/2.-How-to-fetch-tags-in-bulk-with-md5-hash the best way to go about it?

There was also something about Hydrus tag archives in the manual, but the archives linked there are several years out of date.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d9a9ce  No.13504

I would really like to use hydrus, but I feel I'm losing the browseability of folders. How do you "browse" certain core tags easily?

Like, right now, one folder I browse frequently is based on specific characters where the directory structure is: "my-images/series-name/character-name/". I've imported a bunch of these folders and given them tags, but now I miss the ability to see, at a glance, which series I might want to browse.

I figured out that in hydrus I could mark certain tags as "favorites", which is a start. With my old folders I could quickly navigate the hierarchy using the enter+backspace keys to go up/down and arrow keys for moving around, and I can't do that with hydrus tags. It's also clear that the list of favorites would be unwieldly at some point.

Let's forget the hierarchy for a moment. Each character is a nice conceptual "album" and it'd be great if I could just browse my "character" tag and easily switch from one character to another. I can double click on a favorite tag to bring up that tag search, but if I double click another tag it does an "AND" search…I first have to navigate to another part of the window to remove the first character's tag.

So I'm stuck. I really want to move to tags though so I can easily browse by artist or other tags which cuts across my "/series-name/character-name/" folder structure. But I want those tags to provide the same browseability had I used a "/artist/character-name/" folder structure.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291  No.13507

>>13504

I personally have a bunch of tabs set up with tags to organize some my db without having to search every time. For example, I have a blanket tab for all 2d art which then has other tabs in it that are tagged for whatever else, including an Artist tab that then has a bunch of tabs that are organized by artist. It wouldnt be hard to do the same for what you want, though it is maybe a bit less convenient than a simple folder structure though you gain way more than enough to make up for it.

Speaking of, a feature i'd really like to see is the ability to add a "master tag" to a tab so that that tag is required for all searches in that tab and all others that are in that branch. Would make it a bit less tedious to organize that way.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291  No.13510

>>13507

And though im not sure its within the scope of the program, actual folder-like functionality would be great to see. The way I would imagine it would essentially just be creating tabs and giving them a presence in the actual gallery screen instead of just the tab bar, while being able to navigate through them like an actual file explorer.

Organizing everything through tabs is kind of hard on the eyes and awkward to use if you have a bunch of them, so this would be a godsend. I'd probably lose some time going full autist and setting up my own folder structure to organize absolutely everything, though.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d9a9ce  No.13511

>>13510

The way I would imagine it is that you can specify a folder structure in a separate input box, below the input box for the search string. So you might say:

folders: <namespace1> / <namespace2> / <namespace3>

which would automatically populate a virtual folder structure with the results of your search. Items that were returned by the search but don't have a tag for namespace1 would be put inside a 'misc' folder at the bottom. (and recursively, there would be such 'misc' folders inside each level of the hierarchy)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

74f865  No.13531

File: d0ade66e08166da⋯.png (8.19 KB, 704x91, 704:91, a.png)

Is there any way to make the duplicate filter value pngs over jaypegs? I tried to set it to negative but it only goes as far as 0.

Also, since when was this implemented? I just found it now when searching for backup options.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a3189e  No.13546

I accidentally deleted a tag, my intention being to replace it. Using the show deleted option in the tags menu, I can still see the tag, but it isn't applied to the files it once was. Is there any way to re-apply a deleted tag to files?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8adcb7  No.13552

new to hydrus, is there a way to change a namespace's color? I added two new namespaces recently and they both were the color of one I already had.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

84886b  No.13553

I am unsure on whether to post this here or in the bugs thread, but after updating the program to the latest release, I noticed that I no longer can press Ctrl+C to easily copy the file's path. However, the hardcoded shortcut section still references that as the shortcut to copy files into the clipboard. Is this change by design, or is it truly a bug?

>>13552

If I'm not mistaken, you can do that by going through File > Options > Tag Presentation

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0cde32  No.13555

Where is the place with the most current info on getting downloaders to work? I am searching but I keep finding outdated info.

I am not good with computer. I just want to download every single picture of my waifu on the entire internet.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2900fc  No.13561

Hey lads, apologies again for lateness. New schedule calls for me to check this four times a week, let's see if I can stick to it this time.

>>13496

Under options->files and trash, turn on the advanced file deletion dialog. It provides an option to delete a file without leaving the deletion record.

If the files are already deleted, there isn't a way to do this I don't think. Maybe you can do something clever like searching for the files in 'all known files' domain and then re-deleting them. You know what, I'll make a job to add a 'clear deletion record' thing to the right-click menu for these files.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2900fc  No.13562

>>13497

As I understand it, Pixiv now prohibit nsfw results from non-logged-in searches, and the hydrus pixiv login script silently broke some weeks ago, I think due to them adding captcha. Please use Hydrus Companion to log into pixiv now:

https://gitgud.io/prkc/hydrus-companion

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2900fc  No.13563

>>13499

Yeah, that md5 lookup process is decent. The other solution is the Public Tag Repository, which I specifically designed to solve the problem, but if you are new to hydrus, you might want to hold off on that for a bit. The PTR is now a sledgehammer, and if you only ever want to tap out tags for a couple thousand files, it might not be right for you.

>>13531

I folded that into a changelog a little while ago, I think. I basically just exposed the internal values I was using for the score to users.

I will make it so you can value pngs over jpegs. Out of interest, can you talk a bit about your situation where that is useful? Is this for, say, a set of patreon rewards, like 'raw' ultra-high-quality png versions? In general, most users I have talked to about this tend to have many more situations where the png is some clipboard.png imageboard copypaste post of the original jpeg, so they generally prefer jpeg to be the higher weight be default.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2900fc  No.13564

>>13546

Some deleted tag storage and workflow is borked, so let me know if this doesn't work:

Open up the file(s) in manage tags dialog, set to 'show deleted', and then double-click the 'deleted' tags that have '(X)' after them. The double-click should apply a re-add action.

I think you can also just try to add the tag as normal, and the delete will be overwritten.

I am assuming this is for 'my tags' tags, and not something like the PTR. Undeleting tags for a tag repository is more complicated and really needs a janitor.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2900fc  No.13565

>>13553

Ah, thank you for this report. I am pretty sure that is old–I think all the copy stuff is now under file->shortcuts->media, with action names 'copy_(bmp/file/path/sha256_hash)'. I'll clear out that old text.

>>13555

Here's the main repository:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/Download%20System/All-in-Ones/Single-Sites

Open up Lain in network->downloaders->import downloaders and drag those pngs on her. It isn't always perfect when there are object conflicts, but it generally will import everything automatically.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291  No.13569

So I have a couple questions here, first I was wondering if its possible to automatically add a specific tag to every imported file that DOESN'T have another specific tag thats parsed from wherever im importing the file. For example, I'm trying to categorize my collection by 2d and 3d files, 3d is usually tagged on boorus but 2d usually isnt so its kind of a pain doing it manually.

Second, how do I completely delete the ptr so that I can re-download it? Im fairly sure my ptr is corrupted because a lot of tags are mixed around where they shouldn't be. I've tried deleting it from the manage services menu and making a new one but that just processed the repository updates that ive already download again and doesn't fix the jumbled up tags. Dunno if re-downloading it would even fix the problem but I figured i'd try.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

389651  No.13570

Is there a way to clear all previous deletions without reason?

I've done some fuck ups in the past and now that there's deletions explanations it's a good time to unfuck my hydrus.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

21bc67  No.13571

>>13569

I don't think you can do this 2d/3d tag logic atm. My best suggestion for now is to have a search page set up with:

-2d

-3d

(possibly system:known url=somebooru.org)

i.e. 'all the files that are from site x that don't have 2d or 3d)

And then once a week or so go:

F5

Ctrl+A

F3

Type '2d', hit enter

Ctrl+R

On it, to do this manual job in a single batch.

For the PTR, I am generally confident that your PTR is not corrupt, but the tags were applied to the file incorrectly due to human error or a mis-parse. Sometimes, for instance, a website changes its layout and a bunch of 'favourite' or 'users also liked these tags' get sweeped into a bunch of files, which slowly gets corrected on the hydrus side from user petitions and janitor approval. Simple mistakes are fairly common, and the chance of a PTR file being corrupt and that not being noticed by the client during processing is infinitesimal, so my bet is on the former. The client knows exactly what the update files should be, and if there is a single byte difference from what the server has, it throws all sorts of errors.

If you notice completely the wrong tags, with no correct tags, on many many files, particularly if it is the same kind of swap (like everywhere you expect to see 'series:touhou', you actually see 'character:samus aran'), it is rarely possible that your db has switched up some master tag definitions, possibly due to a hard drive fault. Can you provide an example hash of a file that has bad tags, and what tags you see on it? Right-click it and hit share->copy->hash->sha256. I'll check it my end and let you know if I see radically different tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

21bc67  No.13572

>>13570

I do not think so. If you are feeling brave, and you are not connected to any file repositories, and you absolutely have a good backup in case this goes wrong, try running the sqlite3 executable in your install_dir/db directory and copy/pasting each line here in turn:

.open client.db
delete from deleted_files where hash_id not in ( select hash_id from local_file_deletion_reasons );
.exit

Then boot the client. Do a sanity check, a test import on a previously deleted file or something, just to make sure everything seems good. Hang on to the client.db backup just in case this screwed something up and we need to reimport its old data.

Oh, and run help->debug->data->clear db service info cache, which will force your client to recount deleted file numbers next time you look at them.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

74f865  No.13576

File: 330029baa87ad33⋯.png (433.97 KB, 707x900, 707:900, 330029baa87ad33816894e99f3….png)

>>13563

>I have talked to about this tend to have many more situations where the png is some clipboard.png imageboard copypaste post of the original jpeg

I scrap boorus and yiff more than image boards, so pngs are usually actual pngs instead of Trojan jaypegs. I like pngs better since they're usually cleaner even when scaled down, I only opt for jpgs instead if the resolution difference is big enough to make it worth it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

100de7  No.13597

Do parent tags do anything if you have "auto-add entered tags' parents on add pend action" disabled? I disabled it because I don't like how it will add them if you enter the child tag, but it won't remove them if you undo it before clicking apply (for example, if you entered the child tag by mistake) and that's caused me to apply non-applicable tags to files on a few occasions without me noticing. I figured that turning the option off would instead add the parent tags once you actually click apply, but instead, it seems to just ignore parent/child relationships entirely now.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

be64c5  No.13598

>>13576

Thanks, this is interesting.

>>13597

Yeah, at the moment it only applies on add/pend actions, which means 'add a tag to my tags' or 'pend a tag to a tag repository'. 'delete a tag from my tags' or 'petition a tag from a tag repo' actions, which remove tags, do not apply parent associations at any time.

The reason for this is hydrus generally doesn't know why a tag was added. So if you had the situation of:

add character:asuka

(series:evangelion added as a parent)

add character:rei

(series:evangelion would be added, but it is already there, so no action)

'oh, actually rei isn't in there, I'll remove it'

remove character:rei

series:evangelion is not touched

You don't want to remove 'series:evangelion' because it could be there for legit reasons, and the code isn't clever enough to work through the human reasons for it being there.

Potentially, I could improve this logic by, say, only removing a parent if there are no valid children, or adding a history to manage tags dialog so it remembers that a tag was added by parent implication and not direct human action, but this is a complication on already quite complicated behind-the-scenes logic. I leave it to the human for now and will revisit it when the siblings/parents system is more intelligent.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

de39ab  No.13623

Is there any way to export .jpgs files with the tags embedded in them so you could view said tags with windows explorer or android or other programs? If anyone would know a way to add .jpg tags from .txt files that would also be grand.

Is there a way to disable certain tags/namespaces from being written to the .txt on export?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4455e0  No.13626

File: 2bfe1c61cf7f184⋯.png (19.77 KB, 1615x116, 1615:116, ClipboardImage.png)

I'm trying to import https://www.deviantart.com/karbo/art/A-snack-in-a-trap-828646610 via hydrus companion but it's not working, any idea why? All other deviantart posts import just fine

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

afca12  No.13627

Just added Hydrus Companion add-on to my Firefox, and I can't seem to figure out how to make the add-on extract tags from, say, Pixiv when sending the page to Hydrus. From what I'm seeing in the settings and advanced features it requires putting in your own regex or something, and I have no idea where I'd begin with that. Is there an easy way to just allow me to directly pull tags from pixiv when I send a page to Hydrus?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5b632f  No.13629

File: 65d82677eabcb7c⋯.png (51.08 KB, 852x420, 71:35, ClipboardImage.png)

File: 7dca5ede429008c⋯.png (28.62 KB, 918x227, 918:227, ClipboardImage.png)

My Windows updated and I can't run hydrus anymore, what could it be? This appears on the logs. The second one is the error if I try running client.py through cmd

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9d7547  No.13636

>>13623

I know there are programs that can do auto population and scraping of NTFS metadata, but I do not have any personal experience so I do not know any application names. Other lads who have done it needed to write scripts to convert .txt to/from xml files.

Yes, unfortunately you cannot filter the tags exported to .txt yet. It is on my list.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9d7547  No.13637

>>13626

I am afraid it works for me, which is actually a surprise because I understood the hydrus DA parser/login stuff was having trouble recently, particularly with nsfw–can you say more about the error? What happens when you try to import, does it give 'error' or 'ignored'? If you check the import list on the downloader, does it have an error in the 'note' column? Can you right-click->copy and paste that here?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9d7547  No.13638

>>13629

Did you upgrade from a very old version? I forget which one it was, maybe around 330, I jumped from 32 bit to 64 bit, and those errors look a bit like that. The instruction back then was to do a 'clean install', which sounds simple but you absolutely have to do it correct:

1) Make a backup.

2) Delete everything in your install directory except the 'db' directory and its contents.

3) Install as normal.

This clears out the old 32-bit dlls from your install directory while leaving your database intact.

If you are updating from a very old version, more than 50 versions at once, you may have trouble even if you can get it to boot. If you know what version you were before, I recommend only updating about ten versions at a time. So if you were 325 before, try doing 335 and then booting the client. If that works, then try 345. And make sure you have a backup, just in case something goes wrong.

Let me know how you get on, I am happy to help.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9d7547  No.13639

>>13629

Shit, I just read your post again–you updated Windows, not Hydrus. Sorry, can you explain a bit more about which version of Windows this was? I am not sure what is going on here at all. A clean install may help nonetheless.

Have you ever set up hydrus to run in compatibility mode anywhere? Was it set that way maybe by the Windows update?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5b632f  No.13641

>>13639

I updated an older version of windows 10 to a newer one, so I don't really know, it doesn't seem to be running in compatibility mode. I tried doing a clean install and the extract-only zip but both give me the same problem.

I have python installed and tried uninstalling it from my computer but that didn't really seem to fix it. All of the other software I had on PC works fine after the upgrade, the only one that gave me problems was hydrus.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4455e0  No.13644

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4455e0  No.13645

File: e2fde8865cde5e7⋯.png (22.41 KB, 812x242, 406:121, ClipboardImage.png)

>>13644

>>13637

oh right, actually all NSFW posts on deviantart get ignored like this, I have cookies though

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

5b632f  No.13655

>>13629

Okay, so I managed to fix it. I had to install the windows media feature pack ( https://www.microsoft.com/en-us/software-download/mediafeaturepack ) because my new windows is this N version. Just leaving this here in case someone else has the same problem.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b22eff  No.13674

>>13645

Ah shit, I just logged into DA in my hydev browser and got a 'we just updated our layout, here's a video to explain the changes' thing. I wonder if they have done what pixiv did a while ago, which is to roll their new layout out to usernames at different times, perhaps as part of a kind of A/B test. If the new layout breaks the hydrus DA parser, then it will start failing for users at different times as they get switched over.

I am not sure if this is true though, since it only seems to be hitting you for nsfw, and I am still ok, I just tested it again.

I am not a DA user IRL. I recommend you let these downloaders/subs get their 'ignored' status, to keep them in memory. Once we have this figured out, all downloaders should have a button or menu option somewhere to 'retry ignored', at which point you will be able to retroactively get the missing files here. It may also just be some weird cookie sync thing that'll fix itself in a couple weeks. Please let me know if you discover more about this, and I will keep an eye for more user reports to see if this is a more widespread issue.

If I didn't say, have you tried clearing your hydrus cookies for DA under network->data->review session cookies and then resyncing with Hydrus Companion? Maybe there is an old 'hasn't clicked nsfw age-gate' cookie still in your hydrus that is tripping things up.

>>13655

Thank you for this follow-up. I had never heard of 'N' edition before. I will write this down for future lads who run into the same thing.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3e57fb  No.13684

>>13674

might be something with my firefox, looks like all cookies, not just from DA stopped being imported correctly (they're empty) but hydrus companion dev tried with fresh profile and it worked so something on my end probably

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3e57fb  No.13685

>>13684

nevermind, managed to import cookies correctly but still get the ignored status

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

21c58e  No.13690

>>12641

is there a way to query deleted files from the client API?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f7ca0f  No.13693

File: 84e6b23e39eed9e⋯.png (28.76 KB, 440x339, 440:339, Screenshot_20200222_001807.png)

Hi, I have a few neewbie questions:

1. Just installed Hydrus, downloaded and imported 2 GiB of updates. Processing is almost finished by now, pic related shows the directory sizes in …/conf/Hydrus/. So, my virgin Hydrus with no media imported consumes 30+ GiB of disk space. Is it really that big from the beginning? Should I maybe perform some cleaning?

2. …/conf/Hydrus/client_files contains 4 GiB of files with the names like "da2e0e38098fc73502574fbba8832ce7a6f6a88c475ff33e53c152246b0e2afa". What's that?

3. Archive with the repository updates is called 1PTR_UPDATE_FILES-20200216T131032Z-001.zip, so apparently it's from February, 16. I expected that it should cover all the data I need to download, but after the import was done services→review services→repository sync→downloaded was about 70%. So I still had to spent few days waiting while the rest will be downloaded from the repository in 512MiB chunks. Is that a normal situation? Did I do something wrong?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

000000  No.13695

Some faggot has spammed the "female" tag on thousands of unrelated images to shit up the PTR, any way to deal with him besides manually petitioning tags?

At this point I'm pretty sure it's intentional sabotage and not just a broken parser script.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8d0ee0  No.13701

>>13690

Only what is available here:

https://hydrusnetwork.github.io/hydrus/help/client_api.html

I don't think you can do deleted files yet, and now I look, I don't think the file metadata call does file service status (like 'is in my files') yet. I expect this to be added as I work on the api.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8d0ee0  No.13702

>>13693

1) To keep the client fast, network resources low, and to improve user privacy, the Public Tag Repository shares all tags to all users. Essentially your client is mirroring all its data so all work happens clientside, on your CPU. The PTR has been running 8 years or so and now has about 750 million tag-file pairs, which is about 28GB. It will grow a little bit every day.

Adding file import data to the client does not add much space. My IRL 1.7 million file client is 30.7GB total.

Future improvements of this system will allow you to better filter what tags you get from the PTR, I estimate reducing total db size for users who want fewer tags about 67-90%.

2) The extensionless files are the update files your client downloaded from the PTR. They are kept in case the data needs to be reprocessed. If you are interested, they are zipped json, mostly very long lists of numbers.

3) Is this from QuickSync? I do not know enough about the QuickSync archives to talk cleverly about them. It may be missing some. I do not think you did anything wrong. The client will figure out what it needs and fix you up automatically by downloading anything missing and doing more processing when ready.

As you did a very large processing sync all in one go, I strongly recommend you do database->maintain->analyze->full. It will take a bit of time but will ensure that your db is optimised and running fast when you start importing files. Normally, this optimisation happens naturally over time as a client processes the PTR over weeks, but the QuickSync archive tends to happen a bit faster than it kicks in, and now you have hundreds of millions of rows here, that can accidentally lead to extremely slow searches for a couple weeks unless you run that maintenance routine.

Let me know if you have any more trouble.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8d0ee0  No.13703

>>13695

The main janitorial team for the PTR are on discord, if you would like to talk to them directly:

https://discord.gg/3H8UTpb

Best solution now is to petition the tags as you see them. If do not want to go to discord, I recommend you set a 'reason' for the petition with the basic problem here, and the janitorial team will get on it if they do not already know.

There are not excellent janitorial tools for hydrus servers yet. As the PTR grows and more users use it, I expect to improve how all this works. At some point, users may well be getting their own access keys, so bad uploaders can be cut off.

These tags could be malicious, but when I ran the PTR, accidents due to mis-parses or ESL (English as Second Language) confusion vastly overwhelmed any other problem.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

74f865  No.13706

File: e91af8de30ff506⋯.jpg (2.32 MB, 1958x3321, 1958:3321, 64d222943d5e00e4e1e9908b67….jpg)

Is there any way to force redownloading files?

There's a few pictures that are broken due to HDD failure, but the only way to redownload them is to permanently delete the file then either import it or redownload from an URL ignoring deleted files.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

aeee7a  No.13707

How does Hydrus handle continuous downloading of resources with anti-chronological post order? Such as the boorus where the recent posts appear on page 0.

I mean, suppose Hydrus has downloaded X pages and was stopped, in the mean time Y new images appeared, so now when it tries to resume from page X it won't be at the same spot where it left off.

Is this all accounted for and I shouldn't worry when making my own parser?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

74f865  No.13708

>>13707

It checks the pages in order and saves the URL it finds to download. Accessing a page is pretty quick so it isn't any problem unless you stop at X page while there were X new pages of content, it'll ignore the old files but download new ones.

Say there's 5 files per page, you download two pages and stop, while you're not downloading 15 new files get uploaded, it'll continue from that second page next time, meaning it'll skip the first 5 recently uploaded and download the Y files from then on, including the ones you already downloaded.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8ab7c4  No.13719

Is it possible to delete all PTR data from my local database? I just use hydrus to organize my reaction images and realized I don't really want 25+gb of tag data from the PTR on my hard disk

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

62e53e  No.13722

>>13706

Thank you for this report. For hard drive failure, there is: hit database->maintain->file maintenance->review scheduled file jobs, then go to the 'add new work' tab.

You add jobs to files as selected by a normal search, (so sometimes you might want to say 'all jpegs' or something) but since your problem probably hit all files randomly, you probably want to queue up 'system:everything' (or if you do not see that, do this for inbox, then archive) and then hit 'run this search' to load those files to get the job.

In the 'add job' section, make sure 'if file is missing/incorrect and has url, move file out and try to redownload' and then click 'add job'. Then, in the background, your client will check every file in your db that it is what it thinks, and if it is not, it will queue it up in a new download page. Depending on the size of your collection, this may take a very long time and take a whole heap of HDD and CPU since it needs to load and think about every file you have. You might want to break it into batches or pause/play it under that same 'file maintenance' submenu only when you are happy for it to go.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

62e53e  No.13723

>>13707

>>13708

Yeah, if you are making your own parser, as long as the site you are parsing provides files in 'newest first' order, you are good. You only need to concentrate on pulling urls from each page and then any available 'next page' url. Subscriptions have a whole bunch of logic to think about this stuff and decide whether they need to get the next gallery page or if they have caught up to where they were on a previous sync. Then only requirement is that the site serves files newest->oldest like a typical booru.

Note that regular download pages are one-shot and do not do this sort of syncing logic. You can have any kind of 'highest rated first' or whatever sort order and it doesn't matter–these just can't be turned into reliable functioning subscriptions.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

62e53e  No.13724

>>13719

You can delete most of it. Hit services->manage services and find the 'hydrus tag repository' entry for the ptr and delete it. It may take the dialog a minute or two to close as it finishes the big delete job, so give it time if it hangs.

Then, when that is done, it is usually a good idea to restart the client. Then hit database->maintain->vacuum->full, which again will take a few minutes, but will recover your hard drive space.

Unfortunately, the 'master' definitions are not currently removed, so your db will still have a ~7GB client.master.db. I have a plan this year to write maintenance code to recycle this info and recover the space, but it is not in yet.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6bd03e  No.13746

File: a9387c57f50bec0⋯.jpg (46.42 KB, 1365x124, 1365:124, 4745745.jpg)

How do I set hydrus to aline my thumbnails again? Can't remember what update changed this but I remember hydrus was able to aline thumbnails perfectly whenever I clicked on or highlighted an image in the top row. I could never describe it good enough to ask but it became a little annoyance to me.

Another annoyance, whenever I switch between pages/tabs. Say I selected a certain number of images that I want opened to another page/tab. If I delete any images for the new 2nd tab, the 1st tab will reset the scroll bar back to the top. If that makes sense. Before it didn't used to do that and just kept its scroll position like when switching between any other tab and now it doesn't. So basically I want it to keep my scroll position like before.

Both these little annoyances started happening after the last big hydrus project/update so I'm not sure if their was a setting or something I missed.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3e79ae  No.13751

File: 595f8f89ce34246⋯.png (97.9 KB, 860x937, 860:937, processing.png)

I updated by about 20 versions to version 386. I remember there being some changes associated with the public tag repository around the time that I started my old version of Hydrus. I had been able to sync with it, then it stopped working and Hydrus notified me that I had to update to a new version in order for it to keep it synced. I have only updated two weeks ago so I have a lot of processing of the PTR to do, apparently.

My questions are: What is the nature of this processing? What does a single update correspond to, and what does a single definition correspond to? Why does it only do a double digit number of rows per second?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3e79ae  No.13752

Sage for doublepost. I looked into the help files, especially help/database_migration.html. Fucking hell, it was worth taking the effort to move Hydrus to my SSD and keep the media on my HDD. It's now processing 100x faster.

I have some feedback for that help page and the process in general. It seems to me that moving your media files and their thumbnails away from the Hydrus install directory is a more common use-case than moving the SQLite database away from the Hydrus install directory, but that page places the 'informing the software that the database is not in the default location' heading first. I was getting confused while trying to figure out what to do, thinking that I'd have to pass it a command-line argument at some point. But if you set the non-portable locations of your media files and your media thumbnails and move those files, you can afterwards just close Hydrus, move the entire Hydrus directory from your HDD to your SSD, open it again, and it'll just work. Shouldn't that be the thing that is emphasised on that page, and the detail about moving the database to a non-default location should come last? I also found the thumbnail location override setting confusing, because the thumbnail directory path is listed in two places thanks to that setting, and because I don't see why specifying the thumbnail directory's path should work any differently to specifying the media file directory's path(s).

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

aeee7a  No.13753

File: 3c581d01a076370⋯.png (24.72 KB, 794x349, 794:349, ClipboardImage.png)

File: 29808ea66330da5⋯.png (15.3 KB, 455x267, 455:267, ClipboardImage.png)

How do I parse this kind of JSON in my custom parser?

The server returns either a photo object with type=photo and more data in a subtree by the "photo" key with URLs to download images, I can handle that fine. (first image)

Or it can return a "doc" type object with "doc" subtree with an URL to download it. The doc can be any file, I only want to download it if has "ext":"gif". However the URL doesn't have the extension inside of it and I don't know how to do a check by a different key before passing URL to the downloader. (second image)

I tried to make a Veto rule which looks up .doc.ext=='gif' and vetoes if it doesn't match, but then it won't download the regular photos, since they don't even have the .doc subtree.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1e64f  No.13755

>>12641

what's the best way to subscribe to an entire booru site? I'm training a resnet to score files based on my preferences so I'll need to do hardly any work. If I try to do this in a subscription will my client catch syphilis and die? I was going to just do sort:id:asc as the query in the gallery downloader and slowly raise the file limit, occasionally deleting the query when it becomes too big and incrementing the start id like "sort:id:asc id:>10000". Also, since I'd be downloading *everything* I wouldn't want to grab the latest stuff first and accidentally download something weird the moderators haven't purged yet.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1e64f  No.13756

>>12641

>>13755

also, regardless of how I setup syncing, I'm probably going to download deepdanbooru2019 first to speed things along and use the client api to import the urls + tags -> hashes

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1e64f  No.13757

>>13752

>Fucking hell, it was worth taking the effort to move Hydrus to my SSD and keep the media on my HDD. It's now processing 100x faster.

intel ssd optane 900p is literally 10x faster than an nvme SSD so if you gotta go fast you could throw your db on a $300 optane

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8  No.13759

easily the most 'tism pleasing program i've ever encountered. magnificent software is magnificent. also at the risk of sounding gay as shit the "Act from need, not toward purpose" line in the help pages was profoundly impactful. super nice little unexpected bonus, that. all my gratitude are belong to hydev on both counts.

regarding questions, am i ok updating version 370 -> 380 -> 387 or should i hit 373 first and go from there? does the move to qt mean anything for the 10max update interval?

next, i can't seem to get a downloader for 4plebs working. test parse all checks out. keep getting file ignored error 403 no matter what i try or how i tweak things. halp?

and lastly, could you take a look at the code which manages tag parents/siblings via the context menu next time you're poking around in there? specifically the part that preloads whichever tag was right clicked on so that it shows up as already entered in the left-hand column? dunno if it's the preloading that does it or the fact that it's present on every tab/service but those mappings set up via the context menu have a tendency to end up getting cross applied to other services. kinda seems like maybe stuff like "both fields required for new pairing" and/or "don't auto apply uncommitted changes" is getting bypassed somehow.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

63c965  No.13768

I may be the world's worst googler, I apologize in advance. Has the nijie login script been updated lately? Since the recent changes to it none of the 3 I have are doing the job.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6c6282  No.13778

File: 0972dfbaf2bcbf7⋯.jpg (143.71 KB, 1773x1468, 1773:1468, 0972dfbaf2bcbf75849637d0ba….jpg)

Is there any way to, when importing from a Twitter URL, to generate a "creator:" tag with the username?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

18488b  No.13779

>>13778

add this to the twitter parser


[30, 4, ["creator tag", 0, [27, 6, [[26, 1, [[62, 2, [0, "div", {"class": "user-actions"}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]]]], 0, "data-screen-name", [51, 1, [3, "", null, null, "example string"]], [55, 1, [[], "parsed information"]]]], 0, false, "creator"]]

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

63c965  No.13781

Has the OR and AND dialogue been replaced completely? If so, how would I go about using OR in a search? My old tabs using it are still around, but I cannot make a new search using it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

340098  No.13782

>>13779

Works perfect, thank you friend!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

288a15  No.13783

>>13781

hold shift while pressing enter on a search tag to add it to an OR search, or you can click the OR button on the right of the autocomplete popup for a interface for an advanced OR search.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

de0135  No.13784

Does anybody know how I can add the login details for e621?

I've tried some Q&A threads but I can't find the details on how to do this.

I tried the danbooru login script but it doesn't work, I get:

The login for "e621.net" failed! It will not be reattempted until the problem is fixed.

The failure reason was:

Missing cookie "_danbooru2_session"!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3e79ae  No.13786

File: 01a746c070aea18⋯.png (18.26 KB, 205x295, 41:59, fileextensions.png)

What's with these files in my database folders that have no extensions? Adding the usual image file extensions didn't make my image viewer recognise the file as an image, and searching the hash directly didn't find anything either. Are they repository updates from the public tag repository? I did notice that I have 31034 files under 'all local files', 23018 under 'my files', 7979 under 'repository updates', and 37 under 'trash'. The first number is the sum of the rest.

Speaking of which, how exactly do these repository updates work, and why do I have 7979 of them?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8c7a30  No.13788

>>13786

>Are they repository updates from the public tag repository?

Yes

>Speaking of which, how exactly do these repository updates work, and why do I have 7979 of them?

As new tag mappings are added to the repo, they server creates update files and they are downloaded and processed into your DB.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

43d1aa  No.13790

>>13784

Yes seconding this e621 is useless without login! HOW DO YOU LOGIN?!>>13784

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2  No.13793

>>13746

Should be options->thumbnails, "EXPERIMENAL: Scroll thumbnails at this rate…". It'll tick down one thumbnail height at 1.0. Also there is a thumbnail % visible scroll check, which might actually be what you want here–set that to 0% and it'll always scroll.

EDIT: Seems like that isn't working right? I'll look into this in case it is a bug. Might have been Qt that changed that, or some other thumbnail scroll-to stuff I did recently.

I have heard about the 'scrolling to top' issue. I did do some work on this,but I will check this as well. If you are on an older version than current, updating may fix this.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2  No.13794

>>13751

tl;dr: Let it work, it'll take about a month of background work in idle/shutdown time to catch up. Read EDIT below.

I don't have time to do full technical explanation, but: The tag repo sends all tags to clients, basically mirroring the server db to the client, so that all work happens on your hard drive and your CPU.

Every day or so, the server generates compressed update files of all the changes in the past cycle. These are zipped up json, mostly lists of numbers. Then this portable data needs to be integrated into your client, lining up serverside file and tag definitions with what your client knows, and then adding the actual file-tag pairs so you can see PTR tags for your files. There are also auxiliary operations like maintaining the autocomplete count cache and other metadata.

EDIT: Since you are getting double-digit rows per second, are you on an HDD instead of an SSD? The modern PTR has 700 million mappings, and does not work well on a high latency drive like a hard drive. If you cannot move your hydrus install to an SSD, you should pause it now as you will never be able to sync up to date and keep up to date.

Info on migrating a db here:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2  No.13795

>>13752

Fugg, should have read more. Great, I am glad you got yourself sorted.

Thank you for the feedback on the help. That makes sense. I will make a job to give that help page and the UI a pass.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2  No.13797

>>13753

I think you want to make a 'subsidiary page parser'. This is advanced and not well documented, but basically it lets you split a post up into sub-posts. This part isn't important for your case, but what is important is a veto on a subsidiary page parser only cancels the content of that subsidiary parser.

So, something like:

MAIN PARSER
photo content parsers
content parsers anything photo and doc have the same structure for, maybe tags
SUBSIDIARY PAGE PARSER
VETO if no gif
doc specific content parsers like url

In this case:

If you have a photo, it pulls the photo content, then runs the subsidiary parser, which terminates because no gif. = photo url

If you have a gif doc, it pulls no photo content, then runs the subsidiary parser, which pulls gif doc content. = gif url

If you have a blah doc, it pulls no photo content, then runs the subsidiary parser, which terminates because no gif. = ignored import status

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2  No.13798

>>13755

>>13756

I very strongly advise against downloading an entire booru. If you have not seen this relatively new document, I have written several of my thoughts, which are based on experience with many users, here:

https://hydrusnetwork.github.io/hydrus/help/getting_started_downloading.html

That said, some users want to mirror for archival reasons, or because they just want a specific small niche booru, or they are madlads, if this applies to you:

Most boorus will accept '*' as an input query, which will just return everything. You would do as you are thinking here, setting up multiple download page queries in batches, variations on 'id>x', to fill in all the gaps, and then when you are caught up, you would set up a sub for '*' to catch all new content thereafter. If the booru is big like danbooru, it probably runs too fast for a hydrus subscription to to keep up with reliably. My guess for a technical limit, for a client that runs 24/7, would be a booru that spikes during a busy period at more than 100 files an hour.

I will again say getting everything is almost certainly impossible for any human to sift through properly. If you haven't done much archive/delete processing, I recommend you try doing 1,000 files. Time how long it takes to do properly, then multiply that by how many files you are expecting to get. If you want to chase spicy content before it is deleted, I would recommend you try to filter your query to catch that specifically, or find a different site to download from that simply is ok with hosting it. Maybe search for filetypes, if the booru accepts it, or 'does not have a creator tag yet' if the booru supports it, anything to help you separate the wheat from the chaff.

The most productive way to set up subscriptions is for artists you like. You will soon be overwhelmed with that like everyone else, and all the happier that you don't have a 30:1 trash-to-good ratio.

Let me know how you get on!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2  No.13799

>>13759

Thank you, I am really glad you like it. I made it for other Anons like me, so we are brothers.

You should be good on the Qt update step, if you haven't tried it yet. There were no special install instructions, thankfully. As always though, the best thing to do before a big update is just to make a complete backup beforehand. You can then go nuts and even try it all in one step, since if it all breaks, you can just roll back, no worries.

For 4plebs, if you open the 'file/watcher import status' window on the downloader (hit the 'table' icon button), whichever is breaking, and find the url that is failing in the list, is there extra info in the 'notes' column? Right-click->copy notes on that to get it in your clipboard, then paste it into a document editor. It should give more info, and hopefully a full html readout of the error page, which should give more info. 503 is often CloudFlare problems, but 403 is Forbidden, which usually means you are hitting a malformed url. Did you make the downloader yourself? Does it work in the parser testing window? Is there maybe a api url converter in the url class that is hitting the wrong thing? help->debug->report modes->network report mode can help here.

You can also pastebin/email/whatever it to me, and I'll have a look.

Thank you for the report about siblings and parents. I can see how something like that would happen, I'll check it out.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2  No.13800

>>13768

I am afraid I do not know much about it. The one here:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/blob/master/Download%20System/All-in-Ones/Single-Sites/nijie-artist-2018.12.14.png

Looks to be more than a year old, so if the site has just changed, I would not be surprised if it has broken. If so, I will remove it.

For now, for any non-simple-booru, I am recommending people use Hydrus Companion to log in. It is a browser add-on that lets you log hydrus in by copying your browser cookies across. The page with instructions is here:

https://gitgud.io/prkc/hydrus-companion

However, if the layout has changed and login script is broken, the downloader itself may also not work. Let me know if this is true, I'll give the guys on discord a buzz and see if anyone wants to update it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7f99a2  No.13801

>>13784

>>13790

For now, I recommend you use Hydrus Companion, a browser add-on that lets you copy your browser's logged-in cookies straight to hydrus:

https://gitgud.io/prkc/hydrus-companion

This is a good blanket fix for sites that do not have native hydrus login script support.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1e64f  No.13807

>>13798

>If you want to chase spicy content before it is deleted,

nono, I specifically would want the opposite of that

>I will again say getting everything is almost certainly impossible for any human to sift through properly.

that's why I'm training a resnet to do it for me :^)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

74f865  No.13810

Is there a way to have the duplicate filter only show the pixel by pixel duplicates?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8  No.13812

File: 7bef75e044d352f⋯.png (431.09 KB, 1256x1294, 628:647, 4plebs_error.png)

>>13799

cheers lad

welp. said fuck it and went for it. i updated backup then updated hydrus clear through to 388 in one go. nothing 'sploded; db booted; all seems well. happy days. new qt version looks and feels sharp af too. i liek.

regarding 4plebs downloader, still no dice there. i was half hoping the update might just end up sorting it out but no such luck. anyhow. the list below isn't exhaustive but hits the main bulletpoints of what all i tried:

-new custom downloader made from scratch → test parse are successful; download is fail

-edit of first downloader stripped down to bare-bones → test parse are successful; download is fail

-duplicate (structurally speaking) of first downloader dialed in for a different but similar website → test parse are successful; download is successful

-import of cuddlebear downloader with smol modification → no test parse because am jsonlet; download is fail

-reimport of cuddlebear downloader left alone this time → plug-n-play mode so no test parse; download is fail

-simple downloader on default setting after nuking all traces of 4plebs from url classes and parsers → plug-n-play mode/no test parse; download is fail

>"help->debug->report modes->network report mode" can help here.

neat, that's handy af. picrelated. left side is using bog standard unaltered cuddlebear (json parser & 3 url classes from github repo). right side is the one i made (parser and 2 url classes, link here https://pastebin.com/F4kA9kxw ). the 4plebs urls don't "look" malformed and browser will navigate to them just fine but i can't seem to get anywhere with it when i try going through hydrus. all paths lead to 403.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6bd03e  No.13815

File: 8323f2b1bb0a947⋯.jpg (69.21 KB, 1361x432, 1361:432, 45745747.jpg)

>>13793

>Should be options->thumbnails, "EXPERIMENAL: Scroll thumbnails at this rate…". It'll tick down one thumbnail height at 1.0. Also there is a thumbnail % visible scroll check, which might actually be what you want here–set that to 0% and it'll always scroll.

Yeah this is what its always set on for me. I tried changing the "Do not scroll on key navigation" percentage but I haven't really notice anything or at least relating to my issue and setting the "EXPERIMENAL: Scroll thumbnails at this rate" to 0% gave me errors, if that's the one you mentioned to set.

Also I updated to version 389.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6d4bc3  No.13822

>>13801

Friendly neighborhood anon here, I whipped up a login script for e621 based off of the danbooru login. Not sure if it's 100% efficient(never made a login script before) - but it works, at least.


[73, "e621.net login", 2, ["e9438b0ebc12025f09faf1366734b927fef9832ce6d7d37d68f52da37512d0ea", [21, 1, [[], [], [], [[[51, 1, [0, "remember", null, null, "remember"]], [51, 1, [3, "", null, null, "Ph2fFwrvF7603984"]]]]]], [26, 1, [[72, "password", 1, [1, [51, 1, [3, "", null, null, "example string"]]]], [72, "username", 1, [0, [51, 1, [3, "", null, null, "example string"]]]]]], [26, 1, [[74, "hit home page to establish session, ignore passthrough", 2, ["https", "GET", null, "/", {}, {}, {}, [21, 1, [[], [], [], []]], [26, 1, []]]], [74, "hit login page to parse auth token", 2, ["https", "GET", null, "/session/new", {}, {}, {}, [21, 1, [[], [], [], []]], [26, 1, [[30, 4, ["get authenticity token", 14, [27, 6, [[26, 1, [[62, 2, [0, "input", {"name": "authenticity_token"}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]]]], 0, "value", [51, 1, [3, "", null, null, "example string"]], [55, 1, [[], "parsed information"]]]], 0, false, "authenticity_token"]]]]]], [74, "post login form", 2, ["https", "POST", null, "/session", {"password": "password", "username": "name"}, {"commit": "Submit", "remember": "1", "utf8": "\u2713"}, {"authenticity_token": "authenticity_token"}, [21, 1, [[], [], [], [[[51, 1, [0, "remember", null, null, "remember"]], [51, 1, [3, "", null, null, "$5betre%0"]]]]]], [26, 1, []]]]]], [["e621.net", 2, "Login required to access special content (clear your blacklist)."]]]]

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f04aba  No.13825

>>13810

Not yet. Pixel dupes are recognised in realtime at the moment, but I have a plan to make a cache for this data and let you search for them specifically. I do not know when I will get to this.

I expect pixel dupes to be the first scenario you will be able to set up auto-duplicate resolution rules for. (e.g. 'if potential pair is pixel dupe and one is jpeg and one is png, auto-select the jpeg') With luck, we'll have auto-resolution of many imageboard clipboard.png files.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f04aba  No.13826

>>13812

Thank you–I think I figured this out. There is a little-used feature under network->data->manage network headers that lets you set the User-Agent (or any other http header) of hydrus for different network contexts. There should be a couple of examples in there you can basically copy.

If I add a new User-Agent entry for archive.4plebs.org set to my web browser's string, the download works. Please forgive the bad UI–this is even more hacky code than usual. My guess is they have set up something that says 'if the User-Agent is anything other than these known browser strings, return 403 forbidden'.

When you export a complete downloader package under network->downloader definitions->export downloaders, there is a button to include these custom http headers, which other users will be able to import too under network->downloaders->import downloaders.

The files under the domain i.4pcdn.org do not seem to need the User-Agent.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f04aba  No.13827

>>13822

Thank you for this work! I will roll this into 390.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8ab7c4  No.13830

Is there a way to give two IMAGES (not tags) a parent-child relationship or similar?

Similar to the "this image has x child images [link]" feature on borrus. Basically I want some way to link e.g. a two-to-four image set together

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

57d276  No.13831

Does anyone have (or is the Dev planning) a gallery downloader for AGN.PH? Nintend0h has been getting uppity with takedowns, and I wanna rip pokemon shit before they get shitcanned… just in case.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7a1a55  No.13833

>>13826

This is a slightly better version, now that I messed around with the login script settings more. I can't improve much else until the new version where the wx->qt bugs are fixed.


[73, "e621.net login", 2, ["e9438b0ebc12025f09faf1366734b927fef9832ce6d7d37d68f52da37512d0ea", [21, 1, [[], [], [], [[[51, 1, [0, "remember", null, null, "remember"]], [51, 1, [3, "", null, null, "Ph2fFwrvF7603984"]]]]]], [26, 1, [[72, "password", 1, [1, [51, 1, [3, "", null, null, "example string"]]]], [72, "username", 1, [0, [51, 1, [3, "", null, null, "example string"]]]]]], [26, 1, [[74, "hit home page to establish session, ignore passthrough", 2, ["https", "GET", null, "/", {}, {}, {}, [21, 1, [[], [], [], []]], [26, 1, []]]], [74, "hit login page to parse auth token", 2, ["https", "GET", null, "/session/new", {}, {}, {}, [21, 1, [[], [], [], []]], [26, 1, [[30, 4, ["get authenticity token", 14, [27, 6, [[26, 1, [[62, 2, [0, "input", {"name": "authenticity_token"}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]]]], 0, "value", [51, 1, [3, "", null, null, "example string"]], [55, 1, [[], "parsed information"]]]], 0, false, "authenticity_token"]]]]]], [74, "post login form", 2, ["https", "POST", null, "/session", {"password": "password", "username": "name"}, {"commit": "Submit", "remember": "1"}, {"authenticity_token": "authenticity_token"}, [21, 1, [[], [], [], [[[51, 1, [0, "remember", null, null, "remember"]], [51, 1, [3, "", null, null, "$5betre%0"]]]]]], [26, 1, []]]]]], [["e621.net", 2, "Login required to access special content (clear your blacklist)."]]]]

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8  No.13834

>>13826

that did it! thanks so much again man

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a59a6e  No.13835

>>13830

Not yet. The next big push on the duplicates system will expand the 'alternates' file relationship into a full-featured file-association tracking system, as you say exactly like boorus do.

Ideally you'll have multiple new relationships to set like 'recolour', 'costume change', and hopefully some basic indexing as well so you'll be able to order several WIP (work in progress) versions of a file.

It'll also have ui support, so if you are looking at a file in the media viewer that has a parent/master, there will be a quick way to navigate to that without losing your place in the navigation carousel.

>>13831

In general, I do not write downloaders any more. There are too many sites for me to keep up with and get my regular work done. If you have not seen it yet, the main repository for user-made downloaders is here:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/Download%20System/All-in-Ones/Single-Sites

There are several users on the discord working together to make downloaders as well, here:

https://discord.gg/3H8UTpb

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a59a6e  No.13836

>>13833

Thank you, I have copied it. As I said in the bugs thread, I have fixed those two things (I assume you) highlighted, and I gave a quick click through all the other edit login script UI. I think it is all good now.

I'll put off rolling this in for tomorrow's release, just in case you would like to do more. Please let me know once you are up to 390 if you are happy with this as-is or if you have a new one.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

fb696e  No.13844

>>13836

I've updated to 390 and everything looks like it is working as it should - the second script is good to go as is.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6b8283  No.13847

I started getting an error when downloading from Danbooru:

"Could not find a file or post URL to download!"

This just started happening today, and I didn't change anything on my end other than updating to v390. Any ideas?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6795b4  No.13849

Is there a way to move a selection of files from one database to another while retaining the tags? I could do it one tag at a time, but it'd be great to have an easier way of doing it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c48a70  No.13852

Hi, I'm looking at some programs to see what fits my needs and didnt see this in the docs. How does hydrus handle not downloading the same image twice? If I delete a crap image, will I have to worry about it being redownloaded again from the same place? (as an extra detail, I'm guessing you can't help it if it's a different site with the same image though)

Lastly, as a test, I trying to import 25+ mp4 and webm videos that all fail with "stream missing" errors. Is this a known thing? Thx and sorry if it's a silly questin.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0aac4a  No.13854

>>13852

Hydrus will recognize the hash of files that you already downloaded and skip them. You'll see "already in db". Files that you previously deleted will be tracked the same way.

Files that are similar but not identical will show up in duplicates processing, and you can choose how to handle them yourself.

The help section on the Hydrus website has some good info on the basics like this.

I'm not sure about the .mp4 and .webm problem. Stream missing makes it sound like the codec isn't parsing the files properly, but that's just a guess.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0aac4a  No.13855

>>13847

Fixed: I changed the danbooru file page parser to look for "class=image-container" instead of "id=image-container".

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1c92d  No.13857

File: 3da07e7426cf5d6⋯.png (11.62 KB, 1089x102, 363:34, Capture.PNG)

First time user here.

Already add my login credential for both gelbooru and sankaku sites.

Why the gallery download doesn't works with loli tag for gelbooru? it said "The parser found nothing in the document!"

It works normally with other tags for gelbooru.

Gallery download works normally with sankaku for loli tag, but for some reason the tag imported doesn't add loli tag into the downloaded images, everything else got added, I even got loli_face tag but not loli tag.

Also can I changed the position of the windows?

I want to switch the place of preview and thumbnail windows.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1c92d  No.13858

File: fc68a50925a706a⋯.jpg (171.42 KB, 1806x964, 903:482, Capture.JPG)

Also is there a way to make Hydrus preview the thumbnails first before downloading the original image just like Grabber and then I can pick which image to download later one by one using right click or shortcut?

Because downloading each image and then picking which image to saved and delete take too much time and bandwidths.

Grabber can do this but this app doesn't support local database and tags.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

669e9c  No.13859

Is there a way to autotag the images I import without using the bloated public tag repository? I'd only like for the artist to be added, everything else is unnecessary for my purposes.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d9b111  No.13861

>>13854

Oh, the hash makes more sense. Yeah I have no clue about the stream missing, I have the newest ffmpeg codecs I could get from the ubuntu 19.04. Just let me know if there's something I can try.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2a4917  No.13862

>>13861

Responding to myself: looks like the hydrus client didn't detect my ffmpeg codec in the "about" page. I tried a separate client download and it recognized and work.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

63d096  No.13865

After a recent client update a tiny portion of my files have begun showing up glitchy in the client (low to massive distortion). So far I've found 5 out of 200, all of them are webm or mp4.

They ONLY glitch when viewed through the hydrus client, opening them in any other program has them showing up fine.

Is there a way to fix this?

If not, is it possible to edit a file's metadata (mainly import date) so I can import them again and put them back where they belong?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64  No.13867

File: a9d62e7bc64c220⋯.png (7.59 KB, 345x100, 69:20, ClipboardImage.png)

Regarding the URL classes that redirect from a web page to an API call, which checkbox settings take effect? The page class or the API class ones?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d8fdde  No.13868

>>13857

Gelbooru needs the cookie fringeBenefits=yup to view that content, make sure you have it in hydrus.

>>13858

No

>>13859

Parser's can get tags from the site they are downloading from. Most parser's should already get some sort of tags, you can choose which one's it gets in tag import options.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ee6909  No.13869

>>13868

The cookie.txt fixed the problem.

Since there are no way to changed how the GUI looks, is there a way to make Hydrus act like a normal booru so I navigate them from a web browser?

Hyrdus is a great tool for scrapping image from boorus, but the UI navigation is really bad.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

100de7  No.13873

If a hydrus database is corrupted, but in a subtle way that isn't noticed immediately by the client on boot or during automatic maintenance procedures, if that can even happen, then is it possible for the backups that you make to become corrupted too, or is there some kinda check in place to make sure that the backup is safe before writing it to the backup location?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d2387f  No.13875

>>13869

There are some third-party tools that make a web UI for Hydrus. This one is probably the best at the moment.

https://github.com/mserajnik/hyve

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d47a16  No.13876

Is there any way to change the font size of the GUI? I can't find it within the options.

I'm on a 4k monitor and I am going to go blind

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6adfe8  No.13877

>>13875

The last time I tried this it said, 404 when I tried to connect the API.

And there are no tutorial for non tech savvy people.

What is this even mean?

HYVE_API_BASE=/api: the base path of all the API routes. No trailing slashes.

HYVE_MEDIA_BASE=/media: the base path of all the media routes. No trailing slashes.

VUE_APP_IPFS_GATEWAY_BASE_URL=https://ipfs.io/ipfs: the URL to the IPFS gateway you want to use for IPFS links (only available when connected to hydrus client). No trailing slashes.

VUE_APP_HYVE_API_URL=http://localhost:8000/api: the URL to your hyve HTTP API. No trailing slashes.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

543601  No.13879

Is there anyway to change where the temp dir is? I'm trying to free up some space on my ssd and there are ~2600 Hydrus files and 14 empty folders in the AppData\Local\Temp folder. Would like to move the temp dir to another drive or reduce the size if possible

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

000000  No.13880

How do I run Hydrus over TOR?

Assuming I don't want my ISP snooping on my DNS queries to figure out what weird sites I scrape from.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d2387f  No.13881

>>13876

You could make a qss and change the font size in there, there is a readme in hydrus root dir/static/qss. Something like this.


QWidget {
font-size: 100px #whatever font size
}

>>13879

You can change it with a launch flag. client –temp_dir="temp/dir/path"

>>13880

You should be able to add the TOR proxy into hydrus proxy settings in options>connection

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d2387f  No.13883

>>13877

>HYVE_API_BASE=/api: the base path of all the API routes. No trailing slashes.

Thats the path for the API. Say you wanted the API at localhost:8000/ass, you would set it to "/ass"

>HYVE_MEDIA_BASE=/media: the base path of all the media routes. No trailing slashes.

Same thing but for the media is servers. e.g. localhost:8000/media/ass.png

>VUE_APP_IPFS_GATEWAY_BASE_URL=https://ipfs.io/ipfs: the URL to the IPFS gateway you want to use for IPFS links (only available when connected to hydrus client). No trailing slashes.

This shouldn't be important if you don't use IPFS.

>VUE_APP_HYVE_API_URL=http://localhost:8000/api: the URL to your hyve HTTP API. No trailing slashes.

The URL to the API you set in HYVE_API_BASE for the web front end to access.

I agree that it's not the easiest for non tech savvy people but then again neither is Hydrus.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

543601  No.13884

File: 7fcffa918988142⋯.png (6.16 KB, 442x166, 221:83, hydrus.PNG)

>>13881

The client won't launch if I use this as a launch flag

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

543601  No.13885

>>13881

>>13884

I got it to work with two dashes instead. Am I free to delete the old temp files now?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7905a9  No.13888

Is there a way to control, how many images are downloaded per second in a gallery download?

Seems like now it does 1 image per second, and even though I'm in the advanced mode, I can't find a way to change that.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6adfe8  No.13890

>>13883

>but then again neither is Hydrus.

I got no problem in installing hydrus and it got a wiki site explaining on how to use it.

Hyve only got a readme full of tech jargon.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6adfe8  No.13891

>>13868

Loli tag still not imported from sankaku even with cookies.txt added.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d31e4f  No.13892

>>13890

you could try this one then, I thought it was pretty easy to setup, though no documentation. All you have to do though is run npm install then ng serve in its folder after installing npm and angular-cli.

https://github.com/floogulinc/hydrus-web

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7203bd  No.13893

File: 0c202846c26f034⋯.jpg (164.19 KB, 1889x981, 1889:981, Capture.JPG)

I tried Suika https://github.com/Suika/hydrus-docker Because it got build in Hydrus client in it, no problem during installation with docker compose.

I can download images from boorus and browser them with Suika's built-in Hydrus using VNC remote, but when I open the Suika's Webui I got 404 in the API page and no image in the gallery page.

My Suika compose setup https://pastebin.com/wthuSjYE

Since I already got Suika's Hydrus installed and some database and images in my NAS I tried Hyve next.

I added Suika's db folder and client_files folder into Hyve's docker-compose.yml.client.example then ran it with my Portainer stack.

Open webui and got this

>ApiUnavailableError

>Could not connect to hyve API. Please check your .env configuration.

My Hyve client compose https://pastebin.com/3Vfj4TuC

I changed hyve-sync ports: from '8000:8000' to '7000:8000' because my Portainer webui also used the same ports and I can't changed them.

Next I tried to import Hydrus images and tags into Szurubooru, with Szuruporter, dunno if i'm just a brainlet or was it because Szuruporter never got update anymore because nothing works.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8  No.13895

>>13849

https://hydrusnetwork.github.io/hydrus/help/advanced.html

Hydrus Tag Archives under the heading for tag migration sounds like it'll accomplish what you're after.

>>13859

check the link above, read the section called importing and adding tags at the same time. if that's N/A to however you've got your files named or whatever your folder structure is, then probably the public tag repo would be your best bet. can't say for sure as i don't use it myself but there might be a way to whitelist the artist tags and thereby reduce the bloat.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a3a007  No.13901

When I set a custom path for an external program, Hydrus does open it, but it doesn't open the file in it. Do you know why this is?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

72b0bd  No.13902

File: 41f8b03b2e8b6c3⋯.png (90.91 KB, 1314x570, 219:95, 2020_04_04_23_44_38_file_i….png)

>>12641

Why do most of my downloads from danbooru get ignored?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cfaa81  No.13904

Is there a way to count the total number of tags, or similar statistics other than the "how boned am I" window?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

4f77c8  No.13905

Is there anyway using a parser to add to an images notes.

For example, some images have a story in the description and so I currently have to manually copy it into the images notes.

Thanks

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64  No.13906

File: e1e432889f51678⋯.png (23.65 KB, 507x254, 507:254, typo.png)

There is a typo in the character type dialog, "minUmum".

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d34df0  No.13908

>>13902

Are you on the latest version? Danbooru changed some stuff so the parser had to get updated.

>>13905

Unfortunately not yet.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64  No.13909

>>13905

You can use tags for that. You'll lose line breaks, but at least it'll be searchable.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c7dfdd  No.13910

Just wondering, is there a guide in video form for info about Hydrus Network?

The main guide is helping me a bit but i thought a video of footage of the software being used could help too

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

644098  No.13919

>>13877

Till today, i still don't know how to make this run.

Still got "Could not connect to hyve API. Please check your .env configuration."

I don't even know where do I get my API url from.

Here my compose setup https://pastebin.com/jdZK0ite

What I want to do is setup a Hyve on my NAS server using docker, let all my booru images and hydrus database saved inside my NAS.

While using Hydrus windows client to download my images and send them to Hyve server folder on my NAS.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d43b0a  No.13921

Hiya. Recently got into hydrus and am loving how easy it is to download and sort/process the thousands of images/manga etc that I usually download manually, so thanks for that!

Just looking for a solution to the following issue:

Version - 392

Situation - Downloading an artist's work via "pixiv artist lookup" gallery downloader

Problem - Some posts are being ignored with comment "Ugoira veto".

Is there some way to disable this veto and have it pull the ugoira files too? I can't find anything in the changelog about ugoira since version 329, and can't find any settings that would relate to it. Ideally I don't want to have to download and tag them manually.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

644098  No.13922

>>13919

Wrong quote.

>>13883

Fucking captcha.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2207a6  No.13923

>>13921

>Is there some way to disable this veto and have it pull the ugoira files too?

Not with just disabling the veto, It would require an additional parser but even then ugoria isn't a video file, you could only get it as a zip of images.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c4b652  No.13926

Hello, is there a way to automatically hide the cursor after opening the media viewer instead of having to move the cursor around?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

063396  No.13927

Is there a shortcut for undoing a deletion and adding it back to the current view? Neither Ctrl+Z or setting a hotkey for "undo" undeletes the file, I have to manually right click to undelete. If not, am I using the wrong workflow? Thanks

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30  No.13938

The cycle of me being unable to check and respond to my messages every day continues, worsened by current events. I apologise. I'll keep trying new ideas here on schedule and stress budget, the ideal is 24-hour turnaround.

>>13865

Are you viewing these files through the hydrus native viewer, or the new mpv window, do you know? The new mpv player has audio support. You can also see what you have set to display files under options->media. Can you post one of the files, or otherwise link me to it, so I can test my end?

There is no way to easily edit import time right now.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30  No.13939

>>13867

Good point. I think those mostly apply just to the URL in the import queue, the actual URL being actioned, which will typically be the Post Page URL, not the API. Usually, the API conversion happens very late in the download process and is temporary, it only changes what URL is actually hit and what parser is used.

I think only the raw URL stuff, the top three checkboxes in your image, would apply to an API URL, since that stuff applies to the URL itself and recognition and parser match-up.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30  No.13940

>>13873

Unfortunately, a full integrity check is fairly slow, usually 1-10MB/second, so this is not done automatically. You can force a check under database->check->db integrity, and if you would like to set up a routine to regularly do so, it would increase your safety.

Otherwise, your scenario can unfortunately occur. The good news is that if the subtle way isn't something you or the client immediately notices, it is usually something like 'a power cut that turned off your computer while you were committing PTR data malformed a 4KB database page of 'stained brown shirt' tags on a selection of obscure.booru.org files', and since you likely don't have those files, you probably never load them and never run into the problem, and in any case PTR data, which is about 98% of all your db data, can be recovered with reprocessing, so it isn't so big a deal.

Check out 'help my db is broke.txt' in install_dir/db if you would like more background reading on this.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30  No.13941

>>13876

I am working on nicer solutions here that you can select in the client. It'll involve changing the qss as >>13881 suggests, but with 'pt' instead of 'px', and I need to fix some custom text I draw in the program that doesn't yet follow qss rules.

>>13885

Yes, you can delete anything in a temp folder if there isn't a hydrus client currently running and looking at it. Nothing in there is persistent across boots.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30  No.13942

>>13888

All of that is controlled under network->data->review bandwidth usage. It is advanced, so be careful. I strongly strongly strongly recommend against just deleting all bandwidth rules, as you'll likely catch some IP bans and also possibly overwhelm your client DB and UI.

Most of the rules that are in there by default–the 'edit default bandwidth rules' button–are set to ensure new users don't make a mistake and download 5GB of trash by accident.

I recommend you keep the 1 file per second global domain rule, but it is up to you. If you are interested, I have longer thoughts on the philosophy of not overwhelming your inbox and sanity here:

https://hydrusnetwork.github.io/hydrus/help/getting_started_downloading.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30  No.13943

>>13849

Yeah, I need to write a proper 'how to migrate files with tags' guide, as this keeps coming up, but the main solution here is as >>13895 says: use services->tag migration with Hydrus Tag Archives. Check it out, be careful because it is advanced, and let me know if you run into trouble.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30  No.13944

>>13901

It probably all depends on how the program wants to accept that launch path. They aren't all in the format:

program_exe "path"

But sometimes:

program_exe –open="path"

Or it might sperg out about paths with spaces in–don't forget the " marks.

If you open up a cmd/terminal and do a test, does the same format you have hydrus set to do work, or does it similarly just open the program? If it works in the command line, then try turning on help->debug->report modes->subprocess mode and then launching the program. This will popup a bunch of debug messages about what is about to do, and the error may be apparent. If it isn't, let me know what you'll get and I'll have a look.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30  No.13945

>>13904

Not yet. I'd like to expand this stuff in future and have some graphs and things.

>>13905

I hope to have an expanded notes system with note parsing in downloaders in the next couple of months. It will happen in a 'medium sized job' week.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30  No.13946

>>13906

Thanks, fixed.

>>13910

Not that I know of. I agree though, although I am sorry to say I don't have the time or expertise to do it myself.

>>13921

Unfortunately, there is no Ugoira support in hydrus yet, so the veto is the current 'graceful' way to error without an error. I have plans to add it, but it is a tricky format, so I can't confidently say when.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30  No.13947

>>13926

Thanks, I didn't realise that hide tech didn't kick in until an initial cursor move. I assume you are navigating by keyboard? I'll check that out and see if I can fix it.

>>13927

Shift-delete I think is the default. Please forgive this shortcut being a bit hidden, delete keys are funny, so these actions are still hardcoded and not yet moved to the full customisable shortcut system.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

096d01  No.13949

>>13947

Thanks regarding shift-delete. This still requires switching pages, tracking down the deleted item which is troublesome especially with no "order by time deleted" filter, clicking the image, and then shift-delete on it.

My ideal inbox workflow would be (text after * isn't possible currently):

A (delete bad img and auto move to next image)

L (archive image and *auto move to next image)

* Y (undelete image and auto move media browser back to undeleted image rather than tabbing to trash where the images are out of order by deletion time)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bbae30  No.13951

>>13949

Thank you. There is a two part solution here, I think:

1) (plan to work on better trash timestamp stuff deleted) EDIT: I was talking to another user about sorting by trash time not working, but I just tested it now and 'sort by time imported' works ok for trash, as long as you are looking at the 'trash' file domain, rather than 'my files' . I can see how this doesn't work when you are looking at a 'my files' page that happens to have some trash in it, though, so I'll think about this.

2) I have a barebones content undo/redo system, but it only works on a couple of things like archive/inbox, and that not well. The main thing I have to do there is keep pushing on shortcuts, which involves converting program actions from hardcoded calls to a generalised 'command' action system that all works through the same pipeline. As I move more of the program to that, I'll be able to hang those undo-able commands on an undo system plugged into that same single pipeline. Just have to keep working.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6dcd94  No.13956

Hello all,

I stopped updating hydrus after the 8chan debacle and never really went to search on how to continue getting news about it.

I noticed the current version is 398, mines is 318. Could you let me know a list of major things that were added or changed since then?

Also given this situation, I was thinking of deleting the whole database, settings, etc (except the pictures) and start over.

If I copy the client_files folder under Hydrus/db/ would that be enough to start over and keep the pictures or I need to do something else?

I used the Windows extract only version.

Thanks for all the help.

PS. Not required but is there a copy of tags of videogames/cartoons/anime I can borrow from anyone?

Current mine it has a lot of junk tags. Just looking for characters and series names.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c91700  No.13960

File: 81e08b1e8244238⋯.png (2.85 KB, 811x29, 811:29, Capture.PNG)

>>13956

Hello and thank you for everything that you have done for us,

It is my first time running Hydrus on this computer and it's not starting up, it's just taking up CPU power in the background. I don't recall it taking this long to load. I have all the prerequisites checked.

Is this normal? And if so,hould how long sit take depending on whatever pc specs you imagine? I apologize for the stupid question, it's just a little frustrating.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6dcd94  No.13961

>>13960

I think you meant to write to Board Owner/Dev. I'm not him, i'm a normal user only.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9  No.13962

File: abc953eef72057f⋯.png (14.54 KB, 541x480, 541:480, ClipboardImage.png)

I'm trying to make a parser for nyanpals.com/pettankontent but the site needs javascript to show the posts. Is there any way to load the page with JS or am I fucked in that regard?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9  No.13963

>>13956

>Could you let me know a list of major things that were added or changed since then?

If you want to waste 20 minutes reading, you could just go to the github and read the first few lines of each release.

>If I copy the client_files folder under Hydrus/db/ would that be enough to start over and keep the pictures or I need to do something else?

No, hydrus would consider them all junk files without the database file. If you don't want to update you can either export all files with tags(if you want to keep the tags) or just open the new hydrus and drag all the files in to import(you can just drag the db folder for that).

>PS. Not required but is there a copy of tags of videogames/cartoons/anime I can borrow from anyone?

If you don't mind bloat you could just download the PTR and either censor all tags except the ones you want or move all the tags you want to a new tag repository then deactivate the PTR.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

728c15  No.13969

>>13962

> Is there any way to load the page with JS

No, use your browsers development tools network tab to find out where they are loading the content from with JS and parse that instead.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

2010c9  No.13971

I'm having the same issues as >>13960

It's not starting up. I'm running the latest version, fresh install.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9  No.13972

>>13969

I managed to track it back to a JSON page and convert it to

{url:"/img/pettankontent/50a00b3bb9934e314dcbd1e6c614f2ab.png",tags:"miqo'te,keiko"}
{url:"/img/pettankontent/535c19b6fe565f1b90262cbf978285ad.png",tags:"miqo'te,keiko"}

but now I'm fucked. I don't think hydrus has any way of converting this. It's not HTML so that doesn't work, but it's not JSON either so that breaks.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291  No.13975

So I have an issue, I use a lot of personal tags and have them set up with parent tags exclusively using My Tags but they're automatically being applied to the ptr tags as well. Its pretty irritating because I specifically did it to more easily tag things for myself because the ptr has a lot of mistagged images but theres not much point of the tags are automatically being applied to the ptr as well because of the parent tags I set up in My Tags.

For example, I set up a tag called #:3d so that it automatically sorts to the top of the tag list and so I can easily organize 3d images/videos. I also set up parent tags for it (in my tags) so that anything tagged with 3d rendering and the like will get it, now like a week later #:3d is getting applied to everything in the ptr too, among others. Im not sure if thats just on my end but its pretty damn annoying either way seeing as I have to commit ptr tag changes every time I want to get rid of them. Anyway I can stop this without getting rid of the parent tag entirely?

Also, is there an option to make tags from the tag suggestion box in the manage tags window not disappear after you apply them? Before you could scroll through images and tag things without closing the manage tags window, but now the tags disappear from tag suggestions after you apply them until you re-open it. Makes tagging things a bit more inconvenient to do.

I feel like a dumbass typing tag 5 times per sentence by the way

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

64349a  No.13976

I think i messed up my installation/db by just copy pasting the install folder to my HDD with more space. I believe I have it all recovered and have access to the db again, but I can't drag and drop images into hydrus anymore, neither from Chrome or desktop/explorer. I think UAC now complains every time i run the exe. I don't remember if last time it just worked or had to change something to enable DnD.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

951b56  No.13980

>>13952

I recommend you update rather than starting over. There is lots of data in your existing db that won't carry over, and you can't really just switch out your client_files and continue.

I recommend you update 10 versions at a time, as described here:

https://hydrusnetwork.github.io/hydrus/help/getting_started_installing.html#big_updates

Let me know if you run into any trouble.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

951b56  No.13981

>>13960

>>13971

I am very sorry to hear this. I updated from python 3.6 to 3.7 in 393, so that is my biggest guess here. Can you try a couple of things for me?

If you go to install_dir/db/client - 2020-4.log and scroll to the bottom, how far does hydrus think it got in booting? What was the last step? Are there any errors there?

If you download the extract release, if you don't already have it:

https://github.com/hydrusnetwork/hydrus/releases/download/v393/Hydrus.Network.393.-.Windows.-.Extract.only.zip

And extract it to your desktop, does that completely bare and new client run?

It looks like >>13960 is Win 10. What OSes are you on, and is there anything special about your OS? Is it behind on many updates, or have an unusual .NET or Visual Studio Redist situation, or is it Windows 10 'N'?

Could you have an anti-virus that quarantined one of the hydrus dll files?

I recommend you roll back to 392 for now.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

951b56  No.13982

>>13962

Yeah, this >>13969 is the trick here. See what json they are pulling with their js–it usually is really nicely formatted, and actually easier than parsing html.

On Firefox, right-click and hit Inspect Element, then hit the network tab and hit f5 or scroll down to load the next gallery page or whatever. You'll see it come in as it happens.

There are a few 'api' parsers in a default client install that work like this. Check them out, and their GUGs and URL Classes (which use API transformations) to see how it all works. Let me know if you run into trouble.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

951b56  No.13983

>>13972

Ah, can you post an example URL, so I can have a look?

EDIT: I looked it up:

When I fetch:

https://nyanpals.com/kontent?include=pokemon&exclude=nsfw&allow=&limit=10&method=uploaded&offset=0&order=DESC&token=null

I get some nice JSON like what you have here, but it seems to parse ok. The only problem is that comma-separated tag string. There isn't a way atm to get hydrus to parse that single item into multiple strings, afaik, although I do hope to add this.

An easier first solution is to just get the URLs. If you are unsure how to split that into rows in a hydrus parser, you want to change to a JSON formula and get the 'rows' value, then every item of the subsequent list, and then every 'url' entry of those. It should give you a list of URLs.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

951b56  No.13984

>>13975

I am sorry for this trouble. There aren't excellent tools yet to change what parents and siblings apply where. I want to change this in the near future, allowing you to set per-service rules and sibling/parent application.

For now, the only option is in options->tags, uncheck 'apply all parents for all services'. This should keep your parents separate, but I do not think it is perfect. Let me know where it isn't perfect, and I will fix it.

I accidentally made the new tag-removing suggestion boxes not refresh correctly on new media. I will fix this. Once it is fixed and you give it a go, please let me know if you would still like the option to have the tags no longer remove. The ideal here is that all the lists will constantly synchronise to the current media so they never show a tag that is already 'current'.

Tell me about it. Sometimes doing this I have to say the weirdest shit over and over.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

951b56  No.13985

>>13976

Windows doesn't like it when you DnD between programs of different privilege levels. The most common example is one program running as admin and another not. I understand there are more subtle levels that can apply as well, but I do not know the details. Could that be the problem? It sounds like UAC is asking about admin rights there–is hydrus now trying to write its db to Program Files or a similar protected location?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9  No.13988

>>13983

>An easier first solution is to just get the URLs.

That works, but it means I can't get the tags since the URLs link directly to the images, which is the entire reason why I wanted to write a parser.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

64349a  No.13990

>>13985

That must have been it. I did have it installed in program files x86 just now. Reinstalled somewhere else and now DnD works. I guess i forgot where I had it installed last time.

Thanks for the assist.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

951b56  No.13991

>>13988

Right. Unfortunately I don't have a 'split one text into multiple' lego block ready to slot into the pipeline yet. The current string transformer is strictly 1-to-1.

I've just had a look at the code to check. I currently have:

for every text parsed by the formula:

test against string match

convert using string transform

I could try to slot in a new 1-to-n 'string splitter' object before the transform, but perhaps the real answer here is to also make a 1-to-n 'string processing' container object that can hold m match, converter, and splitter objects in any order you want. Then if you want to test a match after conversion you can, or after splitting, whatever you want. I could spam that processing object in several other places, along with the splitter.

I will have to think about this. This specific problem of string splitting keeps coming up. I think it might be worth the extra investment to make a flexible, pipeline-simple system that will solve more problems than try to jam in more hardcoded stuff.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

100de7  No.13992

Are known urls for files supposed to be urls that point to the same exact file (hashes match) or just files that look the same? Like, is it something I would add myself when a come across a file on the web that looks identical to one in my database, or do I just leave adding known urls to the downloaders and such, who can check that the url points to that exact file, and not just one that looks like it?

I wondered this when I saw the known url merging options on the duplicates processing page. I thought that it didn't make sense to ever merge known urls because they're different files, but then that got me wondering if known urls just need to point to files that are "close enough" to be okay. Of course, if that's the case, then that might cause trouble with downloaders skipping downloads of files from urls that are already known urls, especially if the file that would've been downloaded was a higher quality version.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

17c2e1  No.13993

Is there a GUI setting in 393 i need to tick to fix this error? Having the error on tbib and danbooru so far. Also I migrated my image db and upgraded like 50 versions, and I notice one of my older parsers transferred to. Maybe my parsers are outdated? (srry if that doesn't make sense, not 100% on this stuff yet). Here's the error anywho:

Looks like an unsolvable CloudFlare issue: Cloudflare reCaptcha detected, unfortunately you haven't loaded an anti reCaptcha provider correctly via the 'recaptcha' parameter.… (Copy note to see full error)

Traceback (most recent call last):

cloudscraper.exceptions.CloudflareReCaptchaProvider: Cloudflare reCaptcha detected, unfortunately you haven't loaded an anti reCaptcha provider correctly via the 'recaptcha' parameter.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

hydrus.HydrusExceptions.CloudFlareException: Looks like an unsolvable CloudFlare issue: Cloudflare reCaptcha detected, unfortunately you haven't loaded an anti reCaptcha provider correctly via the 'recaptcha' parameter.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

566336  No.13998

>>13993

I should mention my whole comp routes over tor and that hydrus is running via torify

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0b3892  No.14000

File: 8eddfb5283fe190⋯.jpg (254.98 KB, 1200x1200, 1:1, Romantic_vampire_eb2bfa_74….jpg)

Hello guys ! Newbie here, I'm reading the help but I'd like confirmation if I have understood everything correctly: everytime you move an item to the db (drop&load or manual import) hydrus is in fact creating a copy of your item and a thumbnail. Those newly creating files are stored somewhere in the program both in form of hash, thus allowing it to quickly access the data and the thumbnail but not really to access it easily by the user using your regular framework (like going into windows file explorer) ?

If this is correct, can someone tells me how much datas he generated for one item ? I would say it roughly doubles it but I'm not sure. I'm asking because I have a fairly large amount of files that I would like to process but not being able to allowed that much disk space I'd like to estimate the total amount it would required.

I'm of course thinking of investing in some new storages and I'd like to know if someone ever tried to store their datas onto usb drives like it is mentionned in the help. I always have my doubts about that method because of how "temporary" is the nature of those but this seems like a good way to stock new items.

Thanks everyones and special thanks to the dev, this stuff is a dream comes true.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

db4114  No.14002

Is it possible to assign extra mouse buttons as shortcut triggers somehow? The GUI doesn't seem to recognize them.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

23788d  No.14003

>>14000

The files are just copied so yeah double, plus a little bit extra for info about them stored in the database but this is minuscule.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

692d3a  No.14005

File: 54a6c1dab7ef451⋯.png (78.34 KB, 793x667, 793:667, Screenshot_from_2020_04_24….png)

Hello. I apologize if this is a very simple problem but I can't manage to figure it out on my own.

I want to use hydrus on multiple computers and I want them all to be synced together. The way to do it is to use file/tag repositories, correct?

I can't get it to work though, I've set up a server as it's detailed in the help pages and then added a file repository service to it, added an account to the file repository, uploaded a couple of files to the remote service then I added that repository on another device, yet am not seeing any of the files on my other device.

What am I doing wrong?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

572163  No.14101

>>13992

Ideally, it is for exact hash matches.

The internet being what it is, plenty of sites point to other referral URLs that have resizes, or CloudFlare optimises a file in transit for one site but not another, so sometimes two files in hydrus get the same known url, or one file gets two different urls for the same site, but everything works best when each file url only has one file associated with it. When there are url merges or dupes, hydrus has downloader logic to recognise this and ignore that data when it does the 'have I already seen this file?' check.

The duplicate known url merge is not recommended for general use. It works for weird one-off jobs, usually based on user workflow preference where they want to have every possible url available to click in the media viewer, but every time it is done, it weakens the 'already in db' logic and ends up wasting html or file bandwidth. I wouldn't have added it, but urls are always a bit of a clusterfuck, and if you have time to put in to do custom dupe merges or have a weird local network proxy redirect downloader setup that mirrors some crazy urls to the wrong place, it can make sense.

Normal users shouldn't touch it, I think. And in general, I think I'd say leave known urls management to the downloaders. You can add a missing one if you want, if you discover an important one that somehow got skipped, but only if you have the time to get that finicky.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

572163  No.14103

>>13993

>>13998

This is a very new (393) system. Hydrus now has the ability to recognise CloudFlare challenge pages. For simple challenge pages, hydrus can now do the five second wait you sometimes see on your browser. For difficult captcha challenges (which you are likely seeing with TOR, and sites that are under active attack show to everyone), hydrus cannot currently do it and now registers a CF-appropriate error. There is an extension to the new 'cloudscraper' module to add a captcha solver service, which it is complaining about in your error, which I may do.

Did this site work for you in the old version of hydrus? My belief is that it shouldn't–hydrus would have hit the CF page and just recorded a 503 error.

I will update that error message to be less technical, thank you for posting it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

572163  No.14105

>>14000

Hey, yeah, >>14003 is correct. Hydrus owns its files and stores them in its own file structure. It only needs that one copy of the file, but that file cannot be nicely seen with Windows Explorer, so if you still want to view a folder of files with nice filenames, you will want to keep a copy of the original folder after you import to hydrus.

Your situation is common to new users. As I say a bit in the help, I recommend you keep using hydrus for a few weeks with a thousand or so files, and don't delete any of your old files and folders. Most users who like hydrus figure out which files work for them in hydrus (usually booru files are good, but paged comics are currently bad) and find they do not want to keep the nicely named folder-based originals and delete them.

As for extra metadata, hydrus generates a thumbnail for each file, which usually ends up being about I think 1.6% of total file storage space, and the database itself, which if you do not sync with the PTR will start out a few MB and even with a large collection will normally be less than 500MB. If you do sync with the PTR, it will be 25GB or so. If you are not on an SSD, do not sync with the PTR.

For any situation with 100,000+ typical imageboard and booru files, the total effective hydrus size is basically 97% taken up by file size. There isn't all that much bloat.

If you are interested, there's a little more on my thinking about storage stuff here:

https://hydrusnetwork.github.io/hydrus/help/faq.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

572163  No.14106

>>14002

Not yet. I am slowly converting my five different old hardcoded shortcut systems to the unified system under file->shortcuts. Every now and then, I extend it and add mouse hooks for a new window, but support is limited atm. It is currently mostly the media viewer and its related filters I think.

That said, are your extra mouse buttons like OS-level buttons, like 'back' and 'forward'? Now I think of it, I don't think I recognise these yet, and I bet Qt has some nice support for it. I will look into this. It would help if you can list all the non-standard buttons you mouse can do.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0ca948  No.14111

File: dab3ed9b4c0104e⋯.png (5.37 KB, 302x155, 302:155, ClipboardImage.png)

1) How do I give filenames to my images?

2) Is there a way to make the letters bigger? (See pic related)

3) How do I favorite an image?

4) How do I use the collection feature for doujins/comics/sequential images?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b5d17a  No.14114

Is there a reason why Hydrus doesn't support importing Word documents, spreadsheets, PowerPoints, etc.? I really like the idea of getting away from folders, but right now, I have to turn my documents into PDFs to import them.

I assumed it was because these files are essentially made to be edited, and that editing them would mess things up for Hydrus, but I found out today that it does import Photoshop files just fine, though I haven't tried editing them yet.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

674e43  No.14133

>>14103

Close–the site didnt work in old hydrus and actually showed 403 forbidden instead of 504. I'm looking at a proxy to use inside hydrus to hide my tor usage. Any thoughts on services b4 i throw money at this? Thanks again>>14103

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8  No.14156

>>13927

file > options > files and trash > remove files from view when they are… uncheck both boxes for now. next help > help and getting started guide > getting started with files > filtering which streamlines the workflow for archive vs delete processing and functions basically as >>13949 describe here only with different keys: your A maps to delete, your L to spacebar, and your Y to backspace, plus there's one more for skipping any files you may wanna leave inboxed.

>>13975

do you typically set up parents via the top menu (services > manage tag parents) or the context menu (right click > add parents to tag)?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8  No.14157

File: bdb5a343571141b⋯.gif (96.12 KB, 1177x829, 1177:829, string_break.gif)

>>13988

>>13991

>flexible, pipeline-simple system

pre-parsing conversion maybe? dunno json well enough to get much further than this but picrelated. might could work if entered in at the level of a sub-subsidiary page parser.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8  No.14158

>>14111

1) manually type them in prefixed with filename: or click import with tags when you drag files in from a folder

2)see >>13941

3)tag it with favorite or use ratings (services > manage services > add > local rating system)

4)my method is to give each page a unique page:# tag and every page the same title:whatever tag, then set hydrus to collect by title

5)read the help pages m8, particularly advanced usage - general and getting started with ratings, 3/4 of your questions are covered there and explained thoroughly

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3  No.14161

How do I download an entire pixiv of an Artist with Hydrus? Also how does it sort images that are in sets?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3  No.14162

>>14161

I'm in URL Import tab, and pasted in a pixiv link pressed enter and it didn't do anything says "Looks like HTML – maybe the client needs to be taught how to parse this?"

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3  No.14163

>>14162

>>14161

nvm found out, for pixiv I had to install some extension API thing. Hydrus seems very technical, and I've just started using it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3  No.14164

File: 39417cd4bc2f762⋯.png (56.6 KB, 445x1028, 445:1028, ClipboardImage.png)

How do I edit the tags that get automaically fetched into the gallery? For example it fetches "creator:", how do I make it so it automaically changes this to "Artist:" instead? Also some tags such as "long hair", "medium long hair", "short hair", I want to change it too automaically become just "hair" as well. Would be greatful, the Help guide is quite complex for me

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

de61ca  No.14165

>>14164

Such changes require you to edit the parser itself locally. changing the creator namespace to artist is quite simple and only requires a single thing to be changed. as for the hair stuff. more tricky. could be done with a regex replacement prob on the tag content parser.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3  No.14166

>>14165

How do I edit the parser locally? I've also no experience with regex expressions

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3  No.14167

>>14166

okay found out how to change the parser tags using siblings. But still don't know how to change creators to Artists.

Also have a big problem of downloading from pixiv all the tags are in japanese and I'd rather have the Gelbooru and Danbooru tags instead. But I'd have to download from there instead of pixiv which usually has oredered images and source quality.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

de61ca  No.14169

>>14166

really is best to join the Discord for that sort of thing as it will prob require a large amount of handholding.

But you can also read the docs on how to do it. :)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6be94a  No.14170

So i got hydrus companion, when i go on the pixiv website i click on send cookies from this site to hydrus but i still cant have it parse the pixiv images. Can someone tell my retarded ass what im doing wrong?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ec51e8  No.14171

File: f10357834673048⋯.gif (1.66 MB, 1292x875, 1292:875, tag_namespace_change_in_pa….gif)

>>14166

>>14164

>>14167

here lad. best advice i could give you is to read the help guide start to finish so it gets into your head even if you don't fully understand how it all works together yet. don't worry about being able to relate everything in there to what it actually does in the program, just make sense of how the words relate to each other and the info will file itself away for later and make it easier to learn the program just by poking about and seeing what happens.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291  No.14179

Is there a way to sort files so that alternates are grouped together?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291  No.14180

Is the favorite tags window not sorting properly now for anyone else? After I updated to the latest version all the tags were jumbled up, instead of sorting by namespace and alphabetically.

>>14156

Late response but I typically use the context menu to add parent tags

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8adcb7  No.14181

I'm sure this is in the help somewhere I can't find, but is it possible to add page tags without going to each image and just plugging away?

Some of these longer paged things are killing me.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

982291  No.14182

>>14181

If you're adding images that have the page as the filename then you can use that with a page namespace as you import. Can use regex commands if theres more than just the page in the filename too.

Other than that though im not sure of any other methods other than just doing it manually.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c2b97f  No.14183

>>14181

If you have or get the hydrus companion browser extension, you can go to utilities and use the sequential image tagger. you just put in the image hashes in order 1 per line and it will sequentially tag them. So if you have them in order in hydrus you can just select them all, copy the hashes and paste them in.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

c2b97f  No.14184

>>14170

Doesn't sound like you are doing anything wrong, You set it up with your API key and have cookie permissions on that key right?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8b2b0d  No.14186

$$\{red}{redtex}$$

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

11beeb  No.14187

Redtext

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

541472  No.14188

""Bold""RED BOLD

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

541472  No.14189

We are researchers who deal in open-source information, reasoned argument, and dank memes. We do battle in the sphere of ideas and ideas only. We neither need nor condone the use of force in our work here.

VINCIT OMNIA VERITAS

SEMPER FIDELIS

WWG1WGA

Q Proofs & Welcome

Welcome to Q Research (README FIRST, THEN PROCEED TO LURK)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

11beeb  No.14190

"VINCIT OMNIA VERITAS"

""SEMPER FIDELIS""

$$WWG1WGA$$

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

11beeb  No.14191

"VINCIT OMNIA VERITAS"

""SEMPER FIDELIS""

$$WWG1WGA$$

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

11beeb  No.14192

$$\colorblueblurtext$$

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8b2b0d  No.14193

$$\color{blue}{blurtext}$$

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3  No.14194

Has anyone else used imgbrd-grabber before? How does it compare to Hydrus?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

de61ca  No.14195

>>14194

Hydrus is different. It downloads all that grabber does for sure with its tags.

It is easier to edit and add support for new sites on Hydrus and you have more control in general across the board.

But Hydrus is much more than just a simpler download client.

Its a full library manager with a ton of features for those types of users.

You won't find any better than Hydrus for booru/internet content of its kind.

Please feel free to jump on the Discord if you want a more direct convo about what Hydrus can do for you.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

100de7  No.14198

When using hydrus companion (on firefox, if that matters) to download from Pixiv, how do you get it to download a specific file from a post, rather than the entire post. If I go to the direct file url, (the i.pximg.net one) it just returns a 403. With post urls this doesn't happen, so I know it's not a login problem.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e98755  No.14201

>>14198

the direct file url needs a referer header of pixiv.net. In hydrus you can make a pximg.net url class and add a referer of pixiv.net to it or just add it to the http headers without making a url class.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

100de7  No.14202

>>14201

Thanks. I knew nothing about how url classes worked before this, but after banging my head into the wall until it worked reading about it on the help pages to figure it out, I managed to make one after a few tries, and the downloading works now.

I feel like a url class for pximg is something that should be in the defaults. Is there somewhere that you're supposed to submit things like this to get added to the defaults, or at least a third party repo for people to download it themselves?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e98755  No.14203

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64  No.14205

Is PTR up and running?

I tried adding the service as mentioned in the help, but testing connection to ptr.hydrus.network gives an error.

Back then before the big change in PTR it worked for me, then I disabled the sync once the owner changed and now I'd like to enable it again.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ed6436  No.14207

>>14205

Yes it works. Rather than setting it up manually it's easier to just go to "help > add the public tag repository". Try that.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8adcb7  No.14211

>>14182

This sounds like a a good enough way to do it, thanks

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

35f78b  No.14213

I noticed that "Source Time" information is included when pictures are downloaded from galleries, so the parser does know how to read and store that information, but there doesn't seem to be a system tag to search by source time, nor a sorting method in pages.

Is it at all possible to search by source time? I think it would be cool to look at various artists or franchise works from a chronological point of view (or at least as close as the boorus get to that)

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64  No.14215

>>14207

I did exactly that but still testing the connection gives an error and no progress is being done in syncing.

Also, do I have to remove the old PTR service, that points to the non-existent server now, or I'll lose all information associated with it?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

58f0a3  No.14218

How does running Hydrus on an SSD (2TB) compared to a hard drive? Will it make any difference when you have 10000s of images?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9e7558  No.14220

>>14218

Just having the database on an ssd will make it significantly faster.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7b65e6  No.14222

Is anyone using Hydrus Companion on Vivaldi 3.0? It was working but after the update the "send for tags then send to hydrus" context menu option does not display the prompt to input tags.

Any tips on how to fix?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e5852a  No.14225

File: e6aec069ef9c36a⋯.png (24.64 KB, 563x412, 563:412, ClipboardImage.png)

Expired links

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

e5852a  No.14227

File: 3fe88c2972dbd44⋯.png (101.59 KB, 680x564, 170:141, ClipboardImage.png)

>>14220

>a SSD is strongly recommended

>when it's obvious you would use a HDD for huge picture stocking

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

cd48ee  No.14245

>>14244

Meanwhile I have 10TB HDDs

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ade59b  No.14256

>>14114

Hydrus file import and maintenance is currently predicated on the idea that files' filetype can be determined from file content (not filename). As it is generally more difficult to determine a .doc is a .doc compared to recognising a jpeg or an mp4, these types have been slower to add. This is also the reason why I do not yet have a 'any other file' filetype.

Ultimately, to get these files supported, hydrus either has to recognise them a bit better, or I need to improve the maintenance and import code to deal with more complicated and less certain situations. I haven't looked too closely, so perhaps most .docs have a nice file header I can recognise, and I know some open source document formats are really just xml that I can ultimately parse and pull the right tag from, but some, like .txt, are going to be more tricky. Mostly it has just not been too high a priority yet, so I haven't invested the time–I am not opposed to it.

I recommend you not edit files stored in hydrus! Another axiom of the hydrus file storage system is that files are read only, and changing file content of existing files could give you a couple of recoverable but annoying problems down the line. Do an export->edit->reimport cycle if you want to edit.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ade59b  No.14257

>>14133

I am not sure. I am afraid I do not have much experience in much of this. I'm still improving hydrus/cloudscraper integration, so I hope there will be better/more tools in future.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ade59b  No.14258

>>14179

Not yet. That will likely have to wait for the next big iteration of the 'duplicates' system, which will expand alternates into a more fleshed out 'file relationships' system. In that step, I will have thumbs load with their dupes and other file relationships so UI stuff can access it for collecting and sorting. At the moment, thumbs don't know this info, so it is actually secretly fetched from the db every time you right-click on a thumb!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ade59b  No.14259

>>14180

Sorry, I have fucked this up about three weeks in a row now. Everything is sorted correct in 396 except favourites. That'll be fixed next week.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ade59b  No.14260

>>14215

Some users in Russia can't access ptr.hydrus.network. Some DNS ip range ban bullshit iirc.

If you remove the PTR service from your client, yes, you will remove all the mappings. The answer here is to change the settings in manage services to something that works. Can you load https://ptr.hydrus.network:45871 in your web browser? It should make a welcome page with an ASCII lady.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ade59b  No.14261

>>14213

Not yet, but I hope to integrate more of that data into the 'modified date' row that files now have. At the moment, source time is not recorded.

>>14225

Thanks. Some weird dude got upset and started spamming TOS-violation stuff to the server along with some friends, so the old invite url got turned off. Assuming the users who run the discord server are ok at that point about putting a new invite up, I'll fold the new on into 397 and all my help links.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ade59b  No.14262

>>14218

>>14220

>>14227

>>14244

>>14245

The main deal here to get great performance is to put the hydrus 'db' (i.e. the four client.xxx.db files) and your thumbnails on your small expensive fast SSD and your media files on the large cheap slow HDD. DB and thumbs benefit from low latency, jpegs and webms do not.

Doing this takes a small amount of work, but if you have an SSD, I strongly recommend ultimately moving to this setup when your client gets big. Here is the help page on this:

https://hydrusnetwork.github.io/hydrus/help/database_migration.html

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9  No.14264

File: aa45824d6ed87c2⋯.png (15.97 KB, 936x392, 117:49, ClipboardImage.png)

Was the way namespace searches work changed recently? I used to be able to do namespace:* to see all the tags that belonged to that namespace, now it only shows with at least three letters or two and a *.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

60ca44  No.14265

File: d1f92024439e1a3⋯.png (825.63 KB, 1075x1518, 1075:1518, overwhelming_sex_knowledge.png)

I haven't used Hydrus in a while. What did Pixiv fuck up this time and how to I get Hydrus to download lewd images from there again?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

3e1190  No.14266

>>14265

They added captcha to login so you have to add cookies manually or using hydrus companion (https://gitgud.io/prkc/hydrus-companion) to login.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8f1ccb  No.14268

>>14256

That makes sense, thanks!

Speaking of filetypes, is duplicates processing for gifs, videos and other filetypes on the horizon?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8f1ccb  No.14269

Can you add support for Mozilla Archive Format (MAFF) files?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ce9f64  No.14272

>>14258

As a cheap and dirty replacement for now you could implement sorting by the thumbnail filesize, it will group similar images together, since when you make all images the same dimensions and compression level (which is what thumbnail generation does), the file size becomes a function of content.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

69ed8b  No.14275

I'm using version 392 right now. Are there any hidden factors that lead to subscriptions stopping early? (besides the "i've seen these links before" system).

Am I right in assuming that the "on normal checks, get at most this many newer files" is a per-query and not per-subscription limit?

I'm working with a decent number of queries right now, with the intention of having them checked about once per week. I force-checked everything today, and after I set them going, the sub looked through a bunch of queries, then just stopped entirely. I've had some queries stop at like 70/100, some don't even start and are like 0/31.

there aren't any other ongoing simultaneous download or import jobs, no bandwidth limits are being exceeded. The popup just disappears, checking current network jobs shows a blank screen, and going to the subscriptions page shows that my files are for example items: 945/1224, nothing in the "recent delays / error" column.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

94ce1a  No.14276

My sort is currently set to "artist-series-comic-chapter-page", but while the pages sort in descending order, chapters sort in ascending order. Is there a way to make both sort in descending order?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

32bafb  No.14277

File: f6bcef2dcbc5cf4⋯.png (137.02 KB, 869x959, 869:959, 123.PNG)

Am I a big dumb, or are there no default downloader for newgrounds art galleries? There's only ones available for games and movies galleries.

I tried making one myself but it didn't get all the urls, I think due to the fact that the newgrounds art gallery page doesn't load the whole gallery at once and requires you to scroll down to load the rest.

https://matt-likes-swords.newgrounds.com/art/

Notice that it doesn't load past about halfway through the 2016 submissions until you scroll down there, and as a result doesn't grab urls beyond that point. Pic related shows the test url parse results, and at the bottom is one of the urls that is beyond the loading point - it doesn't appear in the test results. It seems making the parser deal with that is beyond the scope of what I can figure out just from clicking around.

Are there premade downloader definitions for newgrounds art galleries already made? If not, is there a way around this issue?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

21d745  No.14281

Anyone ever have a tag search that stops working? Gelbooru one worked fine yesterday but now just results to "done" without fetching any files. Is there a new downloader?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1fae9  No.14282

File: 1e88bb7bb7be30b⋯.jpg (41.83 KB, 640x626, 320:313, selling_deer.jpg)

>>12641

How do I set my database backup to ipfs?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

60ca44  No.14284

>>14266

I did that and it said that the cookies were successfully added, but I still only get safe images when I do an artist search in Hydrus. If I do "send this tab to Hydrus" it can download lewds though.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6bc377  No.14285

File: ba508d1f9cfec6f⋯.jpg (503.28 KB, 748x800, 187:200, ba508d1f9cfec6fe795eb02e28….jpg)

I have a kind of specific question: How can you delete a file including all of it's potential duplicates from the trash?

I did download using my own scripts a couple of years back and … well I managed to get about 4 million files, no idea how many of them I still want.

I imported them into hydrus because I wanted to actually look at them - tagging, the PTR, filtering and duplicate processing are a real godsend.

So now I can (thanks to the PTR and scraped booru tags) finally search and delete everything I do not want to keep. The only issue is that the PTR will not tag all of the resizes and thumbnails and whatever else I managed to download.

Duplicates processing usually finds those files, but I can't really delete them, because they are not tagged.

I *could* go to duplicates and search for files in trash, and that *would* find all the files (even ones with only one duplicate in the trash), but they are not displayed on the media panel so I can't really delete them all at once.

I have to hit "launch the filter", then go through each pair hitting delete on the keyboard, then hit "both files" in the dialog box and finally hit "permanently delete".

Is there any chance I can build a query for that? Otherwise, I will just work something out with xdotool and the "show a random pair" button.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

100de7  No.14286

Is there a way to search for files that have an identical known url, to make sure that I'm not accidentally putting them on the wrong files?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b5d17a  No.14288

>>14286

Yes: system:known url

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9  No.14290

>>14281

I'm getting the same error. The gallery parser stopped working for some reason, maybe gelbooru changed something.

At first I thought I had been blocked from running 3 simultaneous gelbooru subscriptions.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

41b3f9  No.14291

File: 52e1c18e561b303⋯.png (4.68 KB, 512x115, 512:115, easy_import_downloader_png….png)

>>14290

Turns out it was just a simple retarded change. They changed the thumbnail class from "thumb" to "thumbnail-preview" for no reason.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab  No.14295

>>14264

Yes. I cleaned up the pipeline here and removed it, as it was always a hack that could cause problems for some users. I hope to add in an option this week to return it if you want, and an option to allow 'ser' to match 'series:metroid' and all other 'series' tags.

>>14268

Not right now. I am currently recovering from a huge conversion of the UI library from wx to Qt. tl;dr is: all UI is much better now, but I have a thousand small things to do before I can focus on a big expansion of anything.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab  No.14296

>>14269

Can you email me (hydrus.admin@gmail.com) an example of one, or just point me to one here? If the file has an easy reliable file header, I can recognise it and add it sooner rather than later.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab  No.14297

>>14272

That's an interesting thought. I don't easily 'know' thumbnail size atm, but I do know filesize. The 'sort by bitrate' does size / num_pixels for static images, but I'm not really sure if that is really similar to what you are going for.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab  No.14298

>>14275

If you are getting 70/100 kind of numbers, that sounds like bandwidth limits to me. The subs are getting the files they are supposed toand yeah, the 'get this many files' options are per querybut they are quitting those queues early. Subs can stop early for a couple of odd things, like pausing network traffic or opening the manage subs dialog, but if they are doing it on their own, it is probably due to bandwidth running out for that context or, I think, five file errors on that query happening in one run.

If you catch a sub stopping early, open up manage subscriptions and look at it. It should have a 'recent delays?' column that'll talk a bit more about why it stopped and when it expects to continue.

SHIT, EDIT: I just read the rest of your post. Hmm, I am not sure. If you hit help->debug->report modes->subscription report mode and then tell a sub to run, it will spam a whole bunch of popups on its decisions, also writing them to the log in install_dir/db. This may help us figure this out.

It could be a weird bandwidth rule, maybe something like a 15s or 30s delay somewhere, where the sub is saying 'oh, I should stop', and then the delay times out real quick, but some other hardcoded delay isn't requeueing it for ten or twenty minutes.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab  No.14299

>>14276

Hmm, they are both, I think, supposed to sort in ascending order, e.g. 1, 2, 3. Exactly so you can read comics in order. If you go into options->sort/collect and add a new namespace sort just for 'chapter-page', does that work correct? Do the files have multiple series/creator? Some of this stuff is very old tech, and it doesn't work well for those situations.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab  No.14300

>>14277

Newgrounds has always had spotty support. Their galleries and embeds are weird and sometimes change. You might be in luck, because I see a new one here:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/blob/master/Download%20System/All-in-Ones/Single-Sites/newgrounds-2020-04-07l.png

It says it does art, but I know nothing about it. If it works well, let me know and I'll update the hydrus defaults.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab  No.14301

>>14281

>>14290

>>14291

Thanks mate, I'll fold that into 398.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab  No.14302

>>14282

There is no in-hydrus way to do this atm the moment. You would have to figure out a backup regime that somehow pinned your db or its media files to your ipfs daemon.

I may be misunderstanding you. Your db files are very private, and I would not recommend trying to expose it to a public network like IPFS. But if you want to put your media files, your jpegs and so on, in IPFS, then hydrus has limited support for advanced users. I would shy from calling it a backup, though. Try adding an IPFS service under services->manage services and getting it to talk to your daemon. Then you should be able to pin files using thumbnail right-click->manage->remote services.

Be warned that the new IPFS 5.0 was broken for hydrus until I updated my code in the recent 397. It may have patchy support, and has always been experimental.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9c43ab