[ / / / / / / / / / / / / / ] [ dir / random / 93 / biohzrd / hkacade / hkpnd / tct / utd / uy / yebalnia ]

/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Catalog  Archive

Name
Email
Subject
REC
STOP
Comment *
File
Password (Randomized for file and post deletion; you may also set your own.)
Archive
* = required field[▶Show post options & limits]
Confused? See the FAQ.
Embed
(replaces files and can be used instead)
Options

Allowed file types:jpg, jpeg, gif, png, webp,webm, mp4, mov, swf, pdf
Max filesize is16 MB.
Max image dimensions are15000 x15000.
You may upload5 per post.


This board will be deleted next Wednesday. I am moving to a General on 8chan.moe /t/. This board is archived at 8chan.moe /hydrus/!

File: 214bc076862edc6⋯.gif (102.07 KB,450x450,1:1,214bc076862edc61448ea65b72….gif)

37a3b4 No.11953 [Open thread]

I had a mixed week, but I got some good hydrus work done. I added some neat x/y timestamps to the animation scanbar, made a way to launch file urls with a shortcut, and added barebones .psd import support.

The release should be as normal tomorrow, but perhaps a little late if some IRL stuff comes up.

2 posts omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6d7e10 No.11967

>>11962

>I wrote this specification of the XCF format in mid-2006. Later it got included in the official Gimp sources, and unless you're explicitly looking for historic data, you should go there and get the specification instead.

Try here instead

https://gitlab.gnome.org/GNOME/gimp/raw/master/devel-docs/xcf.txt

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

284ef1 No.11969

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

284ef1 No.11970

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

0e1e8c No.11972

File: 79d8445d2b4039a⋯.png (502.41 KB,1100x587,1100:587,79d8445d2b4039ad72ee107763….png)

There may not be a formal specification for the .kra format it looks like, so source code might be your best bet, or just reversing it yourself as it's just zipped up junk.

Here's the Krita stuff if it's useful:

https://github.com/KDE/krita/blob/v4.1.8/plugins/impex/kra/kra_converter.cpp#L56

https://github.com/KDE/krita/blob/v4.1.8/plugins/impex/libkra/kis_kra_loader.cpp#L196

And here's GIMP's XCF loader code which may be of use if the specification is ever unclear:

https://github.com/GNOME/gimp/blob/GIMP_2_10_8/app/xcf/xcf-load.c#L152

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

37a3b4 No.11973

>>11969

>>11970

>>11972

Thanks lads. I'll queue these up and I'll see what I can figure out. Adding 'I can recognise it at least' .psd support wasn't a massive hassle this week, so I expect to do more of this sort of work. But I suspect I'll have to refigure some UI like the system:mime panel checkbox hell if we go too nuts here.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



YouTube embed. Click thumbnail to play.

44b5d1 No.11863 [Open thread]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v343/Hydrus.Network.343.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v343/Hydrus.Network.343.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v343/Hydrus.Network.343.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v343/Hydrus.Network.343.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v343.tar.gz

I had an ok week. The Client API v1.0 is essentially finished, and I did some code cleanup as well.

client api

The Client API now has full 'file search' capability, including fetching file metadata like size and resolution and tags, and also file and thumbnail fetch. As before, the help for these new commands is here:

https://hydrusnetwork.github.io/hydrus/help/client_api.html

But for some tiny last cleanup jobs, the Client API v1.0 is now complete. From now on, I will fitPost too long. Click here to view the full text.

27 posts and 5 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

03fbe2 No.11955

File: 067c964e23215e2⋯.jpg (97.36 KB,640x641,640:641,f7215076-af0f-442c-a17b-53….jpg)

>>11947

I agree. at the moment im only testing on lan. I have no problem waiting for https support before releasing. By the way I had a quick question on the file metada endpoint. The example on the site shows the following bit of json for tags :

"service_names_to_statuses_to_tags" : {

"local tags" : {

"0" : [ "blonde hair", "blue eyes", "looking at viewer" ]

"1" : [ "bodysuit" ]

}

}

I was wondering, why are the local tags split up into two separate arrays? Why is bodysuit separate from the other tags? Why not have a single array like so :

"service_names_to_statuses_to_tags" : {

"local tags" : [ "blonde hair", "blue eyes", "looking at viewer", "bodysuit" ]

}

Should I expect any number of arrays or is it always 2?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

53d6c0 No.11956

>>11947

Yeah, encryption would be pretty important. I personally don't care because lol openvpn server at home, but that's not normal.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

44b5d1 No.11964

>>11955

The numbers there are the 'status' of that list of tags. In hydrus, a file's tags can have four different statuses:

0 - current - the file has the tag for this service

1 - pending - this tag is pending to the service

2 - deleted - this tag was current but has now been deleted

3 - petitioned - this tag is current and is petitioned for deletion

Pending/petitioned relates to the 'pending' menu in the main hydrus client gui. Repository tags are not fixed as current/deleted until pending/petitioned are commited through that menu.

In writing this out, I realise I fucked up here because a 'local tags' service won't ever have 'pending' tags. I will rewrite this help section to have it refer to a hypothetical tag repository.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
Post last edited at

44b5d1 No.11965

>>11964

To follow up here, if you wish to edit a file's tags with /add_tags/add_tags, you will need to use this more detailed tag status information to figure out if you are 'petitioning' or 'rescind pending' or whatever.

For practical purposes, if you want to calculate what tags the file currently 'has' for easy phone display, you can go:

tags = ( current ∪ pending ) - petitioned

Petitioned tags are pretty rare, so you can probably just go ( current ∪ pending ). You'll then be unioning those sets across all given service_names to get a file's 'all known tags' as you would see in the client.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

18ab7c No.11971

>>11964

>>11965

Ah. I forgot tags can have statuses. Thanks for clarifying

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: 4ef4a595735e470⋯.gif (1.51 MB,500x375,4:3,4ef4a595735e4708935b207a7d….gif)

a375ce No.11858 [Open thread]

I had an ok week. The Client API v1.0 is just about finished, and as hoped I did some long-overdue code cleaning. Also are some tiff/webm detection fixes and slightly smoother media viewer browsing.

The release should be as normal tomorrow.

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.


YouTube embed. Click thumbnail to play.

47bce6 No.11805 [Open thread]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v342/Hydrus.Network.342.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v342/Hydrus.Network.342.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v342/Hydrus.Network.342.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v342/Hydrus.Network.342.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v342.tar.gz

I had a good week. There are several new ui features and bug fixes, and webp and tiff now have basic support.

tiff and webp

Tiff files are now supported! Tiff is an old format typically used for photographic and scientific uses (think 12,000x12,000px images of the moon). They'll import to hydrus like any other image and will work on the duplicate file system. I tested 24bit and 8bit (monochrome), but could not find a 48bit example–even the easily accessible NASA stuff was all 24bit–so I don't know if high colour-depth tiffs work. If you have an example of one, please send it to me, and if it doesn't work in hydrus, I'll see if I can figure it out. EDIT: Some tiffs are not supportPost too long. Click here to view the full text.

25 posts and 5 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

47bce6 No.11853

>>11838

A user helpfully pointed me to some here:

http://www.brucelindbloom.com/index.html?RGB16Million.html

They render ok, so I guess a clever sRGB conversion is happening. I also added support for 'MM' tiffs for tomorrow. Should be complete tiff support.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

47bce6 No.11854

>>11840

>>11841

If you have a lot of processing still to do, I recommend you do not do explicit processing from that review services frame. It can only be cancelled through the UI, and after a long time, there is a decent chance the UI will deadlock until the job is done (this is due to some shit code by me that will take a bit to clear up). If the job still has 15 hours of work left, the whole program can hang that long.

I recommend you only let processing happen during shutdown, where it has a timer, and idle time, where moving the mouse will tell it to safely cancel.

That review services button is hidden behind advanced mode and is only ever really a pleasant experience when I am doing some testing or when on a fast SSD without much total processing left to do.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

47bce6 No.11855

>>11842

Yeah, for some complicated websites, an import 'item' produces more than one file or a new page to pursue. It is a technical pain in the ass, but for now, the x/y progress on an importer refers to 'import items processed' not 'potential files imported'.

If you would like to check those 'already in db' files, you should be able to right-click on them and say something like 'open these in a new page'. Since in that note it says '10 seconds ago', they are almost certainly duplicates from above. (I don't know anything about nijie.info, but yeah, it looks like the _0 url without diff/main is a 'cover image' for the subsequent mini-gallery?)

Again, some galleries give the same file twice, even on their nice APIs. I don't know why this parser pulls that 'cover', but I did not write it, so I can't confidently say this strategy isn't needed to catch edge cases. The nuts and bolts of this stuff are frequently fuckery duckery doo, particularly on Japanese sites.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

47bce6 No.11856

>>11844

Thanks, that is interesting. I assume the lossless compression mode is functionally 4:4:4, right? But any lossy encoding is coerced to 4:2:0?

My hesitance for webp in the past is that it still is sRGB, so I don't know how it is useful as we move to HDR in the coming years. Maybe they will do like webm and add new accepted encoders, but I dunno. It doesn't seem to be taking the world of internet browsers and phones by storm as it is. HEIF and FLIF seem to have their good points, but they are still similarly meme status for now.

I'll play with animated webps a bit when I find time, as I'd really prefer not to use shitty gifs for animated thumbs, and I don't want to go completely nuts with apngs either.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

47bce6 No.11857

>>11846

Thanks, I am glad you like it. When you get an app ready and want to share it about, please send me a link to it and I'll put it up in my release posts and help!

Current plan for Client API is to get a simple 1.0 out the door. This should be done tomorrow for 243, where basic file search and file/thumbnail retrieval are finished. You'll be able to find all the files in a client with the tags 'blue eyes' and 'blonde hair' and then shows thumbs and files and tags. It should be possible to replicate very basic booru functionality. After the release tomorrow, please check the updated help here:

https://hydrusnetwork.github.io/hydrus/help/client_api.html

Which will have the last '/get_files/…' queries all filled out.

With the 1.0 done, Client API work will then be fit into the regular weekly 'small jobs' schedule. If someone wants the ability to archive files or search by duration, I'll see if I can squeeze it into a small job. If you have ideas for your app, please submit them any way that is convenient–these release threads are fine, and you can also email me or even DM me on discord on Saturday afternoons US time if you want to talk live.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



YouTube embed. Click thumbnail to play.

229563 No.11732 [Open thread]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v341/Hydrus.Network.341.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v341/Hydrus.Network.341.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v341/Hydrus.Network.341.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v341/Hydrus.Network.341.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v341.tar.gz

I had a great week after being ill at the end of the previous week. There is a bit more than usual in this release because of the extra time. The Client API has some new commands, and there are a variety of fixes and stability improvements.

client api

The Client API now does some fun stuff. If you would like to play with one user's browser extension, which recognises hydrus-compatible URLs–including whether you have the page's file already–and lets you send them straight to hydrus for downloading, please check it out here:

https://gitgud.io/prkPost too long. Click here to view the full text.

41 posts and 3 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

229563 No.11827

>>11783

Thanks. I see one of those definitions is unhappy with UPnP. Feel free to delete the three upnpc executables from install_dir/bin–they aren't needed to run the program, only if you want to explictly tell some hydrus service to keep a NAT port open for itself. They are just copies of the exes available here:

http://miniupnp.free.fr/

The others, I suspect, are as I said before. I haven't had a true positive here yet, but I'd appreciate any future information you have here.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

229563 No.11828

>>11785

Thank you, I really appreciate this recommendation and your thoughts about it. I will definitely revisit this when work on hydrus multi-page support starts.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

229563 No.11829

>>11789

Thank you for this report. My dark mode is a bit of a fudge–I only have control over the colour of my custom objects, whereas everything else (like window borders and backgrounds and button colours) is supposed to be inherited from your OS's current system colours. If you have a window manager that changes your OS colours, hydrus's dark mode switching can be helpful to make sure that my custom objects also change with it.

There's a very speculative long-term plan to move from wxPython to Qt. If this ever happens, theming support will get significantly better, but until then it is all duct-tape, I am afraid.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

229563 No.11830

>>11793

Thank you for this report. Are the updates processing during idle time or shutdown? If you have problems during idle time, can you change your settings under options->maintenance and processing to be shutdown-only, and, say, only 5 minutes? If that relatively small amount of work can go through, can a bit more, like 15 mins? Is there a significantly 'cleanup' lag, like 5 mins of work takes 3 mins to commit?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

229563 No.11831

>>11810

Thank you. That is very strange. Here's SQLite's and Python's info on this:

https://sqlite.org/tempfiles.html#temporary_file_storage_locations

https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir

So, for some reason, a different folder was not permitting a program launched from it to access any system temp location. Windows has simpler temp structure than Linux, so perhaps this really just meant stopping you from accessing the one under AppData/Local/Temp.

Could somehow that directory (or say the client.exe) have got a compatibility mode or some other protected state applied to it? I am afraid this is way beyond my expertise.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
Post last edited at



File: e3d80057357881e⋯.gif (826.95 KB,500x275,20:11,e3d80057357881e6faf3fc41e3….gif)

099694 No.11798 [Open thread]

I had a good week. I fixed several bugs (including I think the Linux >0 distance similar files search crash, finally!), did some neat ui work like better tag autocomplete responsivity, more stable and slightly smoother video rendering, and more 'open urls' options on multiple selections, and added some basic webp and tiff file import support.

The release should be as normal tomorrow.

1 post omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

254bfa No.11801

Sauce, devbro?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

099694 No.11803

>>11801

Dunno, sorry! It was just in my inbox this week.

But perhaps source could be your life, one day.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

76ef5c No.11804

>(including I think the Linux >0 distance similar files search crash, finally!)

Was that at all related to any of my posts:

>>11736

>>11758

>>11797

Because I was having an issue on linux crashing. In the process for searching for duplicates an SQL statement similar to

SELECT phash_id, phash, radius, inner_id, outer_id FROM shape_perceptual_hashes NATURAL JOIN shape_vptree WHERE phash_id IN (1, 2, 3);
would be executed and cause a segfault, not just for hydrus but anything that could execute an sqlite statement. Apparently Sqlite 3.27.1 (and possibly some earlier versions) had a bug where if the size of a list exceeded two items it just crashed. This appears to be fixed in sqlite 3.27.2.

I imagine based on the code I looked at having a search distance greater than 0 would make it far less likely to happen if not impossible.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

099694 No.11806

>>11804

There are quite a few places where the client fetches things with lists up to 256 items long, so if you found that just loading some files and tags was ok, I _think_ you weren't getting hit by that exact issue.

But yeah, your posts are the bug I think I fixed. Only affected some Linux users, and I could not reproduce it. I worked one on one in the week with a user who had the crash and we ultimately figured it was a problem with SQLite hanging on to the phash while I did a Hamming Distance search on it (which requires some low level C++ struct calls and then some mickey-mouse CPU-level binary counting). Something to do with memory access violation is my best guess. Python usually can't crash by itself. Previously, I was doing hamming searches interleaved with the sqlite call iterating each row, but when I separated it into completed batches, so sqlite was 'done' with the rows, no crash happened.

The 0 search distance could do a direct lookup without the hamming check, and that was running fine. It just seemed to be the combination of iterator on sql results and then low-level access on 'bytes' type response from that that caused the crash. Core dumps suggested the crash was in sqlite code, so I guess it was still hanging on to a bytes buffer or something that wasn't cleaned up. Could indeed still be a (different) bug in sqlite, or python's wrapper for it, but I dunno for real.

Let me know how 342 works for you!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

6a00e3 No.11808

>>11801

Owarimonogatari

>>11803

wholesome

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: 60e0e08ae58bb15⋯.jpg (334.26 KB,443x575,443:575,60e0e08ae58bb15563f3b41807….jpg)

b1a0af No.11716 [Open thread]

I had a great week after being ill at the end of last week. 341 is a big release because of the extra time. There are several new Client API commands, bug fixes and more memory reduction, and some important stability improvements, particularly for Linux.

The release should be as normal tomorrow.

4 posts omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a9092c No.11721

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

a9092c No.11722

>>11721

Trying to find a kanban for people to put their ideas on the board.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

fbbc03 No.11724

>>11718

the program tends to eat memory when you heavily use it, let me give you a picture of what I mean

I save every thread on 4chan that catches my interest, at some point the reposts start to overtake the new images to a point that this is a viable option, thy im getting close to the edge of I need some delete functionality for telling me what the file was deleted for. that aside, right now, over the coarse of 2 months I have created around 6400 watchers, adding somewhere around 150-200k images, almost none of them are displayed, with around 100k displayed.

the 100k displayed images takes about 1.5-2gb, I know this from the time I culled the watchers, my current session is 4.3gb and bloats out to 6.5gb on saves.

I have absolutely no idea what takes ram, if they were all current watchers, ok, I would believe that, but from experience current watchers for all the threads adds up to around 500mb or less, so for some reason, the program is using an ungodly amount of ram for reasons that are unclear because just storing all of the the data in text form could not possibly add up to over 20mb much less what I estimate at being over 2gb with a doubling in save.

I should also note that I come from a session that was hitting 14gb on startup and when a save happened would bloat to 18+gb

depending on where hdev is focusing, memory reduction has many different ways to happen.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

65a5d1 No.11725

>>11724

>the program tends to eat memory when you heavily use it

I think that goes for most programs.

When I had 8GB of RAM my average overall usage was around 3GB, but now that I have 16GB it averages 6 or 7 even though I'm doing the same things.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

b1a0af No.11730

>>11718

Bare client is about 115MB ram on Windows, so the absolute minimum is pretty low compared to what many users see. I am about 490MB on my IRL laptop that has about 17k items in its main session (mostly thread watchers).

The python 3 change lead to some pseudo-memory-leaks, so that was a recent chunk I was able to recover, and I am making an important db-memory change in 341 today.

I would appreciate continued feedback here. There isn't a huge amount I can do atm about clients that have hundreds of thousands of pending URLs in their session, but PTR update processing and some other db stuff should be much less traumatic now. There's still more to do, and I still plan to make a memory profiler this year to draw some pie charts or something and squeeze this further.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



YouTube embed. Click thumbnail to play.

066d9e No.11606 [Open thread]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v340/Hydrus.Network.340.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v340/Hydrus.Network.340.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v340/Hydrus.Network.340.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v340/Hydrus.Network.340.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v340.tar.gz

I had a great if busy week. The Client API does more, several bugs are fixed, some new features and options are in, and stability and memory use should be a bit better.

client api

This is still somewhat advanced, but users who are interested may want to start looking in.

The first version of the Client API last week went ok! There were a couple of bugs, but thanks to feedback from some advanced users, I've improved reliability and carved out a better spec. This week brings JSON everywhere, fixes the add_file crash, and adds two neat calls:

/add_urls/get_url_files nowPost too long. Click here to view the full text.

30 posts and 17 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

066d9e No.11704

File: 4e55e5c587407d0⋯.png (27.1 KB,1135x188,1135:188,p.png)

>>11688

Hey, Pixiv is a slightly odd site, so there are two caveats here:

1) Ugoiras are not supported, so they will most likely get 'ignored' status.

2) Any 'manga' page will branch out to create new file import entries in the queue, leaving a stub entry.

Pic related has an example of 2. It is an unfortunate artifact of how Pixiv have decided that some 'mode=medium' URLs refer to one file while others refer to multiple. Although the initial https://www.pixiv.net/member_illust.php?illust_id=73135351&mode=medium url here gets 'successful', meaning it was a queue item processed successfully, it doesn't have any files itself.

Could the manga surplus explain your bad file counts here? It sounds maybe reasonable for the 2033 vs 1731, but not for the 307 to 21. Can you tell me more about what that 307 means? Where did you see it, and where did the 21 come from?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

066d9e No.11705

>>11699

>>11700

Thank you. Your situation does not sound crazy–2k thumbs is fine, for example. I have now done more for this for 341. In particular the db will now spool temp data to disk if it gets too big. I expect many users to see significantly less memory use during PTR work.

Please keep me updated here.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

46aebf No.11711

File: 4722a45faf3ad08⋯.png (45.6 KB,1081x532,1081:532,client_2019-02-25_16-07-41.png)

>>11704

Its possible that its manga and ugoria, but the problem I have is that it says there are 307 new images, but when I tell it to present new, only 21 show up.

my concern is that it's filing things that may, in the future, be parseable under an I have it, and when it rechecks it thinks I have it and skips over it.

>>11703

for this, sorry, im not getting them from exhentai, im finding new artists, usually in a gallery for an artist they will post where you can find their work directly, and I plug it in to download.

effectively, I download a gallery of something in the gallery downloader, and a few weeks later I see an update to it, so I regrab it in the gallery, the best way to put it is these are impulse downloads, not something I want to set a subscription up for. I think this shows off what I would like to avoid fairly well, if there was a way to recheck without adding the same search again, that would be great.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

066d9e No.11712

>>11663

>>11636

I did some work on this today and it went well! I have a 'thumbnail experiment mode' in for v341 under help->debug->gui so you can try it out yourself. It loads the fullsize and resizes in memory. There is not a human-noticeable difference in speed. On my dev machine here, which is a few years old, I added some timers and got (once the file data was in memory) approx 500µs to load a resized thumb and 1.8ms to load a fullsize thumb and scale it, which I was impressed by. I have a Nvidia 1030 here to drive my 4k monitor, so perhaps that is accelerating this.

I am willing to experiment more here, so I will mention it in my release post and see what you and other users find.

After looking at the code, I think that in exchange for simplifying the file system by only having one set of thumbs, I could justify making that single thumb more intelligent in how it swap outs bad-size thumbs when needed (i.e. basically removing the master rather than the resized). So, I think we can get the best of both worlds here, saving space and keeping things fast. I have a job set out for it now, so I'll try to chip away at this in the coming weeks. Thank you for bringing this up.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

60c4ac No.11713

>>11705

Last bit of info is I currently have about 100-110 gallery download ques (none running), that would send the pending tags through the roof but since Hydrus only displays 1 at at time then I dont think this would do much for thumbnails displayed. It would certainly push up high use of the CPU, disk writing and some ram but only when Hydrus is busy. Despite all the stuff I listed I'm only around 450 MB of Ram.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



f32422 No.11676 [Open thread]

I fell ill this week and did not get enough work done to justify a release. I will make some of the time up tomorrow and then continue as normal, aiming for 341 to be out on the 27th.

Thanks everyone!

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

d12ad4 No.11681

>>11676

Take care of yourself and get well soon, hydrus dev.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

dfefa4 No.11683

File: dfdf7da9ce7e4f0⋯.jpg (466.29 KB,1000x653,1000:653,dfdf7da9ce7e4f04591d7d5054….jpg)

>>11676

Thanks for the update, get well soon!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

25baa4 No.11684

>>11683

Is this when she dies? I've never seen this in game

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



293282 No.11455 [Open thread]

Does anyone have a working gallery importer for AGN.PH? Gotta collect it all…

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

bfa640 No.11464

>>11455

Ask for a parser

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

8be70c No.11470

I was able to script about 70% of it but ran into an issue. The site has one of those fucking click through screens. other than that it is practically identical to e621. Just with worse tagging.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

9ce048 No.11521

>>11470

>The site has one of those fucking click through screens.

You mean, the confirm age one?

That may be solved by connecting to the "yes, I am 18" URL directly.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

293282 No.11656

>>11521

it blocks that. you need A verified cookie.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



YouTube embed. Click thumbnail to play.

7c3464 No.11548 [Open thread]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v339/Hydrus.Network.339.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v339/Hydrus.Network.339.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v339/Hydrus.Network.339.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v339/Hydrus.Network.339.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v339.tar.gz

I had a good week. I fixed some bugs and finished one part of the Client API.

client api

(This is only for advanced users for now.)

The Client API is now ready for some experimentation! Although most of this work has gone much faster than I expected, the last step of actually writing the server commands along with some half-decent test code and help documentation is proving a little slower. This week rolls out 'set up API access' and 'add urls' functionality. In the coming weeks, I will roll out 'add file', 'add tags', and 'search files' commands.

The basic (under coPost too long. Click here to view the full text.

22 posts and 5 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7c3464 No.11598

>>11596

After a bit more poking around, I think regular garbage collection wasn't running for some reason. I'll do some more work on this today, it may be related to another issue.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7c3464 No.11599

>>11587

Thank you for this report. Just to check, this is non-Windows, right? Did you happen to migrate an install from Windows? I have had the worst luck trying to get the floating version of the autocomplete to work on non-Windows for these sorts of focus issues, so Linux and OS X installs start with it always in embedded mode, like how the 'manage tags dialog' works.

Please hit options->gui->always embed autocomplete dropdown results window and let me know if that works better.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

dd7e55 No.11603

File: ed96a686209f415⋯.png (2.38 MB,2096x2755,2096:2755,ed96a686209f415c58fcb98791….png)

>>11599

Thank you, it's working now. I almost was about to post that it still wasn't working but the client crashed when I was about to take a screenshot and it started to work after the restart.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

dd7e55 No.11604

>>11599

Oh, and to add to your question: I indeed have migrated from Windows long time ago. I just tried a fresh install and it defaults to that setting, so the problem was just outdated defaults. If I wasn't pantsu-on-head retarded, I would've checked if it was working on a fresh install first. Sorry for wasting time.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f778c7 No.11605

>>11603

I reinstalled 339 and havent noticed the memory spike as bad as before but it is locking up and stops responding pretty often so i still end up having to restart anyways for any downloading. the program still works for main archival and search though.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: 47aff7a65fdde59⋯.gif (244.75 KB,774x900,43:50,47aff7a65fdde59c7f5569108a….gif)

605d3d No.11601 [Open thread]

I had a great if busy week. I fixed the stupid page of pages bug (and a whole bunch more), fleshed out the new client api, added some small new options, improved stability, and added several new downloaders to the client defaults.

The release should be as normal tomorrow, maybe a little late.

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.


b74596 No.11529 [Open thread]

need the size of the thumbnail more than 200x200.

on 4k TV, they look small

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

f7b1c3 No.11546

Read >>10203 then write up a proper proposal.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



File: c1adee5f5ad8a8b⋯.gif (710.97 KB,600x400,3:2,c1adee5f5ad8a8bc16c3f778d7….gif)

641cb8 No.11544 [Open thread]

I had a good week. I fixed a lot of bugs (including a couple of stupid ones I let slip into 338) and finished a first subset of the client api for advanced users to play with.

The release should be as normal tomorrow.

____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
Post last edited at


File: ab23431121b8563⋯.jpg (96.69 KB,492x650,246:325,ab23431121b8563a7bc3b7ac57….jpg)

262b3a No.5115 [Open thread][Last50 Posts]

Gonna start another thread because >>173 is becoming too big for most people.

In the tag manager, whenever multiple tags are selected for multiple files with different tags, if you press the del key, the program will still ask you if you want to delete them all or put them all in the selected files. This does not make sense, if I'm pressing the del key it's obvious that I want them gone.

643 posts and 201 image replies omitted. Click [Open thread] to view. ____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7d5801 No.11526

>>11462

Thank you, this vid is very helpful. It looks like you have a pretty heavy client, with a lot of pages open and a ton of pending tags. Do you have, say, more than 100 pages open? I am wondering if your client is running out of ids for menus quicker because it has all the other stuff going on. Is it possible you can temporarily close some of those pages, restart the client, and see if that eases things up a bit?

Have you tried 'page of pages', btw? This helps nest pages Hit F9 (i.e. choose a new page) and then special->page of pages. You can drag and drop page tabs onto that new page 'folder' and you won't be navigating left and right across a long ribbon of tabs.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7d5801 No.11527

>>11510

>>11517

Thank you for this report. I apologise, I fucked up a change here. I will have it fixed for 339.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

133fd0 No.11532

Hi, having a small problem with the Manage Import Folders option.


TypeError
CallBlockingToWX() missing 1 required positional argument: 'func'
Traceback (most recent call last):
File "include\HydrusThreading.py", line 342, in run
callable( *args, **kwargs )
File "include\ClientGUI.py", line 2608, in THREAD_do_it
controller.CallBlockingToWX( wx_do_it )
TypeError: CallBlockingToWX() missing 1 required positional argument: 'func'

Not a problem with export folders.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7d5801 No.11535

>>11532

Thank you for this report. I made a couple of stupid mistakes last week–this was one. I am sorry for the problem. It will be fixed in tomorrow's release.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

7d5801 No.11543

This thread is getting laggy, so:

N E W

E

W

T H R E A D

H

R

E

A

D

>>11542

>>11542

>>11542

>>11542

>>11542

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.



Delete Post [ ]
[]
[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]
| Catalog | Nerve Center | Random
[ / / / / / / / / / / / / / ] [ dir / random / 93 / biohzrd / hkacade / hkpnd / tct / utd / uy / yebalnia ]