Version 305

Name
Email
Subject
Comment *
File	Select/drop/paste files here
Password	(Randomized for file and post deletion; you may also set your own.)
Archive	Archive [500 char limit]
* = required field	[▶ Show post options & limits] Confused? See the FAQ.

Embed	(replaces files and can be used instead)
Voice recorder	Show voice recorder (the Stop button will be clickable 5 seconds after you press Record)
Options	Do not bump (you can also write sage in the email field)
Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf Max filesize is 16 MB. Max image dimensions are 15000 x 15000. You may upload 5 per post.

HookTube embed. Click on thumbnail to play.

Version 305 hydrus_dev ## Board Owner 05/02/18 (Wed) 22:30:02 ee0a3d No.8754

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v305/Hydrus.Network.305.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v305/Hydrus.Network.305.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v305/Hydrus.Network.305.-.OS.X.-.App.dmg

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v305/Hydrus.Network.305.-.OS.X.-.Extract.only.tar.gz

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v305/Hydrus.Network.305.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v305.tar.gz

I had a great week with a bunch more downloader overhaul work done. The new parsing system now kicks in in a few more places and some sites now support url drag-and-drop by default.

new gallery parsers

I have tightened up last week's improvements on the new gallery page downloader and integrated it into regular downloader pages and subscriptions. Now, if the client recognises a url in any downloader or sub and knows how to parse it in the new system, it will use that new parser seamlessly.

I have also written new parsers for pixiv, danbooru, safebooru, and e621 for exactly this, so if you use any of those, you may notice they now populate the 'source time' column of the file import status window (which is useful for subscription check timing calculations) or that these parsers now pull and associate additional 'source urls' from the files' pages (so although you may download from danbooru, you might also get a new known pixiv url along the way).

A neat thing about these parsers is that if one of these additional source urls has already been seen by the client, the client can use that to pre-determine if the file is 'already in db' or 'previously deleted' before the file is downloaded, just like it would the main post url, saving time and bandwidth. The danbooru and e621 ones even pull md5/sha1 hashes and check those, so if everything works right, you should never have to redownload anything from danbooru or e621 again!

I also fixed the pixiv downloader more generally, which I had broken in last week's url normalisation update (and due to some other pixiv-specific stuff). I apologise for the inconvenience–everything should be working again (although you may have some useless bad urls from v304 that are missing the 'mode=medium' component that you may wish to skip/delete rather than let error-out), and the new pixiv parser fetches the romaji versions of tags now as well. Manga pages aren't supported yet, and tag searching is still down, but as I roll out some more gallery stuff here, I think I'll be able to figure something out.

Another upshot of the new parsers is that the client can now receive these sites' post urls as drag-and-drop events. Try dragging and dropping a danbooru file post url (like this https://danbooru.donmai.us/posts/2689241 ) on the client–it should all get imported in a new 'urls downloader' automatically, with all the new url association and everything! (You might want to check the new 'manage default tag import options' under the 'network' menu before you try this–the whole download system has a foot in two worlds at the moment, and while some parts pull TIO from the old system, the new url-based auto-stuff looks there.)

And lastly, with the help of @cuddlebear on the discord, there is a comprehensive yiff.party API parser in place, also with drag-and-drop support. Due to the shape of the data that yiff.party presents, this creates a thread watcher. You can even set these watchers to check like every 30 days or so, _and they should work_ and keep up with new files as they come in, but I recommend you just leave them as [DEAD] one-time imports for now: I expect to integrate 'watchable' import sources into the proper subscription system by the time this overhaul is done, which I think is probably the better place for more permanent and longer-period watchables to go.

I am pleased with these changes and with how the entire new downloader system is coming together. There is more work to do–gallery parsing and some kind of search object are the next main things–but we are getting there. Over the next weeks, I will add new parsers for all the rest of the default downloaders in the client (and then I can start deleting the old downloader code!).

other stuff

Import pages now report their total file progress after their name! They now give "(x, y/z)", where x=number of files in page, y=number of queue items processed, z=number of queue items unknown. If y=z, only "(x)" is reported. Furthermore, this y/z progress adds up through layers of page of pages!

If you try to close a page of pages (or the whole application), and multiple import pages want to protest that they are now yet finished importing, are you sure you want to close y/n, the client now bundles all their protests into a single yes/no dialog!

If manage subs takes more than a second to load, it'll now make a little popup telling you how it is doing.

full list

- fixed the pixiv url class, which was unintentionally removing a parameter

- wrote a pixiv parser in the new system, fixing a whole bunch of tag parsing along the way, and also parses 'source time'! by default, pixiv now fetches the translated/romaji versions of tags

- finished a safebooru parser that also handles source time and source urls

- finished an e621 parser that also handles source time and source urls and hash!

- wrote a danbooru parser that also handles source time and source urls and hash!

- as a result, danbooru, safebooru, e621, and pixiv post urls are now drag-and-droppable onto the client!

- finished up a full yiff.party watcher from another contribution by @cuddlebear on the discord, including url classes and a full parser, meaning yiff.party artist urls are now droppable onto the client and will spawn thread watchers (I expect to add some kind of subscription support for watchers in the future). inline links are supported, and there is source time and limited filename: and hash parsing

- fixed some thread watcher tag association problems in the new system

- when pages put an (x) number after their name for number of files, they will now also put an (x/y) import total (if appropriate and not complete) as well. this also sums up through page of pages!

- if a call to close a page of pages or the application would present more than one page's 'I am still importing' complaint, all the complaints are now summarised in a single yes/no dialog

- url downloader pages now run a 'are you sure you want to close this page' when their import queues are unfinished and unpaused

- if the subscriptions for 'manage subscriptions' take more than a second to load, a popup will come up with load progress. the popup is cancellable

- added a prototype 'open in web browser' to the thumbnail right-click share menu. it will only appear in windows if you are in advanced mode, as atm it mostly just launches the file in the default program, not browser. I will keep working on this

- harmonised more old download code into a single location in the new system

- created a neater network job factory system for generalised network requests at the import job level

- created a neater presentation context factory system for generalised and reliable set/clear network job ui presentation at the import job level

- moved the new downloader simple-file-download-and-import to the new file object and harmonised all downloader code to call this single location where possible

- did the same thing with download-post-and-then-fetch-tags-and-file job and added hooks for in the subscription and gallery downloader loops (where a parser match for the url is found)

- the simple downloader and urls downloader now use 'downloader instance' network jobs, so they obey a couple more bandwidth rules

- harmonised how imported media is then presented to pages as thumbnails through the new main import object

- the new post downloader sets up referral urls for the file download (which are needed for pixiv and anything else picky) automatically

- improved file download/import error reporting a little

- entering an invalid regex phrase in the stringmatch panel (as happens all the time as you type it) will now present the error in the status area rather than spamming popups

- fixed a bug in the new parsing gui that was prohibiting editing a date decode string transformation

- fixed enabling of additional date decode controls in the string transformations edit panel

- added a hyperlink to date decoding controls that links to python date decoding explainer

- if a source time in the new parsing system suggests a time in the future, it will now clip to 30s ago

- misc downloader refactoring and cleanup

- fixed an issue where new file lookup scripts were initialising with bad string transformation rows and breaking the whole dialog in subsequent calls, fugg

- hid the 'find similar files' menu entry for images that have duration (gifs and apngs), which are not yet supported

- added 'flip_debug_force_idle_mode_do_not_set_this' to main_gui shortcut set. only set it if you are an advanced user and prepared for the potential consequences

- silenced a problem with newgrounds gallery parser–will fix it properly next week

- fixed some old busted unit test code

- rejiggered some thumb dupe menu entry layout

next week

I will try to fit in some more parsers, and I might take a stab at a 'multiple thread watcher' page for advanced users. There's also an experimental new 'open file in web browser' that I had mixed luck with this week and would like to pin down a good multiplat solution for.

____________________________

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 00:33:19 f072cb No.8755

File: f64e3cfd6c23159⋯.png (510.03 KB, 497x732, 497:732, glad.png)

>>8754

If I'm understanding this correctly this is a change I've been waiting for, thanks!

One thing I'd like to ask for (unless it's already been added in this version): An option to copy URLs when filtering duplicates would save me a lot of time. I know that sometimes copying URLs from an "inferior" version would be bad, so I think it should be an optional thing, but again, this would really help me out especially now that more downloaders support getting the real source URLs along the download one.

I download images with Hydrus, then run them through image optimizers before reimporting them and dupe filtering them. I'd like to keep the URLs on the optimized images.

Not sure how hard it would be to implement, hopefully it's not too much to ask for. This would make Hydrus nearly flawless for my use-case.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 01:09:01 f072cb No.8756

>>8755

Meant to do bold, not spoiler. Wasn't thinking.

Do I need to do anything to get the new parsers? I ran a test download on one of those sites and it seems to get the same source URLs it did before.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 01:35:38 93869a No.8757

>>8751

This on the blacklist, this is how I currently see it.

the trash is more or less a dumping ground, either it does not delete till I tell it to or I made it not delete till I tell it to, i'm not sure which.

with the example of scat all the files being moved over to the trash area would still download them, but would also prevent them from showing up again in a download. If I remember right adding more folders/domains for files to get sorted into is something that you have planned, when this happens the functionality of presorting will likely be implemented then but a version now for again the example of scat, having it all sort to trash and then checking though it for anything worth keeping then deleting them from archive would be nice.

as for searching my files for scat then going through them, im not fully understanding what you mean

catching the file on import would keep it out of things you likely already ok'ed, rather than doing a search of everything you already have where you would need to re sort through everything to weed out the bad.

I also make the assumption that if you blacklist a tag, there are going to be VERY few exceptions to it being something you want to keep, rather then just not downloading anything that has it as a tag, moving it to trash to be combed through may be an option people want, I know I would likely use it that way for a few tags myself.

on the topic of of the media panel, at least as it pertains to showing new dynamical, is it possible to rig a work around where you could right click and have an option to 'open newly acquired files in new tab'?

the importer status clearly knows new from already in database, possibly something inside there could be used to open them in a new tab.

either way, here's hoping to see something to do with a multi thread watcher next week.

on a side note with thread watchers. I have been running into some issues where the last few images won't get seen due to how fast a thread can and boards without persistence.

would it be possible to add archive redirects for boards/sites?

something that once the thread 404's it checks the archive once before quitting.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 13:32:41 de6e68 No.8760

Hey, if I remember right there used to be a way to display potential duplicates of one particular selected file, I think you right-clicked the thumbnail? Don't see it now though, is there any way to still do that?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 13:49:05 f19d4f No.8761

File: fed70731dee2e6d⋯.png (5 KB, 501x471, 167:157, t-thanks.png)

File: d23a0bebad8f31e⋯.png (5.97 KB, 418x572, 19:26, hc.png)

Does anyone else have this problem? With Gnome checkboxes and scrollbars are invisible, except with high contrast mode. I've tried multiple different themes and the only one that seems to work is high contrast. This doesn't happen with any other program, I've even tried ones that use WxPython aswell.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 14:00:15 93869a No.8762

Just got this error

DBException

TypeError: buffer object expected

Traceback (most recent call last):

File "include\HydrusPubSub.py", line 135, in Process

callable( *args, **kwargs )

File "include\ClientGUI.py", line 4114, in PresentImportedFilesToPage

dest_page = self._notebook.PresentImportedFilesToPage( hashes, page_name )

File "include\ClientGUIPages.py", line 2554, in PresentImportedFilesToPage

unsorted_media_results = self._controller.Read( 'media_results', hashes )

File "include\HydrusController.py", line 448, in Read

return self._Read( action, *args, **kwargs )

File "include\HydrusController.py", line 154, in _Read

result = self.db.Read( action, HC.HIGH_PRIORITY, *args, **kwargs )

File "include\HydrusDB.py", line 868, in Read

return job.GetResult()

File "include\HydrusData.py", line 1688, in GetResult

raise e

DBException: TypeError: buffer object expected

Database Traceback (most recent call last):

File "include\HydrusDB.py", line 534, in _ProcessJob

result = self._Read( action, *args, **kwargs )

File "include\ClientDB.py", line 8330, in _Read

elif action == 'media_results': result = self._GetMediaResultsFromHashes( *args, **kwargs )

File "include\ClientDB.py", line 5502, in _GetMediaResultsFromHashes

query_hash_ids = set( self._GetHashIds( hashes ) )

File "include\ClientDB.py", line 3957, in _GetHashIds

result = self._c.execute( 'SELECT hash_id FROM hashes WHERE hash = ?;', ( sqlite3.Binary( hash ), ) ).fetchone()

TypeError: buffer object expected

Database Traceback (most recent call last):

File "include\HydrusDB.py", line 534, in _ProcessJob

result = self._Read( action, *args, **kwargs )

File "include\ClientDB.py", line 8330, in _Read

elif action == 'media_results': result = self._GetMediaResultsFromHashes( *args, **kwargs )

File "include\ClientDB.py", line 5502, in _GetMediaResultsFromHashes

query_hash_ids = set( self._GetHashIds( hashes ) )

File "include\ClientDB.py", line 3957, in _GetHashIds

result = self._c.execute( 'SELECT hash_id FROM hashes WHERE hash = ?;', ( sqlite3.Binary( hash ), ) ).fetchone()

TypeError: buffer object expected

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 14:01:24 4e6ed3 No.8763

>>8760

Do you have advanced mode on? It's supposed to be an option on the help section of the top menu.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 14:02:06 93869a No.8764

>>8762

AttributeError

'builtin_function_or_method' object has no attribute 'encode'

File "site-packages\wx\core.py", line 3190, in <lambda>

File "include\ClientThreading.py", line 382, in wx_code

self.Work()

File "include\HydrusThreading.py", line 603, in Work

self._work_callable()

File "include\HydrusData.py", line 1516, in call

self._func( *self._args, **self._kwargs )

File "include\ClientGUI.py", line 4321, in SaveLastSession

self._notebook.SaveGUISession( 'last session' )

File "include\ClientGUIPages.py", line 2714, in SaveGUISession

session_hash = hashlib.sha256( session.DumpToString() ).digest()

File "include\HydrusSerialisable.py", line 180, in DumpToString

obj_tuple = self.GetSerialisableTuple()

File "include\HydrusSerialisable.py", line 219, in GetSerialisableTuple

return ( self.SERIALISABLE_TYPE, self._name, self.SERIALISABLE_VERSION, self._GetSerialisableInfo() )

File "include\ClientGUIPages.py", line 2891, in _GetSerialisableInfo

serialisable_page_tuple = GetSerialisablePageTuple( page_tuple )

File "include\ClientGUIPages.py", line 2877, in GetSerialisablePageTuple

serialisable_hashes = [ hash.encode( 'hex' ) for hash in hashes ]

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 14:02:52 93869a No.8765

>>8764

DBException

TypeError: buffer object expected

Traceback (most recent call last):

File "include\HydrusThreading.py", line 282, in run

callable( *args, **kwargs )

File "include\ClientGUIPages.py", line 813, in THREADLoadInitialMediaResults

more_media_results = controller.Read( 'media_results', group_of_initial_hashes )

File "include\HydrusController.py", line 448, in Read

return self._Read( action, *args, **kwargs )

File "include\HydrusController.py", line 154, in _Read

result = self.db.Read( action, HC.HIGH_PRIORITY, *args, **kwargs )

File "include\HydrusDB.py", line 868, in Read

return job.GetResult()

File "include\HydrusData.py", line 1688, in GetResult

raise e

DBException: TypeError: buffer object expected

Database Traceback (most recent call last):

File "include\HydrusDB.py", line 534, in _ProcessJob

result = self._Read( action, *args, **kwargs )

File "include\ClientDB.py", line 8330, in _Read

elif action == 'media_results': result = self._GetMediaResultsFromHashes( *args, **kwargs )

File "include\ClientDB.py", line 5502, in _GetMediaResultsFromHashes

query_hash_ids = set( self._GetHashIds( hashes ) )

File "include\ClientDB.py", line 3957, in _GetHashIds

result = self._c.execute( 'SELECT hash_id FROM hashes WHERE hash = ?;', ( sqlite3.Binary( hash ), ) ).fetchone()

TypeError: buffer object expected

Database Traceback (most recent call last):

File "include\HydrusDB.py", line 534, in _ProcessJob

result = self._Read( action, *args, **kwargs )

File "include\ClientDB.py", line 8330, in _Read

elif action == 'media_results': result = self._GetMediaResultsFromHashes( *args, **kwargs )

File "include\ClientDB.py", line 5502, in _GetMediaResultsFromHashes

query_hash_ids = set( self._GetHashIds( hashes ) )

File "include\ClientDB.py", line 3957, in _GetHashIds

result = self._c.execute( 'SELECT hash_id FROM hashes WHERE hash = ?;', ( sqlite3.Binary( hash ), ) ).fetchone()

TypeError: buffer object expected

——————-

apparently its related to an import folder or a file that was imported.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ice cold ~baka~ 05/03/18 (Thu) 15:06:30 2f0eee No.8766

>>8765

Thanks for working on the pixiv situation

I tried the pixiv profile:21848 (white datura)

I get 4 urls per illustration:

- one to the image itself which is not supported by pixiv(403 forbidden) but may be potentially useful;

https://i.pximg.net/img-original/img/2018/01/16/01/02/43/66820838_p0.jpg

- one broken url which send to my own profile as an error:

https://pixiv.net/member_illust.php?illust_id=66820838

- one working url correctly sending to the pixiv work's page similar to the one you get while browsing pixiv;

https://www.pixiv.net/member_illust.php?mode=medium&illust_id=66820838

- one working url correctly sending to the pixiv work's page but with reversed illust_id & mode

https://www.pixiv.net/member_illust.php?illust_id=66820838&mode=medium

the last one surprised me, I don't think there is a scenario where they can be produced naturally. I'd like to nkow if anyone did encouter that formatting before.

On the tag matters, While the romanji are fetched the hiragana/kanji which are the one actually wanted are still ignored.

Did I perhaps miss a setting?

While it's better than nothing, it looks like these are automated romanization, not official ones.

for example: da-rinninnzafurannkisu instead of darling in the franxxx., which can only be a phonetic help at best.

I'd rather have the hiragana and not these bad romanji attempt in the database.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 15:14:30 93869a No.8767

>>8762

>>8764

>>8765

Ok, I want to say this is related to imports and haveing imports from 1 folder add to a tab.

The imports will not show up at all, however it seems they were all written to the database and can be found, so at least I know that worked

looking at it, I cant even open the 'yep we imported everything' dialouge in its own window. so it may/is an error brought on from just importing files, thankfully they are fully integrated, as in its findable and in the dup finder…

im considering shutting down the imports for now but not sure if its nessassary.

>>8766

I think you quoted the wrong person

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ice cold ~baka~ 05/03/18 (Thu) 15:21:50 2f0eee No.8768

>>8767

I used your post to open the post box and forgot to delete the link.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 15:23:37 93869a No.8769

File: b498ba512d03be1⋯.png (196.9 KB, 418x209, 2:1, client_2018-05-03_10-21-26.png)

oh I should also mention in conjunction with >>8767

a big thank you for the url links

images were getting imported and it looked corrupted, either the scrapper for the archive fucked up, or they had fun making the files fucked up on their own, not sure which.

having the url that the images were acquired from pop up allowed me to see the thread and see if it was fucked up from there, and the entire thread is almost like this.

That made an oh shit moment go away fast.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 15:32:30 93869a No.8770

>>8768

lol

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 15:50:27 de6e68 No.8771

>>8763

Ah, thank you. I'm the one who just updates about 80 versions at a time, didn't even know about it.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

ice cold ~baka~ 05/03/18 (Thu) 15:51:31 2f0eee No.8772

I see there's more detail on that tag feature on the discord channel, I should have looked there first, sorry.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 21:31:37 8acdb3 No.8774

Do you think you could add a parser for https://archived.moe/? It's an image-board archiving site. I tried adding a parser myself, but there are a few caveats: the used software's API seems to be disabled, parsing the HTML works but all images are hidden behind a redirect URL (with the actual URL in a meta-field). This 2-step process of scraping the redirect URLs and scraping the actual URLs from the redirect site HTML seems a bit tricky.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 22:16:25 8acdb3 No.8775

>>8774

These are what I came up with. I specified the thread URLs ("archive.moe thread") as a watchable URL so I can actually modify it in "url class links". I assigned the gallery parser to "archive.moe thread" and the post url parser to "archive.move file page parser". But this whole system is just confusing.

I think it would have been much less troublesome to let people write some Python to be honest.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 22:16:55 8acdb3 No.8776

File: 3d343234206f8cf⋯.png (3.22 KB, 512x82, 256:41, archived.moe file page par….png)

File: de0257c234572d1⋯.png (3.02 KB, 512x82, 256:41, archived.moe thread parser.png)

>>8775

Oops forgot files:

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/03/18 (Thu) 23:14:40 777dc0 No.8777

File: 1b7a30b07fbc3ee⋯.png (3.4 KB, 512x109, 512:109, vidyart.png)

Gallery downloading isn't there yet, but I still wrote a parser to try this out. It works great!

If I had one request to make, adding "take everything after substring X" to string transformations would be extremely practical - sometimes it's pretty hard to trim a dumb string result due to possible variable length:

Statistics

Id: 348910

Posted: 2018-01-29 13:53:12

By: Kking64

Size: 620x500

Getting the Posted: timestamp here is nigh impossible in the current system (doesn't help I'm probably very bad at it), whereas it'd take 3 seconds with substrings.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/04/18 (Fri) 20:00:49 ac94a9 No.8778

>manga pages still not supported

you can do it Hydev-sempai!

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/05/18 (Sat) 04:05:09 93869a No.8781

Ok got a question because this poped up.

just went through a download of 50k images, found a few deleted images, so went through them to make sure they were deleted for a reason (I have fully deleted less then 300 images so when one pops up as deleted its a flag to check till notes are supported in the detailed status at the very least) so I open them up, I agree with every deleted one except 1 that I have no idea why it was deleted. so I want to import it back

I go to url import and try, its blocked

I allow it to import from there, it goes through

now here is where I have an idea.

I have found no real use for url import other then consolidating failed imports to a single tab

Is it possible to have seperate import rules depending on what i'm using? for example simple downloader I would want to have gifs limited, I would want to have improt bombs checked, but url downloader, im thinking I want to make settings so when I use that, it overrides everything and just imports regardless of what it was.

is there a way to do this currently and if not is there a way to make this possible without to much work?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/05/18 (Sat) 08:37:14 748ade No.8782

Can someone who 'gets' furries explain them to me?

Is it really just the result of a bunch of children who sexually imprinted onto Sonic the Hedgehog when they were younger?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/05/18 (Sat) 18:57:59 93869a No.8787

>>8782

its largely what people were exposed to as a kid. for the western world its anthropomorphic characters.

but lets also make a clear distinction here

there is a large difference between someone who self identifies as a furry and goes to conventions, and someone who just likes the porns art.

now as for just autism and all that shit, look at the anime community and their conventions. they have largely the same kind of crowd, but because their shit isn't deviant by nature they don't get the people who wear adult diapers, shit in them, and then leave them around in public

when they do, its fairly easy to toss that fucker aside and not be lumped in with that faggot.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

hydrus_dev ## Board Owner 05/05/18 (Sat) 22:25:59 ee0a3d No.8788

File: 9b3d4b18190b78d⋯.jpg (46.7 KB, 300x405, 20:27, 9b3d4b18190b78d6c81faf1e27….jpg)

>>8756

>>8755

Yeah, merging URLs is something I would like to do, and I think with the new url classes in place we are just about ready to do it. Adding these new source urls this week has already poked a hole in my url file-status checking logic, but the fundamental idea of 'this file can have multiple urls, and those urls can be true for multilple files, and fuck the fact that the bytes don't match exactly with what is actually hosted' is unavoidable now, particularly with The Eternal Cloudflare and other CDNs serving different file content anyway. Everything is fucked when it comes to network content now, so I am ok with just trying to muddle through going forward.

I expect I'll add a checkbox or something to the merge options on whether you want to merge urls. Default off, probably. If I feel clever, it may be several checkboxes, like 'only do recognised post urls' vs 'do everything', but we'll see.

>>8757

Thanks, I understand better. I'll see if I can figure out some more options for what to do with blacklisted files, then. My week is already blatted, but I'll see if I can do it soonish.

And for 'open in a new tab', yeah, it'll be from the file import status window. Those actually know their files' hashes now, so I just have to write a new menu entry that'll pull those hashes and send them to a new window.

Doing a clever switch-on-death to a new archive site is probably too difficult for this new system I am building, but I have been working with another Anon on doing some archive parsing support, so I expect we'll have options for you to fix this manually in a bit. Then again, I haven't really started on the gallery step of the new parser yet, so there might be a clever hook here if you can pull the archive url from the 'this thread is dead' page. We'll see when we get to it.

I am not sure if I'll have time to do multiple thread watcher this week now. I've had a ton of messages about small bugs–I've now got 27 'must do' jobs, and a typical week is ~8. I might just be catching up on shit all this week.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

hydrus_dev ## Board Owner 05/05/18 (Sat) 22:36:36 ee0a3d No.8789

File: 64e2a5a3a83aba2⋯.png (2.79 KB, 512x85, 512:85, pixiv with jap tags.png)

>>8762

>>8764

>>8765

Thank you for this report. I have a couple more of these from other people.

I am pretty sure what is going on here and am confident I will have it fixed for v306. I think I will be able to fix the broken session/import folder load retroactively, so I recommend you pause everything for now and try again next week. I apologise for this mess–the new download code is having some hash-association problems when the import object is a local path rather than network url.

>>8766

Thanks–I hope to have tools to hide or delete those dupe/useless URLs in the coming weeks.

I think my attached file will fetch the jap tags. Import it in network->manage parsers and then go ''network->manage url class links and double-check that 'pixiv file page' is linked to the correct parser.

Some of those tags seem to be good ('heavenly ass' makes me laugh every time I see it), but I guess any not pre-defined are automated. Pixiv is a nightmare to work with tbh. I just learned today that they are rolling out some new dynamic site markup that is killing parsing for random users, fugg.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

hydrus_dev ## Board Owner 05/05/18 (Sat) 23:00:20 ee0a3d No.8790

File: d716785afc71cc3⋯.png (68.39 KB, 1198x630, 599:315, posted.png)

File: 8859388211ed13d⋯.png (1.76 KB, 512x84, 128:21, posted_2.png)

>>8772

np m8–all this is up in the air atm. I expect we'll need a central destination/service for distributing these pngs by the time we are done here.

>>8774

>>8775

>>8776

Thanks–I just talked to you otherwise. For other anons–we can't quite do this yet, but the upcoming gallery work should better allow complicated redirect solutions like this.

>>8777

Great, I am glad you could get something neat going. I apologise that the UI is still a hellscape–I'll neaten it up and make the test code less of a hassle before I am done here.

I have been using regexes for these 'find one thing in a list of brs' jobs and it has been ok. I do a stringmatch on 'starts with Posted:' and then a stringconverter to strip it off. If you actually want to get the timestamp as integer, you then need a second 'date decode' stringconverter step as I've done in the other new parsers this week. For this example, the magic string would be %Y-%m-%d %H:%M:%S. Attached as an example, hope I didn't fuck it up.

Example data here:

<li>Id: 4225580</li>
				<li>Posted: 2018-05-05 17:31:24<br /> by <a href="index.php?page=account&amp;s=profile&amp;uname=danbooru">danbooru</a></li>
				<li>Size: 3000x4466</li><li>Source: <a href="https://www.pixiv.net/member_illust.php?mode=medium&amp;illust_id=68344468" rel="nofollow">www.pixiv.net/member_illu...um&amp;illust_id=68344468</a></li><li>Rating: Safe</li>
				<li>Score: <span id="psc4225580">0</span> 
				(vote <a href="#" onclick="Javascript:post_vote('4225580', 'up'); return false;">Up</a>)
				</li>

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/05/18 (Sat) 23:06:53 b903a2 No.8791

File: da984f98e3017a7⋯.png (136.06 KB, 1890x1080, 7:4, 1505524718708.png)

>>8787

>>8782

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

hydrus_dev ## Board Owner 05/05/18 (Sat) 23:18:31 ee0a3d No.8792

File: 6ba181a73a6733a⋯.gif (1.18 MB, 291x260, 291:260, 6ba181a73a6733a56e603142db….gif)

>>8778

Not long now, hopefully.

>>8781

I don't think I would easily be able to set up different file/tag import options based on the kind of import page–the new system is moving towards deriving these defaults by url class. I recommend using one url/simple downloader import page, giving it a good page name and setting its file import options exactly how you want, and then either keeping it open all the time or having it in a special gui session so you can load it up whenever you want. Then set the default under 'options->importing->default file import options' to whatever you use and will be opening/closing more often.

>>8782

>>8787

I have read that a variation of 'Erotic target location error', where a person can't socially acceptably find their object of desire in surroundings, so they transform themsely (or imagine doing so) into it, is one explanation–which also fits for futa/transsexuals/guys who want to transform into diapers–and also explains why it tends to be men rather than women who participate in these communities, as women don't seem to get ETLE as much afaik, but that may just be some psychology bullshit that doesn't have any real evidence.

I'd be interested in seeing if anyone has any genetic or cultural research into furrydom or any of these auto-philias. Is 'germans are furries more than anyone else' a meme or legit?

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/06/18 (Sun) 05:18:23 748ade No.8793

>>8792

I dont like this explanation because it seems to muddy the waters between the furries as a phenomenon and the whole set of paraphillias that it spawned later (babyfurs, diaperfurs, etc). These seem like two original distinct entities that at some point merged together to form some unholy union. Furries definitely did not start with them in their "cultural" DNA so to speak so it had to have occured later.

Regarding why women dont seem to suffer from paraphilias,

I dont have any studies on hand but I do recall a nature documentary I saw years back that connected a bunch of dots in my head at the time. The doco was about some male duck or swan raised by a bunch of researchers and when it grew up it had no sexual interest in females of its own species but rather the female handler that raised it. Apparently, the male sexually imprinted onto the scientist during its formative years. That seemed to suggest that for the male sex, sexual matters had to be learned rather than be ingrained from birth. Whereas for females, it is innate.

So when the learning process is corrupted or absent, thats when paraphillias occur (e.g. molested pre-pubescent males turn out homosexual at a significantly inflated rate to their non-molested counterparts, so this is essentially how gays "reproduce"). And in the modern world, there is a LOT of ways to fuck this process up (single-parenting, porn, child abuse, medications) so you have a huge explosion of kinks and fetishes that have nothing to do with reproduction (feet, stuffed toys/inanimate objects, scat, diapers, etc).

I guess you could make the argument that lesbianism is a refutation of this idea, but what I would remind you it's mostly due to a hatred of men rather than an innate attraction to women thats behind it.

>>8791

>>8787

Thanks broskis, I think these two posts explains it the best.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/06/18 (Sun) 12:21:18 93869a No.8794

>>8788

well, either way with the multi tab, I at least know you are trying something soon. I would love to abandon my current system and just use this one for dealing with threads.

for archivers, from what I have seen, they mostly use the same sorting method just different url, you could likely have the threadwatcher run then 404 normally, with a button to check the archive where it would pop up a new tab that check the archive. because all the archives tend to use the same sile/thread storage method, swapping out the chan part with the archive,

https://boards.4chan.org/b/thread/767795966

for

https://thebarchive.com/b/thread/767795966

https://boards.4chan.org/trash/thread/15559105

for

https://desuarchive.org/trash/thread/15559105

I would have a red flag go up if the archive was unreachable or if the files there were leading to bad/unparseable links.

>>8789

for the import, it appears to just not be able to load up the page, all files are being imported correctly, barring a massive download, ill just do a search for new files and get them dealt with then.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/06/18 (Sun) 14:30:46 674318 No.8795

import folders don't open their files in a page anymore

the "if new files imported, publish them to a page" option doesn't work

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/07/18 (Mon) 09:50:46 93869a No.8800

see

I believe are all about the error which according to >>8789

dev thinks he knows the fix.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

hydrus_dev ## Board Owner 05/09/18 (Wed) 16:43:59 ee0a3d No.8804

File: e974bfc227adb59⋯.jpg (159.05 KB, 878x432, 439:216, e974bfc227adb59d247babf3b0….jpg)

>>8793

What interests me most is the auto- part of it. If a group of people were sexually attracted to animal features and simply liked to look at pictures that had those qualities or generally try to fuck animals irl, the imprinting explanation would be sufficient. I am sure many men pick women who have some distinctive feature shared by the women during their upbringing. None of these men would themselves fantasise about becoming or claim to be women.

But that many furries dress themselves up and act as animals (or simply imagine doing so) suggests an additional layer. The hypothesis that being unable to either locate the object of their attraction (muh horse pussy too far away/muh loving qt gf doesn't exist) or find a socially acceptable space to pursue it (don't fuck animals you deviant/fuck off you sperg) then makes for a desire for internal transformation is one plausable explanation. The idea of becoming their paraphilia's object, which is more attainable and socially acceptable, becomes the new source of lust.

This stretches the metaphor a little too far, but if we were remove the female handler from the swan, not only would the swan not be interested in female swans, but perhaps it would pull out its feathers and proclaim to the other swans, "I am a human born in a swan body, respect my identity!" which would swiftly be followed by creating Tumblr and Deviant Art accounts.

I don't know anything at all about scatfags, but I figure those ones who imagine transforming into diapers and toilets and all that other bananas shit may have a similar mental circuit firing. I don't know why they would attach themselves to the otherwise non-scat furry community, unless perhaps their concentrated autism brought them there from Sonic etc… Or perhaps it is a case of one group of extreme outcasts trying to get acceptability from a group on a relatively higher tier.

If autism is a big contributor to paraphilia and autoparaphilia, then that would also explain why it is mostly men and not women.

>>8794

I will think about this thread stuff, but it may be too complicated for the system I am making now.

I've fixed the import folders stuff for today. They were broadcasting invalid file identifiers to the ui, which was breaking pages and sessions. I apologise for the inconvenience.

>>8795

>>8800

Yeah, should all be fixed today. I apologise for the inconvenience. Please let me know how v306 works out for you.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/09/18 (Wed) 17:12:02 4e6ed3 No.8806

>>8804

> which would swiftly be followed by creating Tumblr and Deviant Art accounts.

hahahaha

based dev

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.

Anonymous 05/09/18 (Wed) 19:11:35 93869a No.8807

>>8793

just to point out some of the fetishes

feet - the most common fetish in the world apparently, thr thought behind it is largely an evolutionary one. a fucklong time ago good feet meant you may last longer then someone with bad feet, and because of how long human history made good feet an important aspect it's something that is a byproduct of a time when they mattered far more than they do now.

stuffed toys/inanimate objects - this is generally linked to furries due to what most stuffed toys are, animals/anthropomorphic in some way

scat - what causes this origin I have no idea, however we use to be a very smell heavy species if I remember right, so a bit of that crossing in the wrong areas may cause this to develop

diapers - mostly associated with roleplay, but also with scat, its a fetish that comes along with liking the other a lot of the time.

as for the imprinting, I can only speak for myself. laws in america make it so if you have a piece of porn, and nothing suggests its illegal in any way, but someone can verify a person involved was 17, you are going to jail, there are no clauses for they looked over 18, and pretty much the entirety of commercial porn where you are given security in 'they are all over 18' inst attractive to me at all. in comes hentai and drawings where someone can be drawn perfectly and attractive. even through the law bounces back and forth making age matter or not for drawings, it's still better then what the alternative is for real people where I only really like amature porn.

from anime you can easily jump over to furry porn, but its that move from just liking furry porn to going to a furry convention that is a hard line to cross, and this is where furries have always been deviant comes into play.

while an anime convention will gather people who jerk off to sailor moon, it will also just gather people who like the shows and manga, so its very easy for them to distance themselves from a fringe element

but a furry convention… its pretty much a gathering of people openly admitting to being sexualy attracted to kids cartoon characters. the people with the most disposable income to go to events, or keep merch booths open there are usually the most deviant ones who have nothing else to spend money on, so you will get a glut of them, the people who make no attempt to fit in in society, and out go the normal norms and in comes the 'oh, there is a bag of used adult diapers just sitting there… this is normal for them' mentality. the people who attended either get use to it, or fuck off, which spirals everything to be more deviant.

Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.