528e0b No.1553
Reposting from >>>/tech/457184
Maybe relevant to your interest.
Japan did it.
Automatic anime images tagging with neural networks.
THE FUTURE IS NOW
http://illustration2vec.net/
http://illustration2vec.net/papers/illustration2vec-main.pdf
____________________________
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
528e0b No.1554
Thanks for this link, it looks great! You might have seen that it can also run offline (which is how I prefer this sort of thing) and has a python module, so it would be very easy for me to integrate. I plan to add a 'suggested tags' control to pop up aside the manage tags dialog, which this 'tag feed' could optionally plug into very easily.
Furthermore, it might be something we can feed back into in future, improving their pre-trained models with our own growing collection of tag-feature metadata. I am not sure what data they used to generate their models–I assume a sample of some booru–but they say it knows about 1,539 different tags, as listed at http://illustration2vec.net/models/tag_list.json.gz (including "no bra" and "cum in pussy", jej). If this is just a proof of concept, and there is some easy way to train your own models, I think we can extend it to do a bit more than that!
>tfw within ten years, advanced neural networks will spend trillions of CPU cycles hyper-accurately scanning all human media for different types of semen-related metadata.
I will add this to the list of new things to add, and put it up on the vote list once I have time to work on something new.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
528e0b No.1555
Fucking awesome.
>anime images
It's a shame that it won't run on my unorganized to fuck reaction folder, but it's still useful for mega:///#F!Y0pTnZiY!g0ALqjpneHo-Lr2cJJaBuA
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
528e0b No.1556
>>1554
Hydrus dev you really outdo yourself. I hope you know it's appreciated. I was just sharing it out of interest but you considered adding it as a feature and even contributing back, that's really nice.
Also I knew that not organizing my image collection manually would pay off eventually. h-heh
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
528e0b No.1561
>>1556
Thank you. I am glad to help out a fellow Anon.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
528e0b No.1568
>>1553
>>1554
Fuck yeah, I heard about this the other day and came here to mention it, and based Hydrus dev is already on the case.
I look forward to seeing it implemented in the future. 'need mo tag metadata fo mah program
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
528e0b No.1572
>>1554
>hydrus dev already planning on implementing this
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
528e0b No.1670
Would be awesome to see this implemented.
Some kind of ability to retrain with new data sets, if at all feasible, would greatly add to the value too. The current net is pretty solid for anime stuff, but if you're trying to work with a different kind of image set it's only so so. And even with anime, the data used to train the current system was presumably a lot smaller than it could have been, due to being a proof of concept and all.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
528e0b No.4765
Since this post, hydrus has had some better video support, maybe it's worth referencing other automatic taggers here.
This one takes a video file and creates a list of tags+timecodes, it was initially built to create supercuts of video files containing just the parts that contain the tags you specify. I think it can be used with images as well.
https://github.com/ryanjay0/miles-deep
Yahoo made one for images that just judges what is and isn't NSFW, doesn't tag them any further though.
https://github.com/yahoo/open_nsfw
Warning: nightmare fuel
https://open_nsfw.gitlab.io/
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
528e0b No.5433
From >>>/v/12285254
https://paintschainer.preferred.tech/
It would be interesting to one day have a "colourise" option in image-viewers in the same way we have things like rotate, scale, filters, etc.
Would be very neat for black and white manga.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
874a5b No.6121
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
5fe099 No.6129
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
8c76ab No.6130
>>6129
That Html-table-parser-python3 that you're using crashing when attempting to be installed, when looking for a "setup.py" in the location it's looking for.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
4ed57b No.6133
>>6130
fixed this, there is problem due to that package is not yet on pypi.
so now it have to be cloned first and install it from source.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
8c76ab No.6137
>>6133
Attempting to install the dependancy "Html_Table_Parser_Python3" the way you've got it written down (pip install git+git://github.com/schmijos/html-table-parser-python3.git ) causes pip to throw an error with setup.py not being found. What you actually want is (pip install git+git://github.com/rachmadaniHaryono/html-table-parser-python3.git ) works fine. Running the code as-written from there throws this error message:
Traceback (most recent call last):
File "/usr/local/bin/i2vec-cli", line 11, in <module>
load_entry_point('i2vec-cli==0.1.0', 'console_scripts', 'i2vec-cli')()
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 561, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 2631, in load_entry_point
return ep.load()
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 2291, in load
return self.resolve()
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 2297, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "/usr/local/lib/python2.7/dist-packages/i2vec_cli/main.py", line 15, in <module>
from html_table_parser import HTMLTableParser
File "/usr/local/lib/python2.7/dist-packages/html_table_parser/init.py", line 1, in <module>
from .parser import HTMLTableParser
File "/usr/local/lib/python2.7/dist-packages/html_table_parser/parser.py", line 13, in <module>
from html.parser import HTMLParser
ImportError: No module named html.parser
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
7a9094 No.6145
>>6137
You may want to use python3, because the program only support python 3 version
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
8c76ab No.6149
>>6137
You disgust me
Okay, so after rerunning everything in python 3.5 this time, reinstalling those dependancies, and getting Geckodriver to start, i finally got it running as intended. And…
User@Anon's computer ~ $ i2vec-cli '/path/to/hydrus network/db/client_files/f80/80bf66b390fbb1554ff556a1e878cdd2acffe42a05f83dafbb5cf9695e1c9a11.jpg'
path:80bf66b390fbb1554ff556a1e878cdd2acffe42a05f83dafbb5cf9695e1c9a11.jpg
[[['#', 'General Tag', 'Confidence', '']],
[['#', 'Character Tag', '', '']],
[['#', 'Copyright Tag', '', '']],
[['#', 'Rating', '', '']]]
Running the same image with –format Hydrus nets you
Traceback (most recent call last):
File "/usr/local/bin/i2vec-cli", line 11, in <module>
load_entry_point('i2vec-cli==0.1.0', 'console_scripts', 'i2vec-cli')()
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/i2vec_cli/__main__.py", line 83, in main
print(convert_raw_to_hydrus(tags))
File "/usr/local/lib/python3.5/dist-packages/i2vec_cli/__main__.py", line 63, in convert_raw_to_hydrus
result.append('rating:{}'.format(dict_result[key][0]))
IndexError: list index out of range
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
981c1c No.6155
>>6149
thanks for the bug report, now v0.1.1 should fix that.
can you check if there is any error?
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
8c76ab No.6157
>>6155
No programming errors. Still, the program doesn't actually give out any sort of tags. Testing it with pic related on my own gives out a whole bunch of valuable tags (Even correctly identifying it as a 2hu, which I honestly wasn't expecting.) whereas the command line tool gives the following result
User@Computer ~ $ i2vec-cli '/path/to/file/54d5bb6151a4263ee67e904041366bf6c52ed46b0006ef604da1f77184c3c0d3.jpg' --format hydrus
path:54d5bb6151a4263ee67e904041366bf6c52ed46b0006ef604da1f77184c3c0d3.jpg
2017-06-15 03:53.28 key v=Rating
Good on you for responding to everything, by the way.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
8c76ab No.6158
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
0599d7 No.6162
>>6158
unfortunately i can't reproduce it
https://github.com/rachmadaniHaryono/i2vec_cli/commit/94b422330776d8710b0ee686eda4727743295a67
but with that version, it may help me debug it.
the program now support url and i can get the tag from that url directly
i2vec-cli -nc https://media.8ch.net/file_store/54d5bb6151a4263ee67e904041366bf6c52ed46b0006ef604da1f77184c3c0d3.jpg --format hydrus
url:https://media.8ch.net/file_store/54d5bb6151a4263ee67e904041366bf6c52ed46b0006ef604da1f77184c3c0d3.jpg
1girl
blonde hair
solo
yellow eyes
long hair
breasts
smile
open mouth
arms up
armpits
cleavage
arm up
:d
looking at viewer
star
v
from above
foreshortening
pointing
series:touhou
rating:safe
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
8c76ab No.6165
>>6162
User@Computer ~ $ i2vec-cli https://media.8ch.net/file_store/54d5bb6151a4263ee67e904041366bf6c52ed46b0006ef604da1f77184c3c0d3.jpg/54d5bb6151a4263ee67e904041%E2%80%A6.jpg --format hydrus
url:https://media.8ch.net/file_store/54d5bb6151a4263ee67e904041366bf6c52ed46b0006ef604da1f77184c3c0d3.jpg/54d5bb6151a4263ee67e904041%E2%80%A6.jpg
User@Computer ~ $
Still gets me nothing, even after upgrading Requests. I can see the firefox window open up, and the file being uploaded, but the window closes far too fast to get any of the tags away from the window. I'd suggest creating a small delay between finishing to upload everything and getting the tags, since that's where it seems to stumble upon.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
407d72 No.6170
>>6165
https://github.com/rachmadaniHaryono/i2vec_cli/commit/68f76b484cca5e4603b6ce6fd36ebefc3e7a1d66
now with close delay
i'm really interested about this error and want to fix it for the problem you have
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
17b68b No.6171
>>1554
If you add something like this to the program, I have to ask you if you can also make a 'user to approve tag' section, and if an image is tagged exclusively with tags from the algorithm, a mandatory 'tag manually' tag
This would just keep potential crap tags out and also denote that the image may have more tags that are obvious but were not tagged.
—
it's only after I wrote this that I seen how old the op was, but point stands, would rather have something like this in a quarantine to confirm section rather than seamless
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
1a3057 No.6173
I tried installing and I got the same problem as >>6137 only I don't know how to do it with python 3. Can you tell me how?
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
7964f2 No.6174
>>6173
use
pip3 install -e .
to make it explicitly install with python3. you may want to use `–user` if you don't want to sudo the command.
also you may have to uninstall the previous installation
pip uninstall .
i will put it in doc
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
1a3057 No.6175
>>6174
Thanks! I managed to install it. I also needed to install geckodriver btw.
Now I try it with the 2hu of >>6158 and I get (same if I do >>6162 >>6165):
$ i2vec-cli 2hu.jpg
path:2hu.jpg
[[['#', 'General Tag', 'Confidence', '']],
[['#', 'Character Tag', '', '']],
[['#', 'Copyright Tag', '', '']],
[['#', 'Rating', '', '']]]
Also not sure if it's supposed to be like that but it opened a firefox window with http://demo.illustration2vec.net/ and then it closed that window again
Also it saves a geckodriver.log every time I use it
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
7a9094 No.6177
>>6175
>Thanks! I managed to install it. I also needed to install geckodriver btw.
i will implement option to use other driver such as phantom js or chrome
or maybe another feature to remove geckodriver log as well
>Also not sure if it's supposed to be like that but it opened a firefox window with http://demo.illustration2vec.net/ and then it closed that window again
yes becaues firefox is the default driver for the program
for this picture i'm still looking for it.
there is 'close delay' feature which will pause the program before the program end, and user can examine if anything is wrong with the upload.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
8c76ab No.6180
>>6170
Thanks for the close-delay, that really helps with debugging this stuff out. Here's what i got so far.
$ i2vec-cli '/path/to/2hu/image/Weapon accuracy.jpg' --close-delay 60
path:Weapon accuracy.jpg
[[['#', 'General Tag', 'Confidence', '']],
[['#', 'Character Tag', '', '']],
[['#', 'Copyright Tag', '', '']],
[['#', 'Rating', '', '']]]
I took a screencap of what was going on with the open Firefox window so you could see what's going on there. As you can see, it wasn't what i thought, that the tags just weren't being displayed,but it's just that grabbing the table and displaying the contents in a command line doesn't happen.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
04b0ca No.6187
>>6180
that is quite unexpected.
if the table is not recognized, the 4 header row (general tag, character tag, etc) should also not recognized
now i add `dump html` option to dump the table to html file. if it is correct it should write html and when that file opened in browser it should be like screenshot
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
8c76ab No.6193
>>6187
The option, if anybody is checking, is –dump-html.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
d25c47 No.6278
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
1e2012 No.6286
>>6278
thank you for the example, i will implement it
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
ade5db No.6287
http://illustration2vec.net/papers/illustration2vec-main.pdf
>We browse several web services (e.g., Danbooru and Safebooru) and collect 1,287,596 illustrations and the associated metadata
https://nyaa.pantsu.cat/view/887969
>http://gelbooru.com/ ID:1 to ID:3470395
Seems like the model could be improved if someone has the storage and compute power. Then there are 1.7 million more from zerochan. There are slso some site rips on rutracker, for example a partial one of sankakucomplex.
https://rutracker.org/forum/viewtopic.php?t=3641643
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
c9796e No.6295
Okay, so i finally managed to make the dump-html thing work. The secret was to not to wait at the
> /usr/local/lib/python3.5/dist-packages/i2vec_cli/__main__.py(61)get_tags()
-> dump_html_to_file(bb_tag.html)
(Pdb)
prompt like i was expecting, but to type in "BB_tag.html". Here's the Html prompt that was spat out. This was copied and pasted from my terminal, but it still reads as a valid web page when copied and pasted into html.
>The body was too long.
Dammit, codemonkey.
https://pastebin.com/RC0RfLdb
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
f1c594 No.6316
>>6180
>>6295
the update have been uploaded. it is now using only requests libray and parsing only json response from the server.
it is using method mention by >>6278
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
c9796e No.6332
>>6316
Traceback (most recent call last):
File "/usr/local/bin/i2vec-cli", line 11, in <module>
load_entry_point('i2vec-cli==0.1.1', 'console_scripts', 'i2vec-cli')()
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/i2vec_cli/__main__.py", line 143, in main
session = Session(driver=driver)
TypeError: __init__() got an unexpected keyword argument 'driver'
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
85445d No.6333
>>6332
>>6193
hi, i just made server feature for this program.
this also fix typeerror on init method.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
c9796e No.6426
>>6333
Cool trips.
Traceback (most recent call last):
File "/usr/local/bin/i2vec-cli", line 11, in <module>
load_entry_point('i2vec-cli==0.2.0', 'console_scripts', 'i2vec-cli')()
File "/usr/local/lib/python3.5/dist-packages/pkg_resources/__init__.py", line 561, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/local/lib/python3.5/dist-packages/pkg_resources/__init__.py", line 2649, in load_entry_point
return ep.load()
File "/usr/local/lib/python3.5/dist-packages/pkg_resources/__init__.py", line 2303, in load
return self.resolve()
File "/usr/local/lib/python3.5/dist-packages/pkg_resources/__init__.py", line 2309, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/usr/local/lib/python3.5/dist-packages/i2vec_cli/__main__.py", line 20, in <module>
import peewee
ImportError: No module named 'peewee'
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
c085d2 No.6428
>>6426
peewee already on requirements.txt. but on next version all package required will be on setup instead of requirements
there is ongoing index page work in progress
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
c9796e No.6444
>>6428
running pip install peewee seems to have fixed the "tags not being read from the page" problem. Consider all of these bug reports fixed.
>>6332
>>6295
>>6180
>>6175
>>6165
>>6157
>>6149
>>6137
>>6130
I'm gonna start using this a lot now, thanks.
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
63a5e1 No.10671
Necrobumping for great justice
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
706f3c No.10689
Disclaimer: this post and the subject matter and contents thereof - text, media, or otherwise - do not necessarily reflect the views of the 8kun administration.
