Mass media downloader

Name
Email
Subject
Comment *
File	Select/drop/paste files here
Password	(Randomized for file and post deletion; you may also set your own.)
* = required field	[▶ Show post options & limits] Confused? See the FAQ.

Flag
Oekaki	Show oekaki applet (replaces files and can be used instead)
Options	Do not bump (you can also write sage in the email field) Spoiler images (this replaces the thumbnails of your images with question marks)
Allowed file types:jpg, jpeg, gif, png, webm, mp4, pdf Max filesize is 16 MB. Max image dimensions are 15000 x 15000. You may upload 3 per post.

File (hide): 1100acac76f7861⋯.jpg (116.9 KB, 580x435, 4:3, 1412758025081.jpg) (h) (u)

▶Mass media downloader Anonymous 07/02/18 (Mon) 17:22:13 No.937733 [Watch Thread][Show All Posts]

Does anyone have a good mass downloading alternative to downthemall? Since legacy firefucks addons arent allowed anymore i can't mass download media from threads anymore.

I'm on manjaro, so if you've found something for linux that works on firefox that's fine too.

>inb4 stop using firefox

>inb4 install the meme linux

▶Anonymous 07/02/18 (Mon) 17:24:45 No.937735>>937737 >>937741 >>937743

>he cant make a bash script to download pics and webms

all you need is curl and sed, you retard

▶Anonymous 07/02/18 (Mon) 17:35:15 No.937737>>937743

>>937735

This.

▶Anonymous 07/02/18 (Mon) 17:48:50 No.937741>>937743

File (hide): b988228fa51216f⋯.jpg (74.98 KB, 1024x1024, 1:1, what_you_said.jpg) (h) (u)

>>937735

Never done it before, everyone has to start somewhere anon.

▶sage sage 07/02/18 (Mon) 17:55:02 No.937743

>>937741

Don't listen to >>937735 or >>937737. Read this: https://www.gnu.org/software/wget/manual/wget.html

Anyways this thread is shit.

sage

▶Anonymous 07/02/18 (Mon) 18:24:55 No.937754

install gentoo

▶Anonymous 07/02/18 (Mon) 18:34:40 No.937758>>937760

Nah, fuck it...

First, fetch the page with the program of you choice, then use this:


sed -r "s@(https://media.8ch.net/file_store/[0-9a-f]+.(jpg|jpeg|png|gif|webm|mp4))@\n\1\n@g" |\
grep -E "https://media.8ch.net/file_store/[0-9a-f]+.(jpg|jpeg|png|gif|webm|mp4)" | sort | uniq

It will output the links to all the files in a thread that you can use as input to your download manager.

Example:


wget -O - <thread_address> | /path/to/script | wget -i -

▶Anonymous 07/02/18 (Mon) 18:41:32 No.937760>>937784 >>937805

>>937758

>all this shit

https://www.gnu.org/software/wget/manual/wget.html#Types-of-Files

▶Anonymous 07/02/18 (Mon) 19:40:27 No.937784>>937788 >>937880

>>937760

>OS is built around text utilities making this task easy as hell

>let's replicate it in our tool!

Wget is for bloat lovers, curl is the solution.

▶Anonymous 07/02/18 (Mon) 19:50:02 No.937788>>937789

>>937784

>Wget is for bloat lovers


    2963 text files.
    2912 unique files.                                          
    1901 files ignored.

github.com/AlDanial/cloc v 1.76  T=1.45 s (737.7 files/s, 189698.3 lines/s)
---------------------------------------------------------------------------------------
Language                             files          blank        comment           code
---------------------------------------------------------------------------------------
C                                      442          23288          32185         112757
m4                                      19           1431           1932          16507
Python                                  17           2346           1863          15194
Perl                                    44           2405           3202          13497
C/C++ Header                           199           4446           9963          12399
Markdown                                24           1160              0           3303
D                                      218            332              0           2896
CMake                                   27            710            682           2685
Bourne Shell                            22            410            545           1448
make                                    29            293            574           1367
DOS Batch                                6            249            266           1313
C++                                      4            138            149            495
Pascal                                   3            175             71            464
YAML                                     5             14             23            414
Visual Basic                             1             30             60            341
Windows Resource File                    2             17             47            110
Gencat NLS                               1              2              0            109
Bourne Again Shell                       1             17             23             97
Windows Module Definition                1              1              0             60
awk                                      2              6             52             58
Lisp                                     1              1              2              7
---------------------------------------------------------------------------------------
SUM:                                  1068          37471          51639         185521
---------------------------------------------------------------------------------------


    5842 text files.
    5820 unique files.                                          
    5519 files ignored.

github.com/AlDanial/cloc v 1.76  T=2.06 s (161.5 files/s, 36546.7 lines/s)
--------------------------------------------------------------------------------
Language                      files          blank        comment           code
--------------------------------------------------------------------------------
C                                50           6330           8762          32858
Perl                            105           1834            880           7470
Python                          103           1456           2325           5207
C/C++ Header                     43            710           1164           1526
Bourne Shell                     10            307            304           1303
m4                                6            150            205            887
make                              8            125            170            460
Bourne Again Shell                3             46             67            160
YAML                              2             11             90             74
lex                               1             29             65             73
Markdown                          1             21              0             63
--------------------------------------------------------------------------------
SUM:                            332          11019          14032          50081
--------------------------------------------------------------------------------

▶Anonymous 07/02/18 (Mon) 19:51:11 No.937789>>937791

>>937788

sorry i fucked it up (im drunk). first one is curl

▶Anonymous 07/02/18 (Mon) 19:53:55 No.937791>>937793

>>937789

Curl is also a library. You can't compare like that. Try comparing the binary sizes (which isn't perfect, but clearly better that CLOC, here):


> du -h /usr/bin/{wget,curl}
512K    /usr/bin/wget
240K    /usr/bin/curl

▶Anonymous 07/02/18 (Mon) 19:58:01 No.937793>>937794

>>937791

>512K

So? Hello world in rust is 452K. I don't care about a few hundred kibibytes of bloat.

▶Anonymous 07/02/18 (Mon) 20:05:21 No.937794>>937795

>>937793

You don't seem to understand the concept of "context". Maybe your simple mind will understand better "feature creep" than "bloat".

▶Anonymous 07/02/18 (Mon) 20:06:41 No.937795>>937798

>>937794

>feature creep

<DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, Telnet and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, HTTP/2, cookies, user+password authentication (Basic, Plain, Digest, CRAM-MD5, NTLM, Negotiate and Kerberos), file transfer resume, proxy tunneling and more.

hmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm....

▶Anonymous 07/02/18 (Mon) 20:17:26 No.937798>>937799 >>937869

>>937795

Did you even read the post that started it all? I'm tired of faggots, keep using wget if you like it. You should use httrack for even more FEATURES!!!1!

▶Anonymous 07/02/18 (Mon) 20:19:30 No.937799

>>937798

>You should use httrack

I do, actually. It's pretty good.

▶Anonymous 07/02/18 (Mon) 20:41:55 No.937805

>>937760

This. Don't listen to the retards who insist on doing it the cool 1337 way, you don't need to do that.

▶[k00l]shamoanjac!!v.CX2tveU. 07/03/18 (Tue) 02:12:47 No.937869

>>937798

>scope of project is to provide as many protocols as possible

>this is what they actually do

<FEATURE CREEP

Do you even understand what feature creep means? A mail client having a calendar feature, a web browser having a source code editor, a text editor displaying images, is feature creep, because it's outside of their scopes. A net client implementing widely used protocols isn't feature creep, because it's within its scope. Now, if curl compiled javascript or some shit like that, it would be feature creep.

▶Anonymous 07/03/18 (Tue) 02:52:50 No.937879

>Not using legacy firefox

Good goyim, fear the (((exploit))) boogryman

▶Anonymous 07/03/18 (Tue) 03:04:25 No.937880

>>937784

Your life is bloat. Kill yourself.

/tech/ - Technology★

General

WebM

Theme

User JS

Do not paste code here unless you absolutely trust the source or have read it yourself!

Favorites

Customize Formatting

Filters