I want to use machine learning on news articles to predict future events. Does anyone know how to efficiently create a database that has full length articles with date, source, readers, and biases?We can use existing databases to quicken the search.

Email
Comment *
File	Select/drop/paste files here
Password	(Randomized for file and post deletion; you may also set your own.)
* = required field	[▶ Show post options & limits] Confused? See the FAQ.

Flag
Oekaki	Show oekaki applet (replaces files and can be used instead)
Options	Do not bump (you can also write sage in the email field) Spoiler images (this replaces the thumbnails of your images with question marks)
Allowed file types:jpg, jpeg, gif, png, webm, mp4, pdf Max filesize is 16 MB. Max image dimensions are 15000 x 15000. You may upload 3 per post.

File: ed5c85f1ecf832c⋯.jpg (230.82 KB, 1023x765, 341:255, ed5c85f1ecf832c36e9d7d1b27….jpg)

Anonymous 03/03/19 (Sun) 09:08:01 No.1037360

I want to use machine learning on news articles to predict future events. Does anyone know how to efficiently create a database that has full length articles with date, source, readers, and biases?

We can use existing databases to quicken the search.

Anonymous 03/03/19 (Sun) 09:16:21 No.1037361

>I want to use mana to cast magic spell

>I have no idea what magic is or if spells are even possible or if mana even exists

>my very first question is: how can I optimally store mana in amounts equal to the yearly output of a hundred witches?

>serious answers only please: any inefficiency in my storage mechanism will cripple my magic spells later on

use SQLite.

Anonymous 03/03/19 (Sun) 11:19:57 No.1037394

you're gon need some hosting for that big database of yours my man. Try this one https://mzunguhosting.ml/

Anonymous 03/03/19 (Sun) 11:52:53 No.1037404

>>1037394

I need your credit card to verify ur legit

Anonymous 03/03/19 (Sun) 11:55:56 No.1037406

>>1037361

>use existing databases

>else web scraping

Those are my options so I'm hoping there's a good database available

Anonymous 03/03/19 (Sun) 12:31:01 No.1037410

>>1037406

there's https://www.lexisnexis.com/en-us/products/nexis.page

you'll have an easier time selling your snake oil if you base it on the twitter firehose, though.

Anonymous 03/03/19 (Sun) 12:35:00 No.1037412

>>1037410

Thanks m8

As for Twitter do they allow unlimited use API for less than millions of dollars?

Anonymous 03/03/19 (Sun) 14:18:28 No.1037439

>>1037412

no idea, but you could prototype your idea off the fediverse instead.

example conclusion: "tranny posts > 3% --> civilization trending towards collapse."

Anonymous 03/03/19 (Sun) 15:55:39 No.1037454

Similar programs already exist and are used to guide automated stock trading algorithms.

Anonymous 03/03/19 (Sun) 20:06:58 No.1037578

>>1037454

that's what I was planning on

Are these programsavailable to the public?

Anonymous 03/03/19 (Sun) 20:09:00 No.1037579

please look into hadoop sir,,

Anonymous 03/03/19 (Sun) 20:12:30 No.1037581

>>1037579

Hadoop just runs computers parallel in a network. Mapreduce part of it might help some after I get the data which is my first problem

Anonymous 03/03/19 (Sun) 20:24:48 No.1037586

>>1037439

I've heard of mastadon but not the fediverse before. I'm tentative on using social media/blogs since there are many shills that can inflate topics. I might do it anyway to see what I can make of it.

I was planning correlating key words such as nouns with corresponding positive/negative words. Eventually i plan on developing a database that stores simple concepts but as with any project you've got to get it working before you perfect it

Anonymous 03/03/19 (Sun) 23:03:02 No.1037690

>>1037360

>I want to use something that doesn't completely and accurately report current events to predict future events

Anonymous 03/04/19 (Mon) 03:42:25 No.1037760

>>1037360

https://www.halfpasthuman.com/ALTA)_how.html

you're literally asking how to make the web bot report.

this shit is older than you.

Anonymous 03/04/19 (Mon) 04:44:50 No.1037778

>>1037360

I've seen a guy's database that predicts future events based on an amount of spider crawlers that crawl all over the internet. It indexes and bases its conclusions on a tremendous amount of data. Sadly you have to pay to see the reports (they are monthly and there's a maganize of them too), but the archived (~1 year ago? 2? 3?) reports are free.

And also sadly, i forgot its name, although it is something like "webcrawler" or you can just search for web spiders and/or crawlers.

Anonymous 03/04/19 (Mon) 05:43:08 No.1037787

>>1037586

>I'm tentative on using social media/blogs since there are many shills that can inflate topics.

and your plan was to use news articles?

Anonymous 03/04/19 (Mon) 07:59:17 No.1037824

>>1037360

Hmm... This kind of sounds like this one game...

Anonymous 03/04/19 (Mon) 08:08:06 No.1037827

>>1037787

They may be biased but they're more influential than 10,000 nobodies on Twitter with 0 followers

Anonymous 03/04/19 (Mon) 08:33:02 No.1037831