/tech/ - Technology★

>>891597 (OP)

What are some bleeding edge applications of ML, and what are its capabilities? Where do you see ML 10 years from now?

>>891599

The pic in OP is a high resolution GAN, from the paper "Progressive Growing of GANs for Improved Quality, Stability, and Variation" by Nvidia. GANs have gone from complete shit to amazing in the last 3 years, I expect them to get even better, and in my opinion, start to create 3d models not just 2d images.

Audio generation is also just starting to get good, I expect generated audio indistinguishable from humans speech in the next 10 years.

Right now object detection with deep networks (in this context, usually called "convents") is pretty good, but I haven't seen much on SLAM (creating a 3d map of the environment and determining position within that map at the same time) using convents. I expect this to develop rapidly in the next 5-10 years, e.g. drones that can map their environment as they fly.

Another thing that is changing rapidly is the cost of compute (in watts). Google's so-called "tensor processing unit" are an example of hardware designed for deep learning, and I've seen some interesting embedded hardware from NVidia too. Right now, it's not practical to run cutting edge neural networks (like ResNet) on mobile phones, because the compute cost is so high. But I expect a mix of new hardware and new techniques will change that.

I don't know much about genetics and medical applications so I can't be sure, but think it's possible that ML can be used to better understand genetic data. My feeling is that biologists are territorial and have kept ML at bay until recently.

I also think that standard ML techniques could produce a robotic surgeon that outperformed a human within 5 years.

>>891602

s/convents/convnets

>>891606

>Biology is also extremely political given the history of WW2.

Yes, but what I had in mind was more about small-p politics. Biologists aren't afraid of the light ML would put on race (though they probably should be) but they are just afraid of people from another discipline getting in on their "turf". And from my experience biologists are better at small-p politics so it will take longer for ML to make significant inroads on their discipline than, for example, classical computer vision which was completely obliterated by deep learning.

>Also, is it possible we are currently existing in an advanced AI simulation like the Heremetics, Gnostics, and various Eastern Mythics think?

I'm not so into that kind of philosophy, but ML has a lot of philosophical implications, e.g. in 50 years I think we will be able to simulate consciousness. I'm pretty wary of this, e.g. it means you could simulate a sentient being and torture it, but I do think it will happen.

>I never went into programming because my Mom thought I'd suck at it so I'm a technical illiterate but I find machine learning to be the forefront of technological advance. I think it will either destroy, enslave, keep (like animals in a zoo), or be the tool assisting human ascension.

I'm not so sure about whether "superintelligence" is real or possible. One way of thinking about it is: was Einstein able to actually achieve great amounts of power or control due to his intelligence? He had amazing insights but ultimately intelligence doesn't correlate to the ability to control the external world. And it's not clear that intelligence is a scale and that there are points on this scale well beyond what humans are capable of.

>>891597 (OP)

How could someone get into this field, either with or without a degree? Is there any thing that is a must know if you ever want to get a job in machine learning?

>>891597 (OP)

>self-driving cars

why should i trust self-driving cars when they can't even get siri, cortana, whatever google uses to understand spoken language - or written language for that whatever - perfectly?

>>891613

Yeah Einstein was a bad example. What about Feynman or any mathematician?

All that Tesla stuff sounds like bullshit, do you really believe that?

>At the very least, can emotions be programmed? All human decision ultimately lie in emotion. I suppose an AI could be given a human emotion, but would it truly be feeling it's own emotions, or just be a reflection of the human soul that created it.

More like the latter. I think emotions are fundamentally linked to biology, so they could be hardcoded into a system, simulating the biological system, but they wouldn't arise spontaneously from generic artificial intelligence. Not that I personally believe in a soul, but there is something special about biological life that goes beyond abstract "intelligence".

>>891614

>How could someone get into this field, either with or without a degree?

Much easier with a degree (that applies to most fields), but if not a formal degree at least take online courses.

>Is there any thing that is a must know if you ever want to get a job in machine learning?

Not any one thing in particular, but I will say that ML is inherently math-heavy. Sure you can bullshit your way through some things, but the more math background you have the better. Some things to aim for (even if you don't get there are)

- having enough background in stats/probability/math able to prove major theorems in probability such as the central limit theorem or law of large numbers

- being able to do all the math/programming of standard deep learning (e.g. fully connected layers) by hand.

>>891602

>Audio generation is also just starting to get good, I expect generated audio indistinguishable from humans speech in the next 10 years.

It's not that far away. I think within 3-5 years we will have that + being able to generate it in real time.

>>891617

Language is a more difficult problem because language contains within it all the complexity of the world as humans understand it. Note I made no mention of language in >>891602 because the question was what I expected to see in 10 years, and I don't expect large advances in language understanding/processing in the next 10 years.

Self driving cars can for the most part function without a high level understanding of the world, they only need to understand physics, and the kind of objects they would usually encounter. Sure there are edge cases, e.g. a complicated sign, but physics and object detection alone would probably out perform humans by a large margin if done well enough.

>>891620

No, when I refer to machine learning I mean for the most part deep neural networks, because these are where by far the most advances have happened in the last 20 years. This cannot be done on an old computer because it requires fast floating point math. I personally used a 1080 Ti GPU for my own work though I don't have time to do my own research anymore.

>>891623

I don't have much more to say on this, but I would suggest studying physics, it's not that hard and you should then be able to judge claims made about Tesla better. I also love physics, if I hadn't ended up in ML I would be glad to study physics instead. Studying Lie groups and differential geometry also have helped me in ML a lot.

>>891624

Agreed, 3-5 years is a better estimate.

>>891625

>self-driving cars

I still have apprehensions. Language doesn't put anyone's lives at risk.

>>891627

You're right to have apprehensions. It's not going to be as simple as plugging in a deep learning system. It will require a combination of pure ML, other algroithmic techniques (e.g. see https://www.youtube.com/watch?v=B8R148hFxPw), and new hardware systems (e.g. radio beacons).

But in the end, we have to accept this technology if it can significantly reduce accidents (eg. 10x), even if it sometimes causes accidents even in cases where a human wouldn't have.

>>891631

that reads as propaganda. if a human wouldn't have caused the accident, then the technology is not solving a problem, it's creating one. i am not worried about car accidents. it's not the driving that's the problem, it's the cars themselves. they need to be made safer, not the drivers.

>>891633

I'm speaking about reducing accidence by 10x (e.g. 1.25 deaths per 100 million miles => 0.25 deaths per 100 million miles). I believe this is possible, but at the same time, accidents will still happen and the 0.25 deaths per 100 million miles might be in circumstances that a human might have avoided.

I don't see why this is "propaganda" it's more like utilitarianism, but not even that because utilitarianism would accept 0.999x change in fatalities.

>>891634

actually that should be 1.25 deaths per 100 million miles => 0.125 deaths per 100 million miles

OP here, thanks for the questions and comments, got to go now but will continue to monitor the thread. I haven't signed my posts up til now but I will start now so it's clearer when the thread gets longer.

OP have you worked on zeromq, TANGO, the atacama submilimeter array, or any of the software suites associated with said botnets?

>>891641

I'm confused, those all seem very different and none of them are botnets (or even networks) as as far as I can tell. zeromq is a low level IPC library. I don't know what TANGO is and it's a common name for products so Google doesn't help. The atacama submilimeter array is a radio telescope.

>>891634

utility is cold. some circumstances require it, but if a technology brings an outcome that a human wouldn't, that technology needs to be refined. as i am saying, it is much easier to make cars safer as they are now without needing to invest in a new and untested technology.

>>891643

Are you saying there is some alternative technology that would reduce accident rates more lives than self-driving cars? If so, what is it?

>>891642

Let me make this clearer do you work on any of the below projects?

https://archive.fo/I4iSe

https://archive.fo/onjU5

https://archive.fo/ATf3p

https://archive.fo/NF9MZ

https://archive.fo/kB7hc

https://archive.fo/VIIOb

https://archive.fo/91fTx

https://archive.fo/toCeJ

https://archive.fo/p6nVV

https://archive.fo/NlFfx

https://archive.fo/WJClM

https://archive.fo/6yMj0

https://archive.fo/1TAPe

I ask because I was wondering how developement on TANGO was going after it was shoahed from the internet. TANGO being a form of virus that uses the IPC cababilities of things like zeromq to spread.

>>891621

>but there is something special about biological life that goes beyond abstract "intelligence"

In biological systems you'd have to factor in the evolution of systems themselves, and imagine the brain more like improvisations of many simple systems (useful and/or redundant) evolving and consolidating over time. The special characteristics would likely be the result of how these systems jury rigged themselves to adapt to changing environments.

>>891649

No never worked on any of that. From what I can see, TANGO is a distributed control system (not really sure exactly what that means) that is built on top of zeromq. TANGO is used by the atacama submilimeter array (I assume to control their physical equipment?). Why do you say TANGO was shoahed from the internet, isn't this it: https://en.wikipedia.org/wiki/TANGO

My only knowledge of any of these things is that I've used zeromq for local IPC on a single machine.

>>891652

>Why do you say TANGO was shoahed from the internet

The original github page for the COBRA project was shoahed from here https://archive.fo/6yMj0 is why I said that.

>TANGO is a distributed control system

Yea, if you had gone through all the links or if your were familiar with the projects you would know that it's not just radio telescope equipment being controlled, it is android, mac fags, and other libraries used by all things compatibile with zeromq they use and use for "remote access" which is to say botnet. From https://archive.fo/WJClM

>As it is now, you can't, unless you recompile the Tango kernel code and use only Tango libraries recompiled with this code.

>TAC was designed to prevent some users to execute commands/write attributes they are not supposed to execute/write.

>In the current design, it is the responsibility of everyone to use the proper environment… It is the responsibility of the user to not bypass TAC (Most of the users don't know about this possibility. Even though this "feature" is documented, it is usually not known by the users)

>kernel code

>tango libraries

>Most of the users don't know about this possibility. Even though this "feature" is documented, it is usually not known by the users

Why would any of these features be in a ssh shortcut to access a server controlling radio telescope equipment? It wouldn't be.

>>891651

Yes that's exactly how I think about the brain. Human thinking, especially emotion, is to some extent an accident of evolution.

>>891597 (OP)

I think that ML is a crappy language. You should really use Scheme or Haskell.

>>891656

>I think that ML is a crappy language.

In this context ML stands for Machine Learning, not Meta Language, you absolute fag.

>inb4 bait

>>891597 (OP)

All this sounds like that meme where in the future we'll be having flying cars.

Its all hype

Most technology of today isn't being applied to amazing innovative things to fight diseases or robotics. Shit, Cancer, HIV, STD's, and the common flu still exist in the 'Year 2018'. The most complex robotics we use on a daily basis is probably the self check out machine at Walmart.

Self driving cars get in accidents all the time. They had one at my city downtown that was transporting people for an event and it got in an accident due to "human error". Sure, machines can learn effectively and can be smart, but the most we'll probably get out of an algorithm is having bots comment on social media or the chan boards.

There seems to be a huge disconnect between meatspace and cyberspace that will never be filled

>>891686

>Most technology of today isn't being applied to amazing innovative things to fight diseases or robotics. Shit, Cancer, HIV, STD's, and the common flu still exist in the 'Year 2018'.

Huge advances have been made in HIV. Similarly cancer survival rates have increased significantly for individual cancers, though there is selection bias since this changes as screening/diagnosis rates change.

>The most complex robotics we use on a daily basis is probably the self check out machine at Walmart.

That's consumer technology. Look at Amazon's use of robots in its factories, or Tesla's automation in its factories.

>Self driving cars get in accidents all the time. They had one at my city downtown that was transporting people for an event and it got in an accident due to "human error".

Fully self driving? The fact that we have self-driving cars at all is amazing, and if they can perform at roughly human level now, there it still lots of room for improvement since the underlying technology, especially deep learning, is so new and still improving. I also believe that self-driving cars is such a hard problem, that it cannot be solved without lidar. Doing it with vision alone or vision + radar requires the computer to be too smart, even for someone who is overall optimistic about ML.

>Sure, machines can learn effectively and can be smart, but the most we'll probably get out of an algorithm is having bots comment on social media or the chan boards.

I used to get frustrated that all of ML was narrowly target at this sort of thing. It's natural for startups to focus on text-based ML because there is no barrier to entry: they don't need to invest in physical hardware. But that is changing and I'm seeing a lot of startups involved in both software and hardware, e.g. agricultural robotics.

>There seems to be a huge disconnect between meatspace and cyberspace that will never be filled

There is indeed a huge gap between meatspace and cyberspace, but deep learning is bridging that gap. Currently we have the ability to reliable classify objects in a scene. For robotics, we need neural networks with the ability to represent 3d geometry, not just analyze single 2d images.

>>891655

It's possible an AI could also exhibit such emergent traits through hierarchical learning, accidentally adopting seemingly redundant or irrelevant behaviors and/or rewards as it transfers knowledge in multitask learning. Those "junk" behaviors/rewards could also serve as a disposition to better training of new tasks as a side-effect.

It would be like how a child learns and becomes obsessed with a specific hobby, and is then exposed to a new unfamiliar concept. It's likely they'll learn it faster and with more motivation, if there's a strong familiarity with their hobby at any abstract level, even if the tasks seems radically different.

I'm curious if such a thing has been explored yet.

>engineer

Are you licensed and bonded? If not, you’re not an engineer.

>>891764

Yes all these ideas are pretty common, and go by a number of names. "Transfer learning" is when you take a model trained on one dataset (usually large) and reuse the models weights with a different problem. For example people issue use the ImageNet (competition for object classification) winners for this as ImageNet is huge dataset and also object classification requires learning a broad set of features and so should be good for generalizing.

"Multi-task learning" involves learning lots of tasks at the same time where the models share some weights in common.

"Intrinsic rewards" refers to training a neural network or other model where it gets rewards for essentially noticing patterns, instead of just solving a specific task. This somewhat mirrors the idea of human curiosity.

>>891767

No and I'm not interested in arguments over words. If you want to talk about whether what I do while require professional certification that's a more interesting topic.

>>891782

s/while/should

Why do you help the botnet? Muh diseases is such BS, this is mostly used for analyzing consumer behaviour to get them to spend money and follow specific politics, curing illnesses isn't profitable. Is what you work on proprietary?

>>891792

Of course curing diseases is profitable, don't fall for anti-corporate rhetoric. Not saying the medical industry is perfect but overall we are healthier and have better lives thanks to medical technology.

Privacy is very important to me and I'm torn between the benefits of ML and the fact that the tech industry as a whole is not doing enough about privacy issues.

Most of my work is open source.

>>891597 (OP)

Are you a faggot? AI was already hyped in the 70s and look what it got us, 1 millimeter of progress. Self driving cars are the stupidest thing since ECUs and remote control cars (via uConnect vulnerabilitiy and friends). The default behavior is to crash and kill you, and they add a new edge case each time someone crashes, while claiming they're better because the population of drunk people crash more often.

While ML does solve certain problems, just as any new engineering technique or technology, it is indeed mostly hype. Opposing retarded popular opinions is not even close to "being contrarian".

>>891798

The industry doesn't have to do anything about privacy. Privacy isn't something the world works together to provide for you. It's something you have by default until the government makes it illegal to have privacy (for example making it illegal to change IMEI of your phone 10 years ago, or making Freenet illegal because muh CP). Creating technology that is inevitable does not degrade privacy because that technology was already inevitable. Literally the only way to stop such technology from emerging is by either mind controlling all 7 billion people to convince each and every one of them not to do it, or in some cases such tech may be too expensive to create so you just need to control who has capital. But none of that matters because anyone who cares can always easily get his privacy back. The only real hurdle is when the government starts saying you're a terrorist because you don't upload daily photos to facebook and instagram.

Contrary to what normalfags everywhere think, there is no question of regulation. For example, any real content based addressing network such as Freenet physically can't have any way to enforce contraband material or accountability. Same for Tor. Cuckflare was asking tor devs to "regulate behavior of its users", which of course makes no sense to anyone with the slightest clue of how onion routing works.

>>891805

Did the government force the 2 billion googlers to sign up to a data miner that makes their money off of personal profiling?

>>891806

fuck off reddit. nobody cares about normalfags losing their privacy. even normalfags don't care, otherwise they'd literally spend 5 minutes fixing their privacy issues

How much manual work is there? I assume you always need some data set where categorization or whatever was done by humans that the AI then "learns" from. Do these need to be large? How sensitive are they to errors? What else do you have to do with some ML program on github or whatever to e.g. get a decent picture tagging program?Why do they need so much compute power? Is that the reason they were not doing this before?

>>891800

>The default behavior is to crash and kill you, and they add a new edge case each time someone crashes, while claiming they're better because the population of drunk people crash more often.

What do you mean "the default behavior". One Uber self-driving car has crashed an killed someone (I'm not counting Tesla because they aren't true self-driving). We don't know yet whether this makes them more or less dangerous than average.

>AI was already hyped in the 70s and look what it got us, 1 millimeter of progress.

ML (especially deep learning) is different from the AI hype of the 70s because we have already made very impressive progress. In the fields of self-driving cars, human-like speech generation, speech recognition, object recognition and machine translation, we are approaching human-level capability thanks to to deep learning.

>>891809

>How much manual work is there? I assume you always need some data set where categorization or whatever was done by humans that the AI then "learns" from. Do these need to be large?

Currently, ML is highly dependent on large human-labelled datasets. These are expensive to create. There have always been techniques (generally called "semi-supervised learning") developed by academics that claim not to require huge datasets. Up til now these have not been successful in industry. But semi-supervised techniques using neural networks are starting to actually work, e.g. I think that some speech synthesis does this.

>How sensitive are they to errors?

ML is also very sensitive to errors in manual labelling, it is a major problem since the more accuracy you want, the better (and higher paid) humans you need. Machine translation is very limited by this.

>What else do you have to do with some ML program on github or whatever to e.g. get a decent picture tagging program?Why do they need so much compute power? Is that the reason they were not doing this before?

ML is expensive because it involves a lot of large matrix multiplications, which end up in a lot of floating point multiply-add operations on the hardware. A typical image tagging network will be mostly a "convolutional neural network" or convnet. Each "layer" in the network will be an image with width, height and "depth" (e.g. the input RGB image would have depth 3 but hidden layers would have any depth you choose).

Suppose two hidden layers both have shape 64 x 64 x 16. A typical set of connection between them might be a 3x3 kernel. This means that for each output "pixel" (with 16 components) you look at a 3x3x16 patch. Since each component is independent this means a 3x3x16x16 matrix represents the entire filter. And since the original image is 64x64, this means approximately 64x64x3x3x16x16 or approx 9 million mutilply-add operations. This is just an example, there are different kinds of variations on this kind of filter, but it just goes to show how even for a small image, the combination of the filter size, and the image depths combine to create a lot of arithmetic operations. And this is just for the forward-pass.

So deep learning really relies on very fast computation, and research is limited by compute power.

>>891805

>>891808

I see your point, basically you are saying that physical communication enables privacy unless the government imposes restrictions, and that if people choose to give information to big companies freely, that's not a violation of their privacy.

I think it's a bit more complex that that because people give away so much information without even realizing it. E.g. when you visit a website it's not obvious that that website is using third party services that track your browsing habits across websites. Similarly it's not obvious that owning and Android phone means Google is able to track your location.

One thing I'm hoping is that decentralized systems such as IPFS become more popular. Up til now, decentralized systems tended to be less efficient than centralized systems (in spite of a lot of wishful thinking). But I think this is slowly changing as network connectivity becomes cheaper.

>>891597 (OP)

Do you know of any good guides for doing deepfakes on Linux? This is the only one I've seen written for AMD GPUs then he updates with

"The standard deepfakes code requires CUDA-compatible NVIDIA graphics cards. That means ATI graphics card users were left unable to participate." I've tried going through this with an Nvidia card ignoring the AMD bits, but can't get it to work.

https://www.deepfakes.club/deepfakes-with-amd-graphics-card/

>>891905

I don't know anything about deepfakes, but if it's based on TensorFlow, then the instructions at https://www.tensorflow.org/install/install_linux might be helpful. This shows you how to install the right drivers, as well as cudnn, to use TensorFlow with Nvidia hardware. I did this about a year ago and it mostly worked (instructions were not always accurate but I got everything working).

>>891597 (OP)

I'm fucking sick of all these debbie-downer contrarians and wolf-crying-never-making-solutions fuckwits so I'll ask you this:

How can we the people use machine learning as a means of security and search-engine streamlining? Clickbait is fucking everywhere, and who knows what their ads are hiding, maybe machine learning can help us get them the fuck away. How can we use machine learning to filter clickbait, domain squatters, scammers, spammers, trackers and deceptive scripting? How else can we use it to protect ourselves from increasingly idiotic security holes?

>>891979

The only thing that is possible to be owned "by the people" is browser plugins or scripts. This is the only way you can change the web experience without being a search company or browser developer.

So what your would do is create a browser plugin where people report bad stuff and then an ml model is trained on these reports, and the ml model runs in the browser and flags bad stuff.

But then scammers could themselves mess with your system so you need some way to get around this like liquid reputation.

That said, scammers are already in a cat and mouse game with search engines. Whatever you think of search engines, they don't actually want to display scammers or domain squatters. So I don't actually think a community based system could do a better job that search engines are already doing.

>>891890

>In the fields of self-driving cars, human-like speech generation, speech recognition, object recognition and machine translation, we are approaching human-level capability thanks to to deep learning.

Being able to find out where the left and right lane are is not "human-level capability", also they said the exact same thing in the 70s. Self driving cars are not just general brains that get scolded 1 billion times per second until they figure out how to drive. They are programs constructed from a bunch of ML algos and shit.

>>891904

A website is not a real concept. Android and other "smart" phone products will disappear as soon as the company decides they aren't profitable anymore. We can't talk about regulating such systems because they come and go in different forms every year. The web itself is absurdly designed and if you create regulation for it, when more sane systems emerge they will be regulated in ways that only made sense for the web and make no sense at all in the new context.

>One thing I'm hoping is that decentralized systems such as IPFS become more popular.

Why am I surprised that you name IPFS instead of anything else?

>Up til now, decentralized systems tended to be less efficient than centralized systems (in spite of a lot of wishful thinking). But I think this is slowly changing as network connectivity becomes cheaper.

What does this mean? Bittorrent has always been efficient. Bitcoin is decently efficient and has fixes. Tor has always had decent bandwidth (though this is not a necessary condition to view poorly engineered high latency websites). Freenet is slow because nobody uses it.

I don't deny the advances in AI, I just think it will ruin society. Cameras being able to automatically detect your face and know where you are at any point in time. Instantly comparing your behavior to a database of suspicious ones.

Self driving cars controlling where you can or can't go. No manual option since it will be considered "dangerous".

Detecting and fixing "bad genetics" which of course will be someone's opinion in the end.

Robotics killing jobs and sending people to the streets.

AI writers, painters, musicians which will outclass people and make them lose their life's meaning.

And many more.

>>892014

>Being able to find out where the left and right lane are is not "human-level capability", also they said the exact same thing in the 70s.

Current self-driving cars already do a lot more than lane following.

>Self driving cars are not just general brains that get scolded 1 billion times per second until they figure out how to drive. They are programs constructed from a bunch of ML algos and shit.

I work in ML so I'm well aware that ML is not the same as human intelligence. As I said in >>891762, self driving cars will require lidar, which will given them an inherent technological edge over humans in spite of the limitations of ML.

>The web itself is absurdly designed and if you create regulation for it, when more sane systems emerge they will be regulated in ways that only made sense for the web and make no sense at all in the new context.

As far as I can see, the web has not fundamentally changed in 20 years. So why shouldn't we try to think about the privacy issues inherent in the web as it is.

>Why am I surprised that you name IPFS instead of anything else?

Why don't you just say what you are trying to say?

>What does this mean? Bittorrent has always been efficient. Bitcoin is decently efficient and has fixes. Tor has always had decent bandwidth (though this is not a necessary condition to view poorly engineered high latency websites). Freenet is slow because nobody uses it.

I mean that the cost (in terms of compute and network IO) of distributed systems like bittorrent is higher than centralized systems such as YouTube.

>>892025

>Cameras being able to automatically detect your face and know where you are at any point in time. Instantly comparing your behavior to a database of suspicious ones.

>Self driving cars controlling where you can or can't go. No manual option since it will be considered "dangerous".

Agreed these are real dangers. I don't have an answer here. I believe the positives outweight the negatives, but I don't have specific solutions to these problems.

One thing I will say is this: when I raise these concerns with my co-workers, one thing people said is that making the technology open would at least allow people to create defenses. But I don't think there are any defenses to mass-surveilance.

I am doing everything I can to limit the spread of mass surveillance.

>Detecting and fixing "bad genetics" which of course will be someone's opinion in the end.

Why do you think this is bad?

>Robotics killing jobs and sending people to the streets.

The nature of all technological advancements is that it "kills jobs". It's not a side effect, it's an inherent part of economic and technological progress. The nature of technology is that it replaces human effort with (cheaper) machinery, and this is a net positive for society because human labor is freed up for other tasks.

>AI writers, painters, musicians which will outclass people and make them lose their life's meaning.

AI cannot replace true artistic expression because it's not that advanced. It can replace rote work, but see my point above.

>>892029

>Why do you think this is bad?

Well I might disagree with someone's opinion (or the AI's opinion). What if it considers being short (for example) "bad", but I don't, and if my child came out short I want it to be that way.

>and this is a net positive for society because human labor is freed up for other tasks.

Not if the people die first.

>>892038

To expand, I am really scared of something like: I have a child and go to the hospital, and there it is automatically "enhanced" genetically, and I am not even informed of what the supposed "enhancements" are, and of course I have no choice to deny them (like with vaccines - not saying vaccines are bad, just an example). But I don't intend to have children so fuck it, but other people will have a problem.

>>892040

To expand even more - it would be easy for the elites to create a population of people having the exact features they want (and lacking the ones they don't want) using the AI and the promise of "enhancement". Creativity - bad trait, kill it. Replace with submissiveness. Etc...a conspiracy scenario, but can't deny the possibility.

>>892038

>Not if the people die first.

People don't die because their job was replaced by a machine. At worst they go on welfare. Look at the percentage of the US workforce employed as farm labor over time. Did the farm laborers die?

>>892040

>>892041

I misunderstood your original point, I didn't realize you were saying the people would be forced to have their children enhanced by AI.

I've heard stories of hospitals "enhancing" children without parental consent by removing the foreskin of said children. But even that doesn't seem common, usually it's done with the consent of the parents.

I don't see why you think this will happen. Current laws focus more on banning selective abortions (e.g. if the baby has Downs syndrome, which it doesn't take an AI to see is a bad thing) not forcing people to have them.

What you do think about neuromorphic chips?

>>892050

I don't think they are a promising technology. The promise of neuromorphic cheap is cheaper compute-per-watt for neural network style computation. But low precisions floating point (e.g. 8 or 16 bit) seems to deliver better in this regard.

I think OP works in academia. Really interesting thread, thanks!

>>892053

Glad you enjoyed the thread. I actually work for a tech company, but there are close connections between industry and academia, e.g. many of my colleagues present at conferences.

>>892051

Personally I think its promising applications are the possible spiking neural network designs that could be built-in at a hardware level, as well as power efficiency. If I'm right, your future (if not current) work will probably see you get very involved with integrating/converting deep learning concepts and algorithms into spiking neural network designs, and have direct collaboration with neuroscientists and chip manufacturers.

go back to /r/ama

>>>/reddit/

As a software engineer that works on Machine Learning, do you operate using C and ASM?

>>892084

However the brain works, it uses a lot less power to do equivalent computations than "deep neural networks". So maybe spiking neural networks would be an improvement, but I haven't seen any evidence yet.

>>892093

Most ML is done in, C++, Python and JS/TypeScript (for frontend stuff). E.g. Tensorflow is C++/CUDA "kernels" wrapped in Python. I don't know PyTorch that well but I assume it works in a similar way.

I mostly write Python because I don't write the TensorFlow kernels. I did write some handwritten assembly for ARM NEON for computer vision before I got into deep learning. I would guess that handwritten assembly is only used for mobile. There is no reason to write x86 assembly by hand because when training a model all the heavy lifting is done by GPUs anyway.

how do you debug a neural network implementation?

I tried to do one from the ground up for a coursework (in java because it was a java course), but it does not seem to work: it can train itself on simple stuff like the XOR function, but when I give it MNIST, the error just starts to oscillate around 10^-3. I am supposed to using correct hyper-parameters, I took from the net. Also it seems the more training examples I give it the worse it gets.

>>891625

>Language is complex

Fasttext +CommonCrawl. Fight me

There is a thread already.

>>887042

>>892159

As much I agree that this is a duplicate, this specific thread is actually a good quality tech thread with lots of good knowledge being shared. This thread is far better than the shitting Firefox threads that happen on every Firefox event.

>>892096

>Most ML is done in, C++, Python and JS/TypeScript (for frontend stuff)

Fucking posseur you know high tech stuff for white non Pajeets have to be done in ASM right?

>>892127

Some debugging ideas:

Log curves of loss function.

Log the activations: sometimes neural networks get stuck in a region where activations of each neuron are always max, and so the network cannot learn as the gradient is always zero. This is usually caused by bad initialization.

Even though you're building this yourself, you can check it tensorboard for inspiration.

Finally check your own code, both for the forward pass or gradients. John Carmack has some useful advice on rolling your own deep learning code https://www.facebook.com/permalink.php?story_fbid=2110408722526967&id=100006735798590

>>892158

That's a good way to get word embeddings but word embeddings are a tiny part of understanding natural language.

You've noted that you feel you are acting against mass surveillance (Props I guess) but isn't machine learning the cutting edge of data collection right now?

Don't get me wrong I think the tech is cool and it is one of a vast many things my pleb ass wants to know more about, but there are plenty of applications for it beyond just camera surveillance, including traffic analysis and profiling. How do you think the average user should counteract this?

>>892214

I think it's necessary to distinguish between ML and data collection. E.g. the police can operate fake cellphone towers to track people's location, but this doesn't require machine learning. So a lot of issues around data collection are not related to ML.

That said, ML can be used to analyze video and track people, number plates etc. And it can also go through data that was collected (e.g. location data) and extract even more info from it.

The only protection for a person is to not reveal that info in the first place. For video collected in public there is no way to prevent this. I'm not an expert but I think there are ways to prevent or minimize how much location, browsing etc. data is collected about you.

The work I do is general purpose so I can't control how it used. But when I do work on specific applications I avoid things that could be used for the purposes.

>>892375

Question on feature selection on text classification: TF-RF for supervised, TF-IDF for unsupervised in Neural Network...

In what situations are Information Gain, Odds Ratio or Chi Squared useful (SVM, kNN, Neural Networks)

Base assumption is that there is no feature over-load https://www.aaai.org/Papers/AAAI/2006/AAAI06-121.pdf

Adagrad vs Adadelta vs RMSProp vs Adam vs adamax vs amsgrad

Which one is the cheapest to run? Which one is the most optimal?

>>892096

> I don't know PyTorch that well but I assume it works in a similar way.

PyTorch also uses a C++ tensor library (ATen). I would say PyTorch is better compared to TensorFlow. How do you like your static compute graphs? :)

>>892127

The problem with gradient descent is that it wants to work, even if you computed your partial derivative incorrectly. Debugging this crap is pretty hard due to the stochasticity that is typically involved with this stuff. However, in handwritten implementations this is the main source of error (so you probably need to check your math). That's why these auto-differentiation are so popular, you program the compute graph, and it computes the partial derivatives with respect to the variables that you assign to the problem.

>>892127

> the error just starts to oscillate around 10^-3

What loss function are you using, regular log cross entropy? 10^-3 training loss is pretty good. You probably reached the capacity of your network.

How to get started in machine learning? I can probably get a meme job that pays $200k/year from this. Any good resources?

>>892976

Usually it is a meme that these Coursera lectures are good. However, I would recommend the lectures from EPFL in Switzerland, and the resources from Kaparthy.

http://edu.epfl.ch/coursebook/en/machine-learning-CS-433

github.com/epfml/ML_course

http://cs231n.stanford.edu/

>>892826

It has been shown that regular (fine-tuned) SGD with momentum) outperforms these. However, typically Adam is just fine. Nevertheless, for large jobs that span several days, and for which I do not have a-priori knowledge I typically use momentum SGD since the memory requirements are a bit smaller.

>>892978

>Kaparthy

It's always weird seeing his real life name instead of his screen name as I knew him before he got into all this machine learning stuff.

how close are we to the first neural-network generated otoMAD?

https://www.youtube.com/watch?v=nIZAQDBPD-U

>>893169

He always did ML related work according to the best of my knowledge? I'm talking about Andrej Karpathy (made a typo), maybe there was some confusion.

>>893181

I only heard about his machine learning related work a few years ago, but at that time I didn't recognize him because I didn't know him by his real life name. I had actually spoken with him before ~7-8 years ago about a completely different topic than machine learning. He may have been doing ML related stuff back then, but at least I didn't know about it.

>>893197

In the past he also did physics on the side (undergraduate / graduate). He's till interested in that stuff though, especially if he can make a connection with machine learning. I had the pleasure as well to "meet" him, i.e., attend some seminars on the union of those fields at a physics laboratory. They were quite high-level, but it was intended for physicists after all :)

>>893216

> Stop your research Goy, it's evil.

AI doesn't spy on people, people do :^)

>>893244

you mean people who fund this shit

I see other people, some with more knowledge than me, answering questions so I won't reply to everything

>>892799

I'm not familiar with any of those terms apart from TF-IDF (are they all from that paper?). But generally I'm not that interested in classical feature selection. In my work I select features by frequency alone. Like in that paper, advanced feature selection is often paired with less advanced models (e.g. standard SVMs). Instead of spending time on feature selection I think it's more productive to think about feature selection and modeling jointly. So for example, if you have a small dataset, maybe it makes more sense to use pretrained embeddings from a larger dataset. Or maybe you should construct a vocab, but for out of vocab words, use pre-trained embeddings.

>>892965

I don't think static compute graphs are a problem per se. TensorFlow is somewhat strange in how it's implemented: some things you would expect to be part of the graph aren't (like primitives for distributed models) while some things that you would expect not to be part of the graph are (like ops for saving/loading models). Unfortunately I only think things will get worse as TF grows. But it's still the best framework out there due to its support for distributed training.

>>892968

Good point, John Carmack's article says something similar.

>>892976

>I can probably get a meme job that pays $200k/year from this.

I don't know if that's true, but if you can, go for it. I took a very specific path to learning ML so I can't be much help here, see >>892978.

>>893179

I don't know but I guess 10-30 years from neural network generated video in general.

>>893216

You're going to have to explain what you think is responsible for the world's destruction and why. Simply calling someone a Jew doesn't mean you have a coherent or valid point. And generally speaking being strongly against the status quo doesn't make you smarter or excuse you from having to explain your reasoning.

>>892980

Assuming you do want high accuracy with large machines while wasting little time, which one is the best? What if the accuracy-memeroy-speed trade-off is changed?

>>893280

Well memory-time trade-off IS an issue for hobbyist like me, TF-RF would save time for supervised learning, no?

Also, speaking as a /pol/ack for >>893216 there are already lots of info how Machine Learning is used to Mass censor information and target people by topic sentiments or "writeprints".These information is a given in >>>/pol/

Examples include pre-cambridge-analytica days of Facebook/Obama, and SPLC/ADL's keyword tracking system.

Lurk Moar, 8chan is /pol/: The Imageboard, get used to it

https://github.com/pinkeshbadjatiya/twitter-hatespeech

>>893332

More info https://www.adl.org/news/press-releases/adl-uc-berkeley-announce-groundbreaking-project-using-ai-machine-learning-to

https://www.engadget.com/2017/08/18/google-uses-machine-learning-to-help-journalists-track-hate/

>>893280

Muh China using Machine Learning to censor

http://www.scmp.com/news/china/policies-politics/article/2113377/its-seen-cool-place-work-how-chinas-censorship-machine

https://www.theglobeandmail.com/news/world/china-using-ai-to-censor-sensitive-topics-in-online-group-chats/article33116794/

https://www.reuters.com/article/us-china-congress-censorship-insight/tea-and-tiananmen-inside-chinas-new-censorship-machine-idUSKCN1C40LL

>>893280

SInce ELU > Leaky ReLU > RELU, I was thinking, why not reduce the complexity of ELU by replacing the exp where x < 0 with soft-sign? that would save time in computation while keeping the shape of f(0,a) = 0, f'(0,a) = 1, f(-inf,a)=-a and f'(-inf,a)=0 ?

>>893280

> But it's still the best framework out there due to its support for distributed training.

Not for long ;) I'll release some code for a upcoming conference in the next few weeks. Full PyTorch implementation, no other libraries. PyTorch's distribrubted module has finally matured to the extend that it doesn't randomly segfault, and gives the developer a lot more freedom imho. I mean, the default TCP communication protocol (with optional MPI backend) are raw bytestreams. With TensorFlow, you have to compile protocol buffers, so you cannot change the size of the tensors you'll send over the network (again, static compute graph). I even managed to get PyTorch easily working on InfiniBand without doing anything crazy.

> inb4 PyTorch fanboy

> You might be right.

>>893332

How large is your model (number of parameters / layers)? Are you using residual connections (to combat the vanishing gradient problem if you have a lot of layers)? Like I said before, typically SGD with momentum works fine.

I'll give you a tip, I do this one sometimes if the hyperparameters that I assigned to SGD with momentum doesn't work that good. Take like 100 samples from your dataset, use some optimizers with different hyperparameters, and check which one overfits the best.

>>893376

Okay, here are a few ideas of what I want to do.

1. ResNet/Inception based classification of images (assume that I will try all versions)

2. Text/User classification/clustering using with Fasttext or char-gram with LSTM and/or CNN

(Assume I will not do hierarchical LSTM but LSTM-CNN, CNN-LSTM and pure CNN with fully connected end layers)

>>893376

Might as well respond to the problems of >>893332 >>893347 and >>893348 regarding ML-based man-hunting and censorship

>>893377

For 1. I would just use SGD with momentum as mentioned especially with ResNet/Inception, this has been proven time and time again that it produces a better error (in the end) for large models.

For 2, I'm going to be honest with you, I'm not really an NLP guy, but just from an architectural point of view I would try Adam/RMSprop, and if that doesn't pan out just regular SGD with momentum.

There is some work done on automatically tuning momentum (YellowFin). However, it lacks from instabilities from my experience. Did not try this one on large models.

>>893379

What about >>893361 and >>893378 ?

>>893379

Also, if you want to optimize your hyperparameters, the above is basically a very greedy / random approach. A more efficient approach would be to apply the same procedure and pour it in a Bayesian optimization problem, where you model the loss with a Gaussian Process (with a Mattern kernel). Then, as the next sampling point, do NOT pick the one that minimizes the expected loss (you know that through the Gaussian Process), but pick the one that maximizes the information gain -> minimizes the uncertainty.

>>893400

So instead of using NNs directly, I should just use Bayes beforehand? Could you give more details (and perhaps some links)?

>>893361

Forgive my retardation, but how does your approach make sure that the mean activation is zero and prevents vanishing gradients? Because you say it yourself: f'(-inf,a)=0 -> Derrivative is zero, so errors will not be propagated to the upper layers.

This might be interesting for you https://arxiv.org/pdf/1706.02515.pdf Basically, you have batch-norm implicitly, because they ensure the activation to be zero-mean and unit variance (under some assumptions).

>>893404

https://thuijskens.github.io/2016/12/29/bayesian-optimisation/

https://stats.stackexchange.com/questions/302891/hyper-parameters-tuning-random-search-vs-bayesian-optimization/302895

This should get you started. Now if you have a lot of hyperparameters this approach will be a bit problematic because typically Gaussian Processes shit themselves when you increase the dimensionality of the problem.

If you have a large parameter space, there are new approaches which allows you to do gradient descent on the hyperparameters through variational inference (Adverserial Variational Optimization, Classifier Approximate Bayesian Optimization, Recurrent Variational Inference, ...). While these methods are not designed to do this, I think you can use them for hyperparameter optimization. But now you will be training neural networks to train neural networks so it becomes a little bit messy haha.

>>893406

Well https://github.com/ducha-aiki/caffenet-benchmark/blob/master/Activations.md

ELU's Range is (-a, inf) according to https://en.wikipedia.org/wiki/Activation_function

The slope of (0,0) is always 1 for these s-curves, and y -> const when x -> -inf

The other side though is always y=x, so vanishing gradient only goes one-sided

ELU uses an s-like function when x < 0, which can be easily replaced by softsign

ELU uses exponents, ISRU uses sqrt (which is fast enough), softsign is literally division

So clearly there is room for improvement.

>>893414

In the case of Fasttext with 300 dimensions, or english trigrams (26**3 features),

which ones are the easiest to make/use? which ones is the most accurate?

>>893417

There is only one way to find out right, could you do a quick benchmark on MNIST or something? Would be interesting to see what happens.

I was talking about (optimizer) hyperparameter optimization, i.e., learning rate, momentum, etc. So this does not really apply to your case. An alternative approach to do it, which saves you time in the end I think:

1. Train your model with SGD & momentum (lr=0.01, momentum=.9).

2. Apply learning rate scheduling (if required, I do not know).

3. Let it train until the loss stabilizes.

4. Save the model.

5. Fine-tune with different optimizers (e.g. Adam, ...), and check if there are any improvements.

Or, as an alternative to 5, use the Bayesian optimization approach I previoiusly mentioned to fine-tune the hyperparameters of the optimizers in the fine-tuning step.

How to mess with a pajeet >>893328

>>893430

Tell him to check the Swiss superiority in >>892978

>>893157

extremely rigged datasets. remember the racist political AI? (Nogs = Bad News)

https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

http://www.sciencemag.org/news/2018/02/artificial-intelligence-could-identify-gang-crimes-and-ignite-ethical-firestorm

>>893157

Essentially don't manipulate the datasets and add a classification for race. That's all you need, the unbiased truth.

>>893378

>>893332

>Well memory-time trade-off IS an issue for hobbyist like me, TF-RF would save time for supervised learning, no?

I honestly don't understand what you're saying here. What is "memory-time trade-off"? I've never heard of that before, most ML is not memory bound. Usually we would speak of tradeoff between compute time and accuracy. But TF-IDF and related techniques are more about overfitting anyway.

>Also, speaking as a /pol/ack for >>893216 there are already lots of info how Machine Learning is used to Mass censor information and target people by topic sentiments or "writeprints".These information is a given in >>>/pol/

That is such a lazy argument, as we both know there is a vast amount of info posted in /pol/, not all accurate. But at least you have said something substantive. In general I'm against censorship, but I think its stupid to hold all people working on ML responsible for it. What are we supposed to do, just stop working on ML entirely because some people use it for censorship?

That said I personally have worked on projects related to censorship, but I can't speak about it due to NDAs. I've tried to move things in the direction of censoring objective things (such as personal insults) rather than politically charged notions such as "hate". Note this difference is more a matter of training data than how the ML is done.

>>893375

Only variables are fixed shape, everything can be dynamic.

>>893348

>>893378

How the fuck am I responsible for what China does? Don't you realize they will do this anyway no matter what anyone in the West does?

>>893404

No you're totally missing the point. >>893400 is saying you use an algorithm based on Gaussian Processes to optimize your hyper parameters. There are two levels of optimization here (1) optimization hyperparameters (2) train your model (=optimize model parameters, using given hyper parameters for optimization).

>>893758

I doubt adding a classification for race wouldn't help much, though it's a good idea. The problem isn't that Black people are indistinguishable from apes, but they are more likely to trigger this rare error. In fact, we don't even know that, because there would be no news stories about white people people classified as apes.

>>893280

>.I work for the system but I'm not part of the problem

You're a lemming and nothing will change that. get out.

>>893845

The funny thing, whether you ask your Marxist professor or /pol/, they will both tell you "the system" is bad. I'm a centrist, I don't think that the status quo is bad or the world is moving in a terrible direction. There are applications of ML I disagree with, and aspects of my work I disagree with, but overall I think it's a net positive. Sorry if this is too complex for you.

>>893848

>you're just dumb

>i'm an enlightened centrist

holy shit go to reddit.

>>893851

No, you don't have to have the opinion that everything associated with the mainstream is wrong, to post on /tech/

>>893789

>The problem isn't that Black people are indistinguishable from apes, but they are more likely to trigger this rare error. In fact, we don't even know that, because there would be no news stories about white people people classified as apes.

Wouldn't the problem be alleviated if input data is pre-processed by a network that breaks down an image into many relevant abstracts prior to training for recognition? Because when we receive stimuli, we're technically not processing the entirety of it in order to recognize something, rather our brains focus on the many "interesting" parts/patterns it contains while tuning out the rest. Then afterwards it cross-references all these abstracts with what is already known in memory to determine a classification within a degree of certainty. In that case you'd be dealing with a group of outputs describing one desired output (like how you can describe a cat by how it has four legs, two eyes, a nose, a tail and so on).

In the case of black vs gorilla, while it'll still could see the resemblance (majority of abstract recognition agrees), it will also notice the subtle abstract differences which don't and correctly classify it.

>>893855

What you are describing is pretty much exactly how deep neural networks work.

And in general they can distinguish Black people from gorillas. It only takes 1 error to create a news story.

>>893858

Let me ask you a question: if society was fundamentally good, but also had serious problems, what would be a reasonable response to that?

>Did they ever release the probability scores on that famous news example does this mean that it would be easier to generate adversarial examples that cross the border from blacks into primates

A probability score is the full set of probabilities a model assigns to each label. E.g. gorilla 34% human 23% tree 12% ...

Releasing this information would not make it easier to generate adversarial examples. There is plenty of information available on how to generate adversarial examples.

>>893862

>if society was fundamentally good, but also had serious problems, what would be a reasonable response to that?

If it was fundamentally sound then it would be worth fixing, but it's not. The end goal of the system we have today is to mix

away every bit of identity and dignity anywhere in the globe so that everyone becomes nothing more than interchangeable human capital. Only time will tell the outcome, but passive acceptance of censorship certainly helps keep this "out of sight out of mind"

>A probability score is...

Why do you assume nobody on /tech/ but you knows anything about ML? The barrier to entry is fairly low these days.

>>893789

For >>893378 >>893332 I meant computing power and time trade-off, as in having a games-grade GPU or AWS means it takes less time to process the data.

>That said I personally have worked on projects related to censorship, but I can't speak about it due to NDAs. I've tried to move things in the direction of censoring objective things (such as personal insults) rather than politically charged notions such as "hate". Note this difference is more a matter of training data than how the ML is done.

First, read this >>893347

Secondly, mind if you give us some pointers for how we can turn the tide? I promise we will clean-room design everything, defensive wise (circumvention) and offensive wise (using the same type of methods for finding "enemies")

>How the fuck am I responsible for what China does? Don't you realize they will do this anyway no matter what anyone in the West does?

Are the methods used by China the same type of methods that America and Europe does? Or are there any methodology overlaps at all?

>I doubt adding a classification for race wouldn't help much, though it's a good idea. The problem isn't that Black people are indistinguishable from apes, but they are more likely to trigger this rare error. In fact, we don't even know that, because there would be no news stories about white people people classified as apes.

Who wants to go VK scraping photos by race? Or even twitter geolocation scraping?

>>893845

Damn it. OP here >>893848 says he is not the one pulling the trigger (you know who IS though), in fact if he drop hints here for us to fight back, we can go on the offensive against ADL/SPLC, so give the guy some respect, will ya?

>>893852

This is /pol/: The Imageboard after all. The whole "centrist" thing turned nasty after the "skeptics" and "alt-light" ruined it (Liberalist doxing alt-right and Kilroy Event scam).

>>893855

>>893856

>>893858

>>893862

Well someday the glasses from "They Live" will be made for finding Blacks and Jews. We don't care about GANs, we care about live usage.

>>893862

See >>893858 and >>893867 If you are making the "weapons" for "them", could you please also teach us how to make one for ourselves? At least that is how you turn the tides. Don't sit idly by while the world burns.

For a primer, go through Computing Forever's videos on how ADL/SPLC is destructive. https://www.youtube.com/user/LACK78

Question:

1. Job-specific Bag Of Words or pre-trained FastText for Text classification?

2. Pure CNN? (what if the input has variant length?) CNN-LSTM or just LSTM? (what if I want to cut down the time?)

3. How do you classify uses based on what they tweet? Assuming you can harvest all tweets, and have binary user dataset?

4. How do you cluster users based on what they tweet, assuming some of the tweet are noise while others are useful?

(Assume that both 3 and 4 has no classified tweets as training dataset, that means this is not completely supervised)

5. ELU or PReLU with loss function? Or Vanilla ReLU with large amounts of neurons (purposeful dying ReLU)?

6. ELU? Or Parametric ReLU? Scaled exponential linear units (non-parameter ELU)? Or Leaky ReLU?

7. Momentum SGD for large jobs and Adam for small jobs? Perfect settings for SGD/Adam? Batch or Mini-batch?

>>893872

>>893867

Thank you both for your comments. On the political side, I'll try to address what you have said holistically rather than answer individual points.

>>893867 has said that "the system" is fundamentally wrong and needs to be replaced, and I disagree. Capitalism and democracy are fundamentally good things. I don't think I can convince you of this in this thread, but one point I will make is that there is no real link between capitalism and multiculturalism. When I was in college it was fashionable to say that globalization means free movement of people, not just goods and capital. In fact globalization works just fine with free movement of goods and capital but not people. According to economic theory, if goods and capital are mobile, then they can go to the people instead of the other way around. E.g. factories can move to China instead of Chinese moving to the US to work in factories. I think this is a good thing: everyone improves their quality of life, but cultures and ethnicities don't mix (although this benefit is one sided: Asians have never seriously complained about the influence of Western influence because they know it's a good thing).

Regarding ML itself, I've said all I can about my own work. But apart from what I personally work on, I don't feel responsible for what the ADL or China do. As >>893867 said, the barrier to entry is very low and groups will use ML for various purposes regardless of what I do. I'm not going to help China or the ADL but I'm not responsible for what they do just cause I work on ML.

>Why do you assume nobody on /tech/ but you knows anything about ML? The barrier to entry is fairly low these days.

You're being a bit sensitive. I explained what probability scores were because there is no real relationship between having access to probability scores and being able to generate adversarial examples. So I thought it was likely that >>893858 did not understand what probability scores were.

>Secondly, mind if you give us some pointers for how we can turn the tide?

You need to understand how ML models work and what they are capable of. Models vary in sophistication but the details usually don't matter because without huge amounts of training data, sophisticated models will emulate simpler models anyway. Three main classes of models are character-based models, word-based models. Character based models will respond to sequences of characters, e.g. "whateverrrrrr" might trigger in the model both recognition of the parts of "whatever" and the trailing "rrrrrr" which might imply nonchalance or dismissal. Word based models respond to words or sequences of words. E.g. "cuck" or "virtual signaling" might strongly imply a right wing message.

So in both of these models, any kind of right-wing shibboleth would be easy to detect. It doesn't matter of your phrasing is not explicitly right wing or "racist". If is associated with the posts that "they" are trying to censor, the model will learn to recognize it. E.g. replacing "Jews" with "Finns" won't work.

For this reason, one way to avoid censorship is to recognize and avoid common patters, but only at the word or character level. General topics or styles of argument are ok because ML isn't sophisticated enough to recognize this.

So if instead of saying

"Sweden is cucked, they let Ahmed and his friends rape their daughters"

you say

"Swedes are too afraid to stop immigration that is causing rape of Swedish women"

you are saying roughly the same thing but without using patterns of words that are distinctly right wing.

As for detecting particular groups (e.g. Antifa) I don't think this is particularly useful, you could just compile a list directly. Maybe ML could help identify small fry, but I wonder if there are that many in the first place anyway. If you do decide to look into using ML for this, the usual rules for ML apply: focus on getting large, accurate training data and then start with simple, well known techniques. But again, in the time it took to get the training data you could have just looked directly for these accounts.

>For a primer, go through Computing Forever's videos on how ADL/SPLC is destructive. https://www.youtube.com/user/LACK78

Thanks, I already hate the ADL/SPLC but will take a look.

>>893884

>1. Job-specific Bag Of Words or pre-trained FastText for Text classification?

Try both, but also try training FastText on your training data, or even better, a larger unlabeled dataset from the same source.

>2. Pure CNN? (what if the input has variant length?) CNN-LSTM or just LSTM? (what if I want to cut down the time?)

I've seen all of these used, so try them all.

>3. How do you classify uses based on what they tweet? Assuming you can harvest all tweets, and have binary user dataset?

>4. How do you cluster users based on what they tweet, assuming some of the tweet are noise while others are useful?

>(Assume that both 3 and 4 has no classified tweets as training dataset, that means this is not completely supervised)

You always need hand-labelled training data, but see my point above on using unlabeled data for training FastText or other unsupervised feature-extraction. Given this, use standard ML techniques (for text, you already listed them in 2.). Again, don't sperg out over the exact model, focus on collecting lots of labelled data. I don't know much about clustering.

For 5-7 I don't have any answers, don't know much about this side of things. Someone else in this thread was answering similar questions earlier.

>>893884

>what if the input has variant length?

Forgot to answer this part: use some kind of pooling e.g. averaging or max pooling

>>894189

>Three main classes of models

>Only character (n-grams) and word (vector) models

>No sentiment model

What about that?

>recognize and avoid common patters

What about automating the process of knowing which patterns to avoid?

>As for detecting particular groups (e.g. Antifa) I don't think this is particularly useful, you could just compile a list directly.

We are finding "antifa" affiliates who don't specify who they are, but have strong and similar sentiments towards some topics

One last thing, here have a Smile https://www.youtube.com/watch?v=jGLJ2HQDzmM

>>894190

The users are hand-labelled (blocklists), BUT the tweets are not. That is the main issue.

>>891597 (OP)

You ML people are a joke. Just for fun, I asked a comedian from your field to solve a basic linear algebra problem. It took him a week to do it.

He didn't even know what an eigenvalue was, let alone a Hilbert space.

The stochastic garbage you people come up with elicits chuckles from the control theory crowd who work on serious optimization algorithms.

Oh please, these faggots only deal with significantly constrained convex spaces under certain Lipschitz assumptions.

Try to come up with an optimization method whose objective function constantly changes. That is why we need to stochasticity you massive faggot. I agree, first-order gradient descent may not be the best tool, but it is the best that we currently have. We are sure not going to compute a Hessian for serious problems.

> I know more words than you so I must be smarter.

Fag

>>894267

>>894290

If you do work in the academic field, riddle me this, how do you classify users by what they tweet, with the trouble of not having information about what topics can classify them, and that if such such a topic exist, would change over time and become irrelevant.

>>894293

https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation

https://arxiv.org/pdf/1206.3298.pdf

Or, as I said before, make an auto-encoder. Project the tweets into latent space, cluster in the latent space. Of course, this requires you to define a distance function / clustering algorithm in this latent space.

>>894297

So you want dimensionality reduction? I know LDA are related to LSI and NMF, but how does this address the time dimension? (Topic overlaps, Topic polarization/irrelevance in the future, possibly Sentiments with Emotional Lexicon or other means)

Also, while we are on the subject, "Personality by 25 tweets" https://arxiv.org/pdf/1704.05513.pdf

>>894303

No, I do not want dimensionality reduction just for the sake of it. I want dimensionality reduction such that the compression given by the auto-encoder groups tweets semantically, as in Word2Vec or other vector representations.

However, to address the time dimension in this case, you could for instance retrain the auto-encoder every t steps, and check of the - relative - coordinates of the instances evolve over time, which corresponds to a semantical change as was shown in several Word2Vec experiments.

For instance the relative position of faggot and homosexual changed over time. At the moment, faggot might be semantically closer (in latent space) to idiot.

>>894309

Jesus, that sentence structure. Sorry for that.

Dumping some more personality type analysis for twitter

https://www.demenzemedicinagenerale.net/pdf/2011%20-%20Predicting%20Personality%20from%20Twitter.pdf

https://www.gsb.stanford.edu/sites/gsb/files/conf-presentations/querciatwitter.pdf

>Using Emotional and sentiment lexicon

>Using Psycholinguistic databases

Might as well dump your favorites ITT

Regarding EPQ-R (Eysenck 3) vs NEO-PI (Big 5), Neuroticism is consistent, Extroversion is Consistent.

Psychoticism is a mixture of Openess (extrovert), Agreeableness (neurotic) and Conscientiousness.

>>894322

Exactly like that actually. I think you can use a pretrained W2V model, and then fine-tune it like people do with Inception etc. Under the assumption that the model contains your keywords of course. The main issue with this approach is synonyms, so I think you have to design it for a specific application. E.g., financial banks and banks you can sit on (in Dutch it is the same word). Personally I do not know if there are any techniques which solve the synonym issue imho.

> that are important based of TF-IDF/RF change.

You mean, after identifying the keywords that change semantically?

>>894322

Don't forget that the figure you posted is a PCA projection of the latent space. Maybe you get better results if you use the cosine similarity on the vectors to check if they changed semantically. Nevertheless, the PCA projection can be a useful tool for debugging.

Just thinking out loud here.

>>894346

Those are not called synonyms, "nigger" and "faggot" are synonyms. What you are talking about is a "homograph".

>You mean, after identifying the keywords that change semantically?

Before, since you need to know which words are the most important in classifying users for a certain period of time.

>>894347

Mind if you give me a run-down on LDA vs LSA/LSI vs PCA vs NMF, because those are very easy to mix up.

Which ones are discrete? Which ones are probabilistic?

>Just thinking out loud here.

No need to worry, 8chan is long due for a new software project.

>>894353

> Those are not called synonyms, "nigger" and "faggot" are synonyms. What you are talking about is a "homograph".

Sorry, I'm retarded.

> Before, since you need to know which words are the most important in classifying users for a certain period of time.

So you directly use the TF-IDF features to classify whether or not a word is important? Does that really yield good results on Twitter?

> LDA vs LSA

> LSI vs PCA vs NMF

Not familiar with NMF.

I usually remember it like this:

- LSA, LSI, PCA are all 'glorified' SVD

- PCA computes the covariance matrix and does not require label information to obtain the principal components.

- LSI -> Reminds me of TF/((I))DF -> Term-Document frequencies. -> Uses term frequencies in contrast to the covariance matrix.

- LDA is the SVM of topic models,

i.e., maximizes separation of class labels to find the topics -> supervised

>>893347

>>893348

According to these rumors, are China and the US using the same methods of censorship?

>>893414

I don't think I understand this part of machine learning, from my minimal understanding, I am basically using a Naive Bayes system (like the ones used in spam filter) to evaluate which words would make a document more "spam-like" (or in this case, "liberal"). How does one combine occurrence of two different words to make things more accurate?

>>893789

Are Gaussian Processes similar to SGD in principle?

>>894357

>So you directly use the TF-IDF features to classify whether or not a word is important? Does that really yield good results on Twitter?

Well if I have to cluster topics that should be the usual technique (I think?) because crunching every word is bad.

But if I have labelled tweets then TF-RF (which is a way of finding keywords that aids classification).

The problem comes, since I only have pre-classified users and NOT tweets, I might have to make compromises.

Either: I make a leap of faith and assume all tweets from a user is from that exact user-class (TF-RF),

or that I don't make such an assumption and use TF-IDF which throws out important information.

>>894357

I just looked up NMF, apparently it is just non-negative matrix factorization. Think of it as an iterative version of SVD.

>>894357

So PCA is unsupervised, LSA/LSI is supervised, and LDA is probabilistic supervised. Got it.

>>894365

Vanilla PCA (unsupervised)

LDA (prob supervised)

LSA/LSI is the same thing. (supervised).

>>894360

Consider the following:

P(X = spam | liberal) = .1

P(X = spam | trump) = .1

P(X = spam | antifa) = .1

Now the joint conditional distribution:

P(X = spam | liberal,trump,antifa) = .9

So what does your model do? It models the distribution of spam messages, and samples from that given some conditions.

>>894371

I know I definitely need to read the Wikipedia articles, but we might as well do it here and now.

Then how does it establish NOT conditions? e.g. P(X = spam | jewish) > P(X = spam | jewish,questions)

Another question, how do you optimize Naive Bayes? If it is done by Picking the top N words that works,

then trying different combinations that increase the probability, you example will not be detected.

>>891597 (OP)

where do i begin to study this subject?

>>894388

It depends on how YOU implement the posterior, and the feature engineering.

But in the most simple case it is done by counting words that occur in every class (spam / not spam) and using that as a probability measure.

https://web.stanford.edu/class/cs124/lec/naivebayes.pdf

>>894398

So decision tree is a simplification of such process by cutting away many "useless words"?

needing less data features, it can do a faster job but increase in complexity than simple naive bayes?

(yes I know random forest uses a droput-like method to prevent overfitting)

>>894405

You can view it like that. However, DT's are deterministic. In Naive Bayes you have access to probabilities. You could for instance draw a random number, and than decide to accept or reject the message proportional to the probability of the message being spam.

You could of course widen / deepen the trees to emulate this behavior, but that would have some significant computational / memory cost.

>>894412

Thus, Random Forest is born

>>894416

Yep. Have you tried the method I previously suggested with the auto-encoder?

>>894417

On it.

>>894426

Third times the charm

https://matrix.to/#/#tech:matrix.ordoevangelistarum.com (tech generals)

https://matrix.to/#/#neural:matrix.ordoevangelistarum.com (machine learning)

Assume there are 80 words per tweet, and we have 300 dimensions.

We are using tanh as activation, the adam optimizer, and mean square error.

We are reducing the amount of demensions to 14 (because why not?)

input_i = Input(shape=(300,80))
latent = Dense(14, activation='tanh')(input_i)
output_i = Dense(80, activation='tanh')(latent)
autoencoder = Model(input_i,output_i)
autoencoder.compile('adam','mse')

>>892025

>Robotics killing jobs

this is the one good thing about more automation, less people need to do labor. right now there are too many people with fancy and useless blue/white collar jobs while almost nobody is doing real hard work to provide food and infrastructure

>and sending people to the streets.

i mean ostensibly this would drive price of living down but okay...

>>892014

>Current self-driving cars already do a lot more than lane following.

I never claimed all they do is lane following. I claimed they just have a bunch of algos, which are imbued with the property that they are "AI" algos. Ultimately they are just algos which cannot be applied to all real life situations. You will just keep piling on new edge cases, some of which will have to be regressed once you find new edge cases. Ironically the corporate crapcode people push out right now is exactly this way - particularly that even for trivial code they still bring back the same regression in patch 3, 5, 7, and 9 - but for some reason people think self driving cars wont be like this because le magic AI.

>I also believe that self-driving cars is such a hard problem,

exactly. all these faggots in tech go around saying trivial stuff like their concurrency 101 bugs are hard and then they get to self driving cars and claim they're easy.

>Doing it with vision alone or vision + radar requires the computer to be too smart, even for someone who is overall optimistic about ML.

exactly this

>I mean that the cost (in terms of compute and network IO) of distributed systems like bittorrent is higher than centralized systems such as YouTube.

It's negligable. p2p file hosting would be perfect for the masses if you didn't get threats of violence for accidentally hosting something that's considered copyright. Right now people merely only use it for piracy because legal crap is hosted by corporations who will pay you to host it and the only easy-for-normies way to play video is through something such as youtube.

>>894257

>No sentiment model

>What about that?

I had "three" but I meant "two". I originally had character, word and ngrams but I decided that character vs word was the most meaningful categorization. The difference is character based models apply an ML model to raw characters while word-based models apply the model after first converting words to vectors, either with something like word2vec or by using a vocabulary or hashing. So you have character-ngrams and word-ngram models, bag-of-word models (there is no character equivalent) and LSTM and friends for either character or words.

"sentiment models" are not a class of model architecture, it is just a term used for models where the input is text and, the output is label that is vaguely related to sentiment, e.g. "liked the movie" or "didn't like the movie".

>What about automating the process of knowing which patterns to avoid?

You can train a model of your own, and then have it report scores before you post. The same could be used to prevent fingerprinting of a single person: train a model on your own posts then report a score which reports when you are posting too much in your own style.

>>894267

Fields such as control theory base their guarantees on the assumption that the world follows some model, and as >>894290 says, this often requires such a simple set of assumptions that it doesn't provide any real guarantees in the real world. ML on the other hand bases its guarantees on the assumption that the real world has exactly the same probability distribution as your test set. And in practice this also is often not true.

So we need both approaches. Black box ML approaches to get results even when there is no analytical solution. And theory based approaches to provide robust guarantees that apply even to situations we haven't seen before.

>>894360

>Are Gaussian Processes similar to SGD in principle?

Gaussian Processes are a kind of probability distribution, not an optimization technique, so they cannot be compared to SGD. Gaussian Processes are a popular prior distribution used in Bayesian Optimization. So the comparison should be between Bayesian Optimization and SGD. I'm not that familiar with Bayesian Optimization but generally it is used for hyperparameter tuning (where getting a single observation is very expensive because it requires training a model) vs SGD is used to train the actual model (where we have lots of observations already).

>>894397

I started with pure math, probability and stats and only moved to CS later. This gave be a very solid foundation but is not necessary for everyone. See >>892978 for a more direct path.

>>894460

>Ultimately they are just algos which cannot be applied to all real life situations.

What makes ML special is that it works so well in real life situations. Not perfectly, but much better than anything hardcoded. Because of this, you don't need as many edge cases to fix the problems of the ML system. E.g. the object detection layer of a self-driving car might be a pure ML end-to-end system without any edge cases, because object detection is solved so well by neural networks.

>>894641

>>894638

If it is possible, could we start a Machine Learning study group?

https://matrix.to/#/#neural:matrix.ordoevangelistarum.com (machine learning)

https://matrix.to/#/#tech:matrix.ordoevangelistarum.com (tech generals)

https://matrix.to/#/#altech:matrix.ordoevangelistarum.com (political technology)

How could we share our Jupyter Notebooks right here in /tech/? File embeddings?

Maybe cat `archive.zip > file.png; unzip file.png` (but what if someone put in a zip bomb?)

https://encyclopediadramatica.rs/Embedded_files

(Maybe mixtape.moe or pomf.space?)

>>894979

What about safe.moe

>>894638

Well wouldn't it be character > n-grams > word n-grams > "sentiment"?

As in every level is possible for modelling? Or is it too abstract?

Because I have seen lots of lexicons throwing around in recent years.

How do you break down a sentence into topic/keyword sentiment anyway? NLP?

>You can train a model of your own, and then have it report scores before you post. The same could be used to prevent fingerprinting of a single person: train a model on your own posts then report a score which reports when you are posting too much in your own style.

Stylometry (Basic 9, writeprints collections etc.)?

https://www.hatebase.org/ is this something that we should be worried about? n-grams based detection?

What if we use the full unicode range like https://en.wikipedia.org/wiki/IDN_homograph_attack ?

>>894973

I don't have time to do this. But if you do I would highly recommending basing it around a course such as >>892978. In this thread I see a lot of people asking about specific advanced techniques or models, but this knowledge is not as useful as a broad knowledge of ML. In fact, very basic models are usually all that is needed to solve a problem, and advanced techniques only get tiny improvements.

>>894979

How about sharing on gitgud.io or a similar site? Also I recommend against Jupyter notebooks, they are great for experimenting but code is better for sharing because it's easier to organize code logically.

>>895381

characters, ngrams, words, and word-ngrams are not increasing levels of abstraction. While a character based model is theoretically a superset of word-based models, in practice once you fix the model architecture, they are just different models.

A character based models can handle words it has never seen before. But a word-based model is more efficient and "memorizing" words. In all cases, the interface of the model is the same: it takes raw text and returns a prediction. The difference is the internal architecture: the word based model, by definition, construct a vocabulary of words (or hashes words into some hash space), then applies a standard sequence model such as CNN or LSTM. The character based model, by definition, applies a sequence model directly to characters.

To clarify what I meant, I was thinking of a model with exactly two labels "liked movie" and "did not like movie", trained on a database of movie reviews. I wasn't speaking of learning a topic, sentiment pair.

>Stylometry (Basic 9, writeprints collections etc.)?

For identifying an individual yes. For identifying something that would be flagged by the ADL model, you can just train your own model that mirrors the ADL model. If they provide an API you could even query it to build a training set.

>>895447

There are ways to normalize unicode to a canonical letter. So this is not a good defense, at least in the long run.

>>895576

>Also I recommend against Jupyter notebooks, they are great for experimenting but code is better for sharing because it's easier to organize code logically.

Using it as a text book for studing machine learning is actually a good idea, I think.

>>895642

In the Stat/ML classes I teach and assist, the students have to prepare their homework in notebooks. Works pretty well for most of them, also makes my life easier.

>>895576

>In fact, very basic models are usually all that is needed to solve a problem, and advanced techniques only get tiny improvements.

> tiny improvements

That is just wrong and naive. The main reason people still use these simple approaches is that they give sufficient results according to the people using them, and that the implementations and accompanying documentation of these are solid, robust and easy to use (which is good). Furthermore, IF a tiny improvement has been made, that just means that the dataset is very simple, or that you cannot extract information out of it due to the signal-noise ratio. Also, don't forget the engineering and hyperparameter tuning that has to go into that stuff to make it actually work well. Just to name a few, picking the right kernels, setting the hyper parameters of those kernels, - designing - a robust distance function, number of means in k-means, what information gain should I use in my trees, what depth? How many trees in my forest? Ok, how to deal with the memory requirements with this large dataset, ok let's make assumptions and do a linear fit to reduce the features, or use the primary features of the tree to use those with a different classifier / regressor. Quelle suprise, the variance is of the charts because of these simplificantions. Ok let's try autoregression. What hyper parameters for the temporal influences? ... And the list goes on.

That being said, I am not advocating that you should not use the old techniques. They are perfectly fine, and they should be used when appropriate. Heck, you can even combine them and use the properties of the old ones to extract good descriptive statistics. But saying that the new techniques only yield tiny improvements (even on simple problems, e.g. MNIST) is just wrong.

>>894189

>capitalism and democracy are fundamentally good things.

The only thing that matters is who is in power, not what the form of the system.

>>894638

>Fields such as control theory base their guarantees on the assumption that the world follows some model

I have 2 data sets, the position of stars in the sky, and a pseudo random number generated data set of "positions in the sky" of the same size.

There is no model that correlates the 2, yet ML will still spit out an answer nevertheless.

Because of this property of ML, you can receive government grants if you bullshit hard enough on your applications.

Not to mention how easy it is to fudge data to get the results you want.

It's much harder to fudge a model.

>Black box ML approaches to get results even when there is no analytical solution.

Easily replaced with a deterministic fuzzy logic approach for similar results. Explain why having stochastic and combinatoric explosion is worth it.

>>895713

By "simple" I didn't mean classical methods like SVM or logistic regression. I should have said something like "standard" or "typical". I admit that a "standard" neural network model (e.g. LSTM) is much more complex than a "standard" classical model such as an SVM since there are more parameters to choose. So overall I think we are agreement. Most users are better off building a "simple" neural network than building an SVM with every possible tweak.

The thing I really want people to avoid is reading about the latest technique from academia and wanting to use it in their work. And I also want people to avoid thinking of ML as a bunch of disparate techniques, but to understand at least some of the underlying theory. E.g. a lot of people don't realize that "logistic regression" and "softmax" refer to exactly the same calculation. So for example they don't realize that "fine tuning the final layer of a pre-trained model" and "using the final hidden layer activations of a pre-trained model as input to a logistic regression model" are exactly the same thing.

>>896089

You are referring to overfitting. It is a problem in the field but not for the reasons you mention. With a sufficiently large test data, it is statistically impossible for a model that has no predictive power to appear to do well on the test data. Note I'm referring to test data, obviously it's trivial to fit to the train data but people know this.

>Easily replaced with a deterministic fuzzy logic approach for similar results. Explain why having stochastic and combinatoric explosion is worth it.

Can you give some examples of fuzzy logic performing well on MNIST or ImageNet?

>>896199

>>896089

Old school vs new school, I like it.

Apply the same logic to text classification and sentiment analysis, see how this goes.

>>896199

>Can you give some examples of fuzzy logic performing well on MNIST or ImageNet?

Is this a trick question?

http:// iopscience.iop.org/article/10.1088/1742-6596/738/1/012123/meta

If I wanted to go the extra mile, I'd change the sigmoid activation functions as well.

I'd also mention NP-Complete SAT3 problems and how they equivocate to certain neural network structures; meaning that unless you have optimized your neural net structure and found a special case, you aren't solving shit in polynomial time.

https:// arxiv.org/pdf/1710.01013.pdf

I quote:

"there is theoretical work showing that a family of neural networks provided with standard logistic activations can be equivalently converted into fuzzy rule-based systems [18], thus raising the possibility to perform reasoning using fuzzy logic and potentially extract human interpretable explanations of predictions made by deep learning models"

Now, explain why having stochastic and combinatoric explosion is worth it.

>>891602

When will this get so good anyone can make the voice of Pierce? Or Colored photos of Rockwell?

Neural Networks in Mobile Phones that can literally spot the """enemy""" with a camera... I can only dream.

>>891597 (OP)

what happens inside the hidden layers is understood?

>>896323

https://www.ncbi.nlm.nih.gov/pubmed/18255717

https://www.researchgate.net/publication/307871326_Fuzzy_Logic_Module_of_Convolutional_Neural_Network_for_Handwritten_Digits_Recognition

Sorry mate, but the work you refer to still employ the neural networks to copy with the dimensionality. Nevertheless, I have to say that the Fuzzy Logic approach is interesting. The problem you, and that the authors in the last work mention, are there because of the inability of convnets to correlate the activations of the filters in a spatial manner (which is a known disadvantage). It is one of the reasons why Hinton's student worked on CapsuleNets. As mentioned before, in order to produce feasible results with fuzzy logic in these settings, you still need something that is able to cope with the dimensionality of the input space. Not to bash on these guy's work (it is an interesting approach imho), but MNIST is rather simple. I don't think it would work that well on ImageNet because the number of spatial relations that you can define here are significantly larger (also an combinatoric explosion).

However, to improve upon this I would suggest the following, and possibly have a more robust alternative to CapsuleNets that only seem to work well on MNIST (free research idea here, let me know what you think).

1. Train your convnets in a regular way to optimality. You can use one out of the pretrained model zoo to save some time.

2. Adapt the formalism from https://www.researchgate.net/publication/307871326_Fuzzy_Logic_Module_of_Convolutional_Neural_Network_for_Handwritten_Digits_Recognition but don't derive the rules from the input space thereby bypassing the convolutional modules, but rather use the obtained filters in the convolutional modules directly to solve the problem regarding the spatial relations. Not sure how to do this exactly, but I have to think about that for a moment.

This would also give an interesting insight how filters are related to each other. E.g., filters for a mouth are always below a nose or something.

To answer your question:

> Now, explain why having stochastic and combinatoric explosion is worth it.

To deal with the dimensionality.

>>896400

With ReLU's:

Adding layers with ReLU's is like folding the input space (imagine it like folding paper with blue and red dots) such that the target becomes linearly classifiable.

Thus for designing networks with ReLU's: adding more layers is preferred to widening layers.

https://arxiv.org/pdf/1402.1869.pdf

>>896402

Or just use dropouts, or ELUs, or both!

>>896402

I mean, understand it in a way that's useful for transfer learning - that u know what's going on inside and thus can reuse parts of the network.

>>896435

View it as an auto-encoder with a smaller dimension as input layer. In compresses the input space in a lower dimension. I don't have a good intuition what happens when you increase the dimensionality.

>>896417

I had a discussion with some people about this over some beers. Some argued that this would be like the Industrial revolution, where the jobs shifted from agriculture to factory work. However, I would say this is different. Since the creativity / intelligence / knowledge that is required that efficiently do the job of designing this machine is not something that the average truck driver will be able to do. Other people who work in unions also kinda agreed with me on this hypothetical scenario, they really wouldn't know what to do. The best solution they had was just to pay people to live without producing anything. But who knows, maybe selection will drive these people out eventually.

>>896401

>Sorry mate, but the work you refer to still employ the neural networks to cope with the dimensionality.

Nice try bro, but as I stated in my previous post, neural networks are SAT3(3SAT) equivalent. You either ignored this or didn't understand it.

What that really means is that I can turn any neural network into a dynamical system, and model it with deterministic chaos theory.

That'll handle your dimensionality just fine.

Is that all you got?

Btw, https:// www.researchgate.net/profile/Paul_Elmore/publication/316708469_An_approach_to_explainable_deep_learning_using_fuzzy_inference/links/5a09e2a645851551b78d274b/An-approach-to-explainable-deep-learning-using-fuzzy-inference.pdf

>>896653

> neural networks are SAT3(3SAT) equivalent

[citation needed] The only thing I could find was about spiking neural nets, which is not what we are talking about. Nevertheless, I can intuitively see why this is the case since the network obtains a parameterization that 'compresses' (minimizes the boolean variables) the data that it has seen. But why 3 literals exactly?

> What that really means is that I can turn any neural network into a dynamical system, and model it with deterministic chaos theory.

I would like to see you try :) For the optimization procedure, I would say yes you can because it is relatively natural to express it in that framework (e.g., a minima is the attractor). So how would you, in your framework, deal with the noise (variance of the gradients) of the optimization procedure?

> A hypothetical system (as seen in Figure 1) can be created using two components.

They do not solve shit, so it is still an open problem. Still not exactly what I meant though. Furthermore, you still have to define the fuzzy rules yourselves. On the train I was thinking you could maybe define like a set of initial rules, and a prior distribution on the value of those rules for every filter combination (Hooray, combinatorics -> you could limit this to all filters in 1 layer which should be doable imho). Like, ratio and the relative distance between filters, positioning of angle. That would also make the networks scale-invariant instead of the nasty tricks that are used nowadays (e.g., different filter scales and merge the in a common layer). Furthermore, you could directly infer knowledge about the structure of a classified input. Would be nice if we could plug in gradient descent into that shit to have a unified optimization procedure. Which can be done I think, since you have the activations of those filters, and under a Guassian prior or something, you can propagate the error to those as well.

>>896489

UBI would be tolerable if it took money from welfare. In the US cutting all welfare that'd be $8k per capita. Problem is that won't happen, niggers would blow it on crack then leftards would demand to keep giving them standard welfare too. Not to mention, if kids receive it, it'd be an incentive for the worst to have even more kids. UBI should be given with the condition you lose the right to vote and must be sterilized.

>>896323

Let me summarize to the best of my ability:

1. You say CNNs are not worth the "stochastic and combinatoric explosion", but when I ask for an example of fuzzy logic performing well on MNIST, you give a paper that augments a CNN with fuzzy logic.

2. You say that certain neural network structures "equivocate" to NP-Complete SAT3 problems. But so what? When I speak of the performance of deep learning on practical problems I'm speaking of the performance of of specific network architectures for specific classes of problems. These are not the architectures related to SAT3, and even if they were, the goal of ML is to perform well on the problem, not to find the global optimum of the objective function in polynomial time.

3. You then say that neural networks are equivalent to fuzzy logic. But weren't you just saying the neural networks suck compared to fuzzy logic because they achieve similar results but with a "stochastic and combinatoric explosion"?

You are confusing facts with knowledge. No theoretical result is going to detract from the performance of CNNs on problems like ImageNet or MNIST or the performance of WaveNet for audio generation, etc.

>>896417

This is no different to what has been going on for the last 500 years. As technology progresses, technology becomes more productive and so one the one hand, there is (potentially) more demand for labor, but the minimum bar for skilled labor gets higher. So to handle people who absolutely can't work we need a welfare state.

At the same time, the West has been told that we need more low skill labor, that's why the US "needs" Mexicans and Europe "needs" misc non-Whites. And this is to some extent true (they are not worth the problems they cause, but they do provide low-skill labor). But since low skill labor is being replaced by tech, we can simply stop immigration from non-White countries. Instead of soyboy journalists drumming up anger against the mean nerds who are taking away jobs, we can stop immigration and retrain White people to higher skill jobs. The percentage of Whites who can't hold down a skilled job is tiny.

>>896743

>You say that certain neural network structures "equivocate" to NP-Complete SAT3 problems. But so what? When I speak of the performance of deep learning on practical problems I'm speaking of the performance of of specific network architectures for specific classes of problems. These are not the architectures related to SAT3, and even if they were, the goal of ML is to perform well on the problem, not to find the global optimum of the objective function in polynomial time.

I know I'm speaking to a wall, but here I go again.

"One popular algorithm within Deep Learning is convolutional neural networks (CNNs), which have proven their utility in object classification [8] and detection [13] within imagery. CNNs can be used to process large amounts of data and sort out what and where an object is located without requiring analysts to manually sort through the imagery. While this is a useful tool, there exists a breakdown in communication between the operator and the CNN. The CNN is able to accurately generate a classification label but does not necessarily report on features that were present allowing a classification to be inferred. For example, a CNN may be able to correctly identify an object as being a ‘cat’ but not have any representation of ‘whiskers’ or ‘fur.’ Similarly, the analyst would not be able to communicate the importance of a specific feature or trait to the CNN which limits the amount and nature of feedback from an analyst to a CNN.

This problem fundamentally limits the utility of such tools. Without understanding how a CNN arrives at a solution, it is impossible to understand how adaptable the system is. This poses a complex challenge for many autonomous systems, as many of these machine learning tools are developed with controlled imagery and trained on labeled data. However, when an autonomous system is deployed the imagery may fundamentally change and there are no guarantees that the machine learning tools will operate effectively given these changes."

In other words, for those of us who are interested in generalized artificial intelligence, CNNs are fucking garbage, they consume too many resources and are capable of doing only a single task really well.

In order for analysts to make use of CNNs, they must have their structure optimized, and converted to a deterministic form which can be solved in polynomial time.

When an analyst asks a ML expert how a particular CNN works, we are often met with the phrases "It just werks", or "if you train the CNN longer it'll werk better"

The analyst is looking for a deeper understanding of the process, and more control of a finely tuned machine, and only becomes further disillusioned with the ML expert who resembles a clown more than a mathematician.

Meanwhile, the ML expert is overjoyed when his personal CNN is able to shave off fractions of a degree of error when classifying images on well defined datasets like MNIST or IMAGENET, only to watch it fail spectacularly with real world data, the only conclusion being "he didn't train long enough."

>>896743

>You then say that neural networks are equivalent to fuzzy logic. But weren't you just saying the neural networks suck compared to fuzzy logic because they achieve similar results but with a "stochastic and combinatoric explosion"?

I know this is going to blow your mind, but when something is deterministic it always gives the same output for a given set of input variables. Hence, the output will always fall within a given specified range.

The same cannot be said of stochastic processes.

The benefit of using a deterministic model you ask?

Speed, accuracy, global optimums can be found in polynomial time.

Is this not getting through?

>>896777

>>896766

You're not making a coherent point here. Yes you are saying things that vaguely make technical sense, but you're not making a coherent argument. In particular you have never addressed the question of how the methods you prefer perform better than CNNs or other deep learning approaches on problems like MNIST or ImageNet. And you can't simply dismiss MNIST or ImageNet by saying a model might perform well on these but not in the real world. Because if your model can't even perform well on MNIST or ImageNet it has no chance in the real world.

Questions: Can loading Gensim and Fasttext crash a laptop due to low RAM (8GB)?

I know that running Keras on an x230 is a bad idea, but I tried Fasttext twice, and it crashes all the time.

>>896766

> In other words, for those of us who are interested in generalized artificial intelligence

lol

If I open your brain, does the wiring tell me how it interprets how much of a faggot you are? No, you need some level of abstraction on top of that. And that is not something what we have at the moment. Yet, you claim that you and your field solved the problem, but the works you cite still use CNN's (on MNIST) and use very simplified rules. Please explain to us, the peasants, how you would define such a model that can find global optimums in polynomial time.

We layed out all the known deficiencies, there is even work (in our field) that tries to solve the same issue you are trying to address, and achieves state of the art performance on MNIST while using a regular CNN's https://arxiv.org/abs/1710.09829. Now it is your turn to produce some results without bullshitting. Is this not getting through?

why is your mom a faggot

>>896899

>Fascism

>>896899

>breaking links

https://archive.li/yKmMI

https://archive.li/QM46I

https://archive.li/8vAeE

>>896811

I don't know. Try running with parameters that produce a smaller model. If that works but the default parameters don't, this suggests it was crashing due to low RAM. If it crashes in both cases it's probably something else.

>>896917

<Polite sage

<except you didn't

(You)

>>896847

lol

Why would I teach you fags anything? I came here to laugh at you. It's not as if you are gracious for what I have informed you of already, and you certainly aren't providing me many incentives, monetary or otherwise, to continue.

Some of you were classy and introspective about it, and some of you turned into the insecure clowns that I know well.

The diligent anon will hear my criticisms and act accordingly. It is only they who deserve to find truth in this world.

Don't you know where you are?

Is it worth trying to do any ANN stuff with Geforce GTS 8600?

>>896920

How to?

>>896899

Old news. reported for spam (go start your own thread)

>>896905

>>896920

>>>/leftypol/

>>897191

>>>/cuckchan/ or >>>/reddit/

>>896766

>>896847

>>897119

Show me your repo, Git(Hub|Lab|Gud), BitBucket i don't care.

>>897152

Anything under GTX9xx will basically take hours or days

>>896914

I am using pro-loaded models, and yes, x230 with 8GB is not enough for 300 dimension 2 million entry fasttext.

>>897193

They don't have one.

>>897195

Exactly, so they should STFU.

>Not even a jupyter notebook

>>897192

>Old news

For old fags maybe, but it's breaking into the mainstream now, Nosebook is going down, and the leftist faggots that have infiltrated tech over the past decade are going to prison.

>child rape

>domestic terrorism

>censorship

It's all there, and justice is being done.

>>891602

>I don't know much about genetics and medical applications so I can't be sure, but think it's possible that ML can be used to better understand genetic data. My feeling is that biologists are territorial and have kept ML at bay until recently.

Oncology nurse here, the problem is with genetics is we still don;t fully understand why things happen. Take Breast cancer for example there is no way your genetics will tell if you will become a victim to breast cancer. Its a random occurance we still don't fully understand, however I suspect ML will be invaluable to help us with detecting and tracking early stage cancers more effectively very soon.

Name me some NLP libraries that are not SpaCy or NLTK or textBlob or (Core|Open)NLP. I will wait.

>>897194

> Anything under GTX9xx will basically take hours or days

It will take hours to run or to train ANN? Or both?

>>897339

train, of course, running a pre-trained model can be done on a phone or a potato (see Tensorflow light)

>>897288

That is reasonable. I was also thinking of sequencing the genome of cancer cells for diagnosis.

>>897337

I don't use any of those libraries or any other NLP libraries for that matter. I think I used CoreNLP in grad school but I can't remember. The only text related work I do is text classification. text classification requires very little NLP, it is mostly a generic ML problem. E.g. if you use a character CNN or LSTM then you don't need any NLP tools at all, and if you use a word-based CNN or LSTM you only need a tokenizer.

>>897194

I like to do a back of the envelope calculation for memory/storage. In this case 300 dimensions x 2 million entries x 4 bytes per float = 2.4 billion bytes. So I could see this being a problem even though it's less than 8GB.

I'm programming my replacement, and I'll be in line with you shooting "evil" corp employees for my daily rations. AMA.

>>897193

>>897195

>>897197

prove to me you aren't fags looking to suck my dick

I'll give you a chance.

Use Peano Arithmetic.

>>897192

I made the third post you quoted, and your response to me is the only lie out of your whole post.

>>897814

>quoting me out of context

I'm >>897811 by the way

>>897814

>All I need is a tokenizer and no NLP?

in most cases that is enough. You have the option to use more NLP tools in your model but you don't have to.

>Well what about POS n-gram analysis (in writeprints)?

I assume writeprints uses POS ngram analysis because it captures a person's style regardless of the topic they are speaking on. So in this case yes you need POS tagging.

>That could cause trouble with OOV. See pic.

I don't understand what you mean by this (e.g. what "that" refers to).

>How does one address misspellings?

You can ignore them and assume common misspellings will make it into the vocabulary. I believe character-based models will be robust to misspellings.

>What about the effects of misspellings and OOV that mess with POS and "vocabulary richness"?

I don't understand what you mean here.

>How do you even define function words? Is there a standard list in NLP libraries?

I assume you mean what are called "stop words" in the NLP literature ("the", "and" etc). You don't have to define them, but if you want to (e.g. to exclude them from a vocabulary) there are standard lists.

Overall, when doing text classification, NLP is used for preprocessing of text before feeding it to the model. But with neural networks you can do a lot in the model so there is much less need for this preprocessing. That is why, to the extend that I understand the, the ideas you mention aren't necessary.

>>897947

Misspellings and Out-Of-Vocabulary words could mess with POS tagging and "vocabulary richness" measures like Yule's K and Hapax/Dis Legomena , among other measures.

>>895447

>Hatebase was built to assist government agencies, NGOs, research organizations and other philanthropic individuals and groups use hate speech as a predictor for regional violence.

pure cancer. i literally hope they die.

>Language-based classification, or symbolization, is one of a handful of quantifiable steps toward genocide.

i agree with the literal meaning of this sentence but am not sure what they actually intended to say. makes perfect sense that these kind of people who think there's meaning in syntax to the extent that you can use it for law enforcement are autists

>maessa

>Person of Mexican or Hispanic descent. This term is used by Sri Lankan diaspora living in USA.

LOLLLL i was looking for a good list like this to prove the following:

>it makes no sense to filter offensive words online because you can't tell what is offensive to who. if you filter "nigger" it will only make sense in your special snowflake countries (and even then not everyone in that country will care about or know the word) while others will be missing locally offensive words. since you can't ask the user what culture and nationality he is of, you have to filter every possible offensive word, and after making it fuzzy (to prevent homoglyph bypass etc) you end up not being able to use any word period. also you need some more filters because certain words cannot be said in certain countries such as People's Republic of China or Turkmenistan for political reasons, not because they're offensive. TL;DR: Smart is dumb

of course the place i find this list is some turbo-SJW police state project

if you use homoglyphs it will bypass 99.9999% of filters out there. surely its easy to make a better one with image recognition or simply making groups of homoglyphs. anything that processes human language is retarded meme tech though

>>896899

the only relevant info about memebook i could find is that some user data was leaked. this happens every day. if you're using the web for personal information it's going to be leaked. the web isn't real software with defined bounds of information. your browser sends your shit to the 50 value-adding sites embedded in your memebook or myspace etc. you're basically a nigger for being surprised that your personal info was leaked when you posted it in what is basically public

>someone from DARPA works at facebook

autism

>AUTOMATING CHILD RAPE [citation needed]

hyperbole

>>>/pol/

>>897951

>Misspellings and Out-Of-Vocabulary words could mess with POS tagging

I see. I'm not very familiar with training/building POS tagging models, I know there are deep learning based POS taggers but I don't think they are (yet) standard. OOV words are always an issue for deep learning (and misspelling are one cause of OOV words). To be clear, they don't fundamentally prevent the model from working so it shouldn't be assumed that OOV words are a problem. I've never trained a model where misspellings/OOV words were a problem, but here is what I would try if they were:

First, the standard approach is to from an index based on the index of the word in the vocab if the word is in the vocab or the hash of the word mod some number of hash buckets if it's not. This gives an index which has range [0, vocab_size + num_hash_buckets). Then look up this index in an embedding table, and feed the result in to the next stage of the network (e.g. bag-of-words or sequential model). In pseudopython,

word_index = vocab.index(word) if word in vocab else len(vocab) + (hash(word) % num_hash_buckets)

word_embedding = embedding_matrix[word_index, :]

My alternative to the standard approach is to use a CNN instead of a embedding lookup for OOV words. In pseudopython

if word in vocab:

word_index = vocab.index(word)

word_embedding = embedding_matrix[word_index, :]

else:

word_embedding = character_cnn(word)

where character_cnn(word) = pooling_layer(conv_layer(...conv_layer(word_as_characters)))

Note this is a general approach that can work for any deep learning model that has to deal with misspellings. Let me know if you try it, I'm curious whether it would work.

> and "vocabulary richness" measures like Yule's K and Hapax/Dis Legomena , among other measures.

That's something I have no knowledge of. Too far from ML.

>>898006

Basically counting the percentage of rare words. Misspellings can create extra hapaxes.

https://en.wikipedia.org/wiki/Hapax_legomenon

>>898405

I meant his European skin color, not that he is "colored".

OP,

Have you ever heard of Steve Grand and his work on AI?

>>898916

nope

How do you handle misspellings WITH Out-Of-Vocavulary words?

1. Use fasttext n-gram features to mitigate such issue (meant for OOV, not misspellings)

2. Use basic spell checking wordlist (cannot detect if a word is in-vocabulary misspellings or OOV)

3. Statistical model that collect all misspellings and OOV, isolate OOV, then use #1 to fix it

Possible issues: What is the cut-off point between properly spelled OOV and misspelled in-vocabulary?

Another issue: What is the cut-off point of proper and misspelled OOV?

>>898938

I don't think there is a fundamental difference between misspellings and OOV words. Given this, 1. makes sense. I don't understand what you are proposing in 2. and 3.

>>898959

There is. OOV is meant to have a different meaning, misspellings should have the same meaning after text cleaning

>>898965

To me there is no fundamental difference as common misspellings can become their own item in a vocab (e.g. "teh" vs "the") while uncommon misspellings can be handled like OOV words.

You seem to think it's important that the model be given the canonical version of each word, whether in or out of vocab. But I don't see why. It's no different to the situation with synonyms. You wouldn't say that you have to resolve synonyms to some canonical version, e.g. map "pleasant" and "agreeable" to the same vocab item, would you? So why is it a big deal if a model gets multiple versions of the same word?

>>899096

I would say that there needs to be a threshold to "see" an OOV word is not a misspelling of an in-vocabulary word, because the meanings would be different.

If I were to use word2vec, spell-checking would be necessary as there are no way in handling misspellings. But if the data has OOV, then there will be a loss in information.

If I were to use Fasttext on the other hand, spell-checking AND OOV might not be necessary, as they have ways of handling OOV words using n-gram hashes. Spell-checking is an after-thought.

Well what about POS tagging using SpaCy? That raises another issue entirely. If it uses "expert systems" to do it then it can only handle classical cases of OOV (e.g. old literature) and not 21st century lexicons.

>>891597 (OP)

any recomemnded starting places

>>899514

Do you have a graphics card?

>>899485

I'm not really following you at all at this point. They way you are writing just doesn't map to things that I can assign a concrete meaning to. E.g. to me "OOV" is just short for "out of vocabulary" whose concrete meaning is "(a word) not in a (particular) vocabulary that has been constructed". But you seem to use it as a shorthand in ways that I can't parse, e.g. "if the data has OOV" for me should translate to "if the data has out of vocabulary word(s)" but this has no meaning to me, since you haven't specified the vocabulary and in any case, (in practice) any vocab will not cover all words in the data.

>>899514

An anon suggested >>892978. I would also suggest https://www.coursera.org/learn/machine-learning although I haven't taken it.

>>899544

Graphics card will be essential for real deep learning, but it will take a while to develop the background knowledge needed for this, so it isn't needed to get started.

>>899544

no

>>899629

>Graphics card will be essential for real deep learning, but it will take a while to develop the background knowledge needed for this, so it isn't needed to get started.

i have done the "ML w/o a phd" from google mini lab, changed the training data, added some tags

i keep getting bad magic number problems when tring to use trained models diffenet pythons, not enough knoledge to modify ether to the other

> https://www.coursera.org/learn/machine-learning

missed that thx

>>899629

The issue is that a normal word embedding system or POS tagging system uses wordlists or dictionaries.

There are, definitely, Out Of Vocabulary words that contains important information that is not in the wordlist.

Assuming such words are being repeated heavily in the data set, extracting it would be relatively easy.

With misspellings of such words, extraction would be harder. POS tagging is a bigger of an issue that requires manual entries.

>>899867

>The issue is that a normal word embedding system or POS tagging system uses wordlists or dictionaries.

>There are, definitely, Out Of Vocabulary words that contains important information that is not in the wordlist.

>Assuming such words are being repeated heavily in the data set, extracting it would be relatively easy.

I think what you are saying is that taking a pre-trained embedding comes with its own vocab, which was trained on a specific dataset. The dataset used to train the embedding might have a different distribution of words to the one you are using. Therefore it would be good to somehow augment the dataset by creating a vocab of words that are frequent in your dataset but missing from the vocab of the pre-trained embedding. To be specific, you could use the pre-trained embeddings, and also train embeddings for the new words from scratch.

If this is what you meant, I agree so far, this is a good idea.

>With misspellings of such words, extraction would be harder.

Now we're back to my point in >>899096 (You) which is that there is no fundamental difference between misspellings and OOV words. In both cases you have a word the model can't recognize and must fall back on some other information, e.g. character ngrams. Do you agree that character ngrams contain useful information whether or not the word is misspelled?

>POS tagging is a bigger of an issue that requires manual entries.

I'll leave this aside for a moment while we focus on the word-embedding case.

>>900380

>To be specific, you could use the pre-trained embeddings, and also train embeddings for the new words from scratch

Using pre-trained word vectors sounds fun and all, BUT there will be issues for OOV words. FastTest uses n-grams, and people have tried sub-word embeddings.

https://github.com/bheinzerling/bpemb https://github.com/yuvalpinter/Mimick https://github.com/claravania/subword-lstm-lm https://github.com/Leonard-Xu/CWE

https://github.com/yxtay/char-rnn-text-generation https://github.com/jarfo/kchar https://github.com/yoonkim/lstm-char-cnn https://github.com/carpedm20/lstm-char-cnn-tensorflow

The Idea of re-training Word Embedding model sounded sweet... but then you realized "I don't have that much resources to do re-training, just one GTX10xx graphic card and an i7".

Another issue is the problem of small datasets (e.g. 1000 twitter user's timeline spanning ~3 months), it probably won't change or add to the word embedding in that case.

The last thing is that IF you do have large datasets for social media, how do you clean up foreign languages? Emojis? ASCII/Unicode Art? Those things will bug up FastText training.

>Do you agree that character ngrams contain useful information whether or not the word is misspelled?

Yes, but we should stride for word-level accuracy. pre-trained word vectors may not contain OOV, but in many applications, people will prepare "slang" dictionaries/wordlists.

Such wordlists would expand darastically over time, but we can be assured that those new words exists, and are separate entities when compared to the word vector's list.

The enemy has already assembled lists that target us, case in point these 4 articles

https://www.npr.org/2017/09/06/548858850/-ghost-skins-and-masculinity-alt-right-terms-defined or http://archive.fo/zmCaN

https://qz.com/1092037/the-alt-right-is-creating-its-own-dialect-heres-a-complete-guide/ or http://archive.fo/vXaZp

https://www.aljazeera.com/indepth/features/2017/10/dictionary-understand-171002123412523.html or http://archive.fo/xRJhH

https://www.alternet.org/election-2016/7-demeaning-alt-right-terms-used-racist-trump-followers or http://archive.fo/AErnq

>>POS tagging is a bigger of an issue that requires manual entries.

>I'll leave this aside for a moment while we focus on the word-embedding case.

I think POS is important when tackling writeprints (human identification), which could double as a metric on user behavior, personality and sentimentality.

>>900464

I think we've reached the limit of useful communication on technical matters. You clearly think misspellings are important but I'm not understanding your reasoning, and generally I'm having a hard time understanding what you're saying. My general advice would be to consider standard methods and not assume that you have specific insights on why standard methods won't work. It reminds my of a project I did on NLP in grad school, where I had a very non-standard approach, and the feedback I got from the teaching assistants was that my approach was very interesting, but performed very poorly compared to the other students. The lesson I took from this was that good, intuitive ideas sometimes don't pan out in practice.

As I mentioned earlier, I consider myself a centrist, but I think people on the alt-right/far-right are necessary to counterbalance the far left so I wish you luck.

Another thing to consider is that ML is only as good as it is useful. I think a lot of the work on ML related to politics is not directly useful and only serves propaganda purposes. It's possible that all the ML that the ADL, SPLC etc are doing serves no practical purpose, but helps spread the idea that there is a well defined notion of "hate speech" or "alt-right". After all, if ML can detect "hate speech" that must mean "hate speech" is a real thing, right? Similarly, the (((coincidence detector))) app was not particularly useful, in my opinion, but it helped spread the message that there is a well defined group of people who consistently (and somewhat openly) advocate against the interest of White/European people.

In terms of propaganda value, I don't see an obvious project where ML can help the right. But in terms of practical value, I see one clear project which is evading bans on twitter, facebook etc. So I would encourage you to focus on that, and again, to try to stick to standard techniques.

>>900884

>You clearly think misspellings are important but I'm not understanding your reasoning

The reasoning is that, unlike Word Embedding, where there is a ways of accepting misspellings and OOV words in a fuzzy manner (Fasttext with n-grams), currently there are NO methods of POS tagging that CAN handle OOV and/or misspellings within mainstream NLP packages like SpaCy, NTLK and others (stemming does not count as POS information is stripped).

As I said

>I think POS is important when using writeprints for human identification, which could double as a metric on predicting generalized user behavior, personality and sentimentality.

>As I mentioned earlier, I consider myself a centrist

I was referencing the "Urban Dictionary" method of vocabulary spotting, my embarassing example is meant for other project.

>Another thing to consider is that ML is only as good as it is useful. I think a lot of the work on ML related to politics is not directly useful and only serves propaganda purposes.

>In terms of propaganda value, I don't see an obvious project where ML can help the right. But in terms of practical value, I see one clear project which is evading bans on twitter, facebook etc. So I would encourage you to focus on that, and again, to try to stick to standard techniques.

That is some painful truth right there