The Seventh Sense

Comments (Archived):

1. Networks are great except that they can also amplify biases and unrepresentativeness as some AI researchers from MS are discovering.”The blind application of machine learning runs the risk of amplifying biases present in data. Such a danger is facing us with word embedding, a popular framework to represent text data as vectors which has been used in many machine learning and natural language processing tasks. We show that even word embeddings trained on Google News articles exhibit female/male gender stereotypes to a disturbing extent. This raises concerns because their widespread use, as we describe, often tends to amplify these biases.”* http://arxiv.org/abs/1607.0…* https://www.technologyrevie…

   1. The connection between networks and communities i one of the most critical ones in marketing today. Not understand but the key to how we market, distribute and build brands.They are connected but not at all the same.

      1. I agree with you that crowd-sourced curation is key. Network facilitates the quantity of content and interactions. Community the quality of content and interactions. Quantity and quality are connected but not the same (just as multiplier effect ≠ cause or WHY something happens).The problem in AI is that tools like Word2Vec are created by a handful of Professors and PhDs in AI and it’s their mental models & biases towards language that get imposed over billions of people.Mathematically, that doesn’t even make sense: a handful of academics is not even a representative sampling population in probability & statistical terms.Wrt network effects, we can take a leaf out of Gandhi.The negative side of network effects was also shown during the global financial crisis of 2008. The wrong types of data & algorithms propagated quickly across the banking networks.

         1. We all know why it happens and will continue to happen, although my Borg associates guarantee it will eventually stop.

         2. The wrong types of data & algorithms propagated quickly across the banking networks.Yes! A clear case of emotion overcoming logic and rationality.

            1. Clear case of machine network effects not being intelligent enough to know when to stop.* http://www.nytimes.com/2008…I didn’t need the MS’s latest research to tell me the unseen risks and biases in Word2Vec and other frameworks created to-date in AI.This assumption that the machines are better and smarter than us because they’re not emotional and irrational like we are is an UNTRUTH inherited from Descartes. He was one of the most ignorant and dangerous philosophers because he argued for the separation of mind, body and emotions so that we arrive at this type of Rational Logic:Invalid = 0, valid = 1Female = 0, male = 1 Black = 0, white = 1Angry = 0, happy = 1which then affects how we build AI and results in things like Word2Vec’s binary classifications.Yeah so I invented a system (with Da Vinci’s help) to kick Descartes’ butt back to the Dark Ages where he belongs.

            2. > Clear case of machine network effects not being intelligent enough to know when to stop.Easy special case of the more general result that the machines are not “intelligent enough” even to pee in a pot, and that is a simple special case of the much more general result that the machines are no more intelligent than my rusty lawn mower. They at best are machines, and a rusty lawn mower is not very different. So, instead of a piston and a connecting rod, they have some arrays and Do-loops. Machines? Yes. Intelligence? NO!Again, there is nothing at all intelligent, artificial or otherwise, about artificial intelligence.This determined, obsessive, insistent, grotesque, outrageous, irresponsible, dangerous misuse of language is a fatal cancer on the AI community. Get out your heavy winter clothes, folks: Another AI winter is well on the way. The next AI spring will need about 25 years for another whole generation of naive, vulnerable, gullible suckers.

            3. By the way, AI researchers like Yann Le Cun of FB and Panasonic Labs are now all trying to ADD EMOTIONS to the AI.Hence FB’s emotion reaction buttons but…Hmmn, well… They run into some problems.Someone would have to kick Descartes’ butt AND Bayes’ butt (the source of the Bell curve) at the same time AND pretty much all the existing classifiers like Word2Vec in AI.That’s even before we get to Neural Networks (aka Deep Learning) not being the optimal structure for data parsing and language understanding.

         3. > representative sampling population in probability & statistical terms.”Representative” and “sampling … in probability …,”, i.e., simple random sampling, are not the same thing.Exercise: When are they approximately the same thing?

      2. Agree. For me the AI 2 AI created agoes will show a different way to approach things. The data experts can see churn or a new planet, most will see churn.

   2. https://github.com/kovetski…

      1. interesting, not sure of the relevance.

         1. Nor am I and could not figure it out and hoped someone else would. Mea maxima culpa

   3. I have to suspect that the article makes its first serious mistake already in the title: I doubt that in any meaningful sense they are working in a vector space.Why not? Because a vector space is more than just a set of elements we might call vectors because we need to be able to multiply vectors by scalars and also add vectors. Then we continue with linear independence, bases, linear transformations, etc.What they have may be a set of lists, yes. A vector space, no.A set of lists is hardly mathematics.Glad they respect math enough to want to use it, but they are not really.Math is much of the most powerful stuff in all of civilization, and there may be applications in natural language processing, but it appears that this article hasn’t found such an application yet.Here they are just throwing around terms they don’t understand, and that appears to be one of the main parts of all descriptions of AI by AI workers.AI: AI winter, spring of hope, summer of hype, fall of failure, and AI winter again. Been going on this way for much of a century by now. Silly stuff.

      1. They (Stanford + Google) treat it as a vector space and treat language as if it’s Brownian motion in a matrix.it’s Classical Mechanics as applied to NLP.Look, Professor Chris Manning of Stanford recently conceded: “Higher level language is of a DIFFERENT NATURE to lower-level pattern recognition.”Well, as mathematicians, we know lower-level pattern recognition means probability & statistical methods.And Word2Vec is an example of lower-level pattern recognition… So …That means someone would have to invent tools that are “of a DIFFERENT NATURE” if Nat Language understanding by the machines is to be solved.

         1. Again, yet again, take one of their vectors and multiply it by a, say, a real number, positive, negative, or zero. Now what the heck does that mean in language? Right: Nothing. Next, take two of their vectors and add them. What vector do they get? Right — there isn’t one. So, they don’t have a vector space. So, they don’t have vectors. As I indicated, maybe they have lists.At Google? I’m totally not impressed. At Stanford? WHO at Stanford? K. Chung? I doubt it.Sure, I studied some of Brownian motion: Each sample path is continuous but almost surely differentiable nowhere. Brownian motion is a Markov process and a martingale. It crosses the X axis according to the arcsine law — which is astounding. Even more astounding, there is an envelope for Brownian motion: So, for any h > 0, no matter how small, there are two positive curves h apart that form an upper envelope so that for each sample path almost surely Brownian motion is above the lower curve infinitely often and above the upper curve only finitely often. And similarly for two negative curves. Brownian motion is an independent increments process with the increments stationary in time — so by the central limit theorem it must have Gaussian distribution — and, yes, the mean is zero.So, right, I did study some of Brownian motion.From a suggestion of S. Kakutani, can use Brownian motion to solve the Dirichlet problem. There are lots of details, really, a whole field, in, say,Robert M. Blumenthal and Ronald K. Getoor, Markov Processes and Potential Theory.That is likely a good place to learn a lot more about Brownian motion.Given a sample path of Brownian motion in the plane, there exists a real valued function on the plane 0 on the sample path, strictly positive otherwise, and infinitely differentiable. This was an observation of A. Karr from a much more general theorem of mine. E.g., the sample path can be replaced by any closed set, e.g., the Mandelbrot set and Cantor sets of positive measure. I cooked up this result to settle an old issue in the Kuhn-Tucker conditions.But the idea of Brownian motion on a matrix seems undefined and, really, absurd.Maybe they mean a finite state space Markov process.Brownian motion stands to have nothing important to do with natural language processing. Actually, Markov processes are astoundingly general things, e.g., if let the history be the state, darned near anything can be a Markov process. For this, need to get into being adapted to a current of sigma algebras, and for that need a good course in graduate probability — Loeve, Neveu, Breiman, Chung, and others. Will also want the strong Markov property, that is, still have a Markov process not just at times but also at stopping times — defining a stopping time is a bit tricky.But for a matrix, i.e., a transition matrix, that stands to be a simple, finite state space Markov process and a long way from anything in natural language processing.You are talking stuff from people who don’t know what they are doing, are throwing around math topics they don’t understand, and are wasting time. We’ve done this before. You have been walking in muddy waters. I suggest you get on clean, dry, higher ground.For writing software to do natural language processing, especially understanding, maybe a first step would be to understand basic English grammar, say, diagramming sentences. IIRC that’s been coded. Okay, that takes us into something like syntax. Now for the semantics, for that may have to get at meaning in some quite direct and definite way. What I’m doing getting at meaning in my startup is useful and powerful for my startup but too weak for natural language understanding. I have some very rough ideas on what to do, but they have nothing significant to do with Markov processes.

            1. Word2Vec creators are:* Thomas Mikolov, leader of project, was at Google Brain now at Facebook AI Research* Jeff Dean, now Head of Google Tensorflow* Ilya Sutskever, was at Google Brain now Research Director at OpenAI (Elon Musk’s initiative)* Greg Corrado, Tech Lead at Google* Kai Chen, GoogleGloVe (Global Vector for Word Representations) creators are:* Chris Manning, Professor of Linguistics & CS at Stanford* Richard Socher, ex-Stanford, founder of Metamind now Chief Scientist at Salesforce* Jeffrey Pennington, ex-Stanford now at Google NY

            2. Right: I didn’t see K. Chung mentioned, as inK. L. Chung and R. J. Williams, Introduction to Stochastic Integration.a good place to learn a lot about Brownian motion and stochastic processes in continuous time.Your list doesn’t sound like people who should be talking about Brownian motion. IMHO, if they understood Brownian motion, then they would not be considering it for natural language processing.Your list sounds like a lot of commercial firms with more money than good sense throwing money away.

2. Audible ordered! thanks!

3. great name for a book about seeing around corners.

4. Does not blog often but always from the right mindset: https://joi.ito.com/“specifically, machine learning is still very difficult to do, the people who are training the machines are usually not domain experts” The comments on your July 28th blog “The AI Nexus Lab” were a good exchange from experts to the interested (me) and why I think AI 2 AI created algos are sooo very interesting.

5. I have yet to read a good book on networks. They tend to be long on words and short on insights, which is typical for business books, but I just got this on on audible so I’ll give it a try.

   1. The truth is that most business and marketing books in general are poor. And boring = poor in my mind for any topic.I just reread The Hard Things about Hard Things and need to say that this is one of my favorite reads. I keep learning and its from the heart.

      1. Yeah that was a good one. Seems like the best business books are anecdotal. I really enjoyed Chaos Monkeys as well.

         1. just bough chaos monkeys as i’m traveling all next month and need stuff to read.becoming a bit occupied with the continuum of info–>understanding–>experience–>wisdom –>visionMost crap written is in the first three, taking leaps to believe they are in the latter,Most great stuff is heavily grounded in the latter three.And by definition that drives both realistic inspiration and humility.

            1. Thats a good way of looking at things. Thanks!

            2. Post some titles if you have any suggestions for the “wisdom” category. I started Jacob Needleman’s book “Money and the Meaning of Life” this week, but it’s from the 80’s, so I don’t know how fresh the insights still are. It’s starting off little boring.

            3. Have you read “The Greatest Salesman in the World”?https://en.wikipedia.org/wi…https://en.wikipedia.org/wi…While it has religious overtones at the end, the overall message is sound, IMO.

            4. This book really opened my eyes to the fact that every problem is a people problem: https://www.amazon.com/Secr…And The Personal MBA is a great overview of 150 business books: https://www.amazon.com/gp/a…

      2. Totally agree about Hard Things. Listened to it on Audible and the narrator is excellent as well. As opposed to another book I’m listening to now where the content is good but the narrator, who is also the author, is awful. I’m going to have to just read it myself.

      3. why are most so poor?

         1. Why?They don’t have, or at least don’t effectively use, any methodology that would get them much past campfire stories.Good knowledge in business would about have to be applied social science, and doing good and significant science in any sense in social science is darned difficult.Moreover, the people trying don’t know enough about the good science of the past to have good examples or “shoulders of giants”, e.g., as Newton did, to stand on. About the best they have going for them are statistical hypothesis tests and analysis of variance that they borrowed from, say, production quality control and agriculture. They are at near zero in effective theory construction or anything in math that would help them.Good science in social science is super tough, and the people trying don’t have the right stuff. Those people are nearly all ones who thought that hard science and advanced math were too difficult, so they went without good math tools and abilities to a field where good science is even more difficult. Result: There aren’t many good results. Then for the special case of business, the situation is, right, still campfire stories.Some of the campfire stories with examples and anecdotal evidence can make suggestions that might be useful in practice, but that is a long way from good knowledge.Besides, the suggestions can be wrong about as often as right. Long medicine was in that situation, e.g., boil these rat tails and try that, and, eventually, adopted “First, do no wrong.”. Then for a long time medicine did diagnosis and then stopped because they knew that they didn’t really know what interventions might be safe and effective. Slowly with some science, e.g., the microbe theory of disease, biochemistry, anesthetics, antibiotics, medicine began to make progress. Social science is still a long way from any of that.That’s why.

   2. which is typical for business booksI stopped reading business books a long time ago. I don’t even remember the last one I had the patience to read one. What I don’t like is everything is edited in one direction towards the point that the author is trying to make with no opposing points of view other than perhaps a “to be sure” type paragraphs.Edit: For the purpose of this comment I consider business books to be on soft topics that are a matter of opinion and experience and not topics that are technical in nature.

6. So, Ramo is interested in networks, apparently the Internet or other networks built using the Internet. He sees issues of money, power, threats.He seems to find something significant in common with Trump’s success, the rise of ISIS, and AI. Uh, right away, I don’t find anything significant and new from a combination of or in common to those three.So, Ramo’s seventh sense is his one step past an old sixth sense and says that when look at something, think also about the implications of its network connections (or lack thereof). Okay. But, we’ve been doing that for 20+ years, right?So, we see:1. More perfect markets, e.g., for each seller, easier to get more buyers to consider what is being sold, and for each buyer easier to consider more sellers.One place where there may still be big gaps is used industrial equipment. If you are selling a good, used chiller able to cool down 5000 hogs just killed and hanging, may have a tough time finding many competitive buyers. Same if trying to buy such an item. Same for a lot of good, used big stuff — equipment, supplies, buildings, land, etc.Sure in its early years, Sears went shopping, found some good products from all over the country or world, described them in their catalog, and used a reputation for quality and honesty, return privileges to sell the products all over the US and the world. So, they had a network of suppliers and then a network of buyers. For the physical movement, they used trains, trucks, USPS, etc. Now Amazon is doing much the same but better. So, is Wal-Mart. Okay, but we know that. It takes a darned good Web site, but lots of organizations can do that.2. Better access to information. There’s a lot of information in the world, and a huge fraction of it is readily available in some trillion Web pages on the Internet.3. One bottleneck is that we would like still better information. That usually takes work.4. With so much in information, products, services, places, etc., we need better means of search, discovery, recommendation, curation, notification, and subscription. Keyword/phrases are poor for maybe 2/3rds or more of the whole need, and finding better means has been slow.5. With the Internet and related networks, it should be easier to create and sell niche products. Sure, Etsy is doing some of this, but the sense was that there would be much more.6. Really, there isn’t as much information readily available as we could wish. One bottleneck is that only a small fraction of the population is actually very good at presenting information, in text, still images, video lessons, etc. There is a lot of junk information out there and less of the good stuff than I expected by now.What am I missing from Ramo and his subject?