How Machines Learn

How Machines Learn

On the internet, the algorithms are all around you. You are watching this video because an algorithm brought it to you (among others) to click, which you did, and the algorithm took note. When you open the TweetBook, A the algorithm decides what you see. When you search through your photos, A the algorithm does the finding. Maybe even makes a little movie for you. When you buy something, A the algorithm sets the price and A the algorithm is at your bank watching transactions for fraud. The stock market is full of algorithms trading with algorithms. Given this, you might want to know how these little algorithmic bots shaping your world work, especially when they don’t. In Ye Olden Days, humans built algorithmic bots by giving them instructions the humans could explain. “If this, then that.” But many problems are just too big and hard for a human to write simple instructions for. There’s a gazillion financial transactions a second, which ones are fraudulent? There’s octillion videos on NetMeTube. Which eight should the user see as recommendations? Which shouldn’t be allowed on the site at all? For this airline seat, what is the maximum price this user will pay right now? Algorithmic bots give answers to these questions. Not perfect answers,
but much better than a human could do. But how these bots work exactly,
more and more, no one knows. Not even the humans who built them, or “built them”, as we will see… Now companies that use these bots
don’t want to talk about how they work because the bots are valuable employees. Very, VERY valuable. And how their brains are built is a fiercely guarded trade secret. Right now the cutting edge is most likely very ‘I hope you like linear algebra’, but what the current hotness is on any particular site and how the bots work,
is a bit “I dunno”, and always will be. So let’s talk about one of the more quaint but understandable ways bots CAN be “built” without understanding how their brains work. Say you want a bot that can recognize
what is in a picture. Is it a bee, or is it a three? It’s easy for humans (even little humans), but it’s impossible to just tell a bot
in bot language how to do it, because really we just know
that’s a bee and that’s a three. We can say in words what makes them different,
but bots don’t understand words. And it’s the wiring in our brains
that makes it happen anyway. While an individual neuron may be understood, and clusters of neurons’ general purpose vaguely grasped, the whole is beyond. Nonetheless, it works. So to get a bot that can do this sorting, you don’t build it yourself. You build a bot that builds bots,
and a bot that teaches bots. These bots’ brains are simpler,
something a smart human programmer can make. The builder bot builds bots,
though it’s not very good at it. At first it connects the wires and modules in the bot brains almost at random. This leads to some very… “special” student bots sent to teacher bot to teach. Of course, teacher bot can’t
tell a bee from a three either; if the human could build teacher bot to do that,
well, then, problem solved. Instead the human gives teacher bot a bunch of “bee” photos, and “three” photos, and an answer key to which is what. Teacher bot can’t teach, but teacher bot can TEST. The adorkable student bots stick out their tongues, try very hard, but they are bad at what they do. Very, VERY, bad. And it’s not their fault, really,
they were built that way. Grades in hand, the student bots take a march of shame back to builder bot. those that did best are put to one side, the others recycled. Builder bot still isn’t good at building bots, but now it takes those left
and makes copies with changes in new combinations. Back to school they go. Teacher bot teaches – er, tests again, and builder bot builds again. And again, and again. Now a builder that builds at random,
and a teacher that doesn’t teach, just tests, and students who can’t learn, they just are what they are, in theory shouldn’t work, but in practice, it does. Partly because in every iteration, builder bot’s slaughterhouse keeps the best and discards the rest, and partly because teacher bot isn’t overseeing an old-timey, one-room schoolhouse with a dozen students, but an infinite warehouse with thousands of students. The test isn’t ten questions, but a million questions. And how many times does the test, build, test loop repeat? As many as necessary. At first students that survive are just lucky, but by combining enough lucky bots, and keeping only what works, and randomly messing around with new copies of that eventually a student bot emerges that isn’t lucky, that can perhaps barely tell bees from threes. As this bot is copied and changed,
slowly the average test score rises, and thus the grade needed to survive the next round gets higher and higher. Keep this up and eventually from the infinite warehouse (slaughterhouse) a student bot will emerge, who can tell a bee from a three in a photo it’s never seen before pretty well. But how the student bot does this, neither the teacher bot nor the builder bot, nor the human overseer, can understand. Nor the student bot itself. After keeping so many useful random changes,
the wiring in its head is incredibly complicated, and while an individual line of code may be understood, and clusters of code’s general purpose vaguely grasped, the whole is beyond, nonetheless, it works. But this is frustrating, especially as the student bot is very good at exactly only the kinds of questions it’s been taught to. It’s great with photos, but useless with videos or baffled if the photos are upside down, or things that are obviously not bees, it’s confident are. Since teacher bot can’t teach, all the human overseer can do is give it more questions, to make the test even longer, to include the kinds of questions the best bots get wrong. This is important to understand. It’s a reason why companies are
obsessed with collecting data. More data equals longer tests equals better bots. So when you get the “Are you human?” test on a website, you are not only proving that you are human,
(hopefully), but you are also helping to build the test to make bots that can read, or count, or tell lakes from mountains, or horses from humans. Seeing lots of questions about driving lately? Hmm…! What could that be building a test for? Now figuring out what’s in a photo, or on a sign, or filtering videos, requires humans to make correct enough tests. But there is another kind of test that makes itself. Tests ON the humans. For example, say entirely hypothetical NetMeTube wanted users to keep watching as long as possible? Well, how long a user stays on the site is easy to measure. So, teacher bot gives each student bot a bunch of NetMeTube users to oversee, the student bots watch what their user watches, looks at their files, and do their best to pick the videos
that keep the user on the site. The longer the average, the higher their test score. Build, test, repeat. A million cycles later, there’s a student bot who’s pretty good at keeping the users watching, at least compared to what a human could build. But when people ask:
“How does the NetMeTube algorithm select videos?” Once again, there isn’t a great answer other than pointing to the bot, and the user data it had access to, and most vitally, how the human overseers
direct teacher bot to score the test. That’s what the bot is trying to be good at to survive. But what the bot is thinking, or how it thinks it,
is not really knowable. All that’s knowable is this student bot
gets to be the algorithm, because it’s point one percent better than the previous bot at the test the humans designed. So everywhere on the internet, behind the scenes,
there are tests to increase user interaction, or set prices just right to maximize revenue, or pick the posts from all your friends you’ll like the most, or articles people will share the most, or whatever. If it’s testable, it’s teachable. Well, “teachable”, and a student bot will graduate from the warehouse
to be the algorithm of its domain. At least, for a little while. We’re used to the idea that the tools we use, even if we don’t understand them, someone does, but with our machines that learn we are increasingly in a position where we use tools, or are used by tools, that no one, not even their creators, understand. We can only hope to guide them with the tests we make, and we need to get comfortable with that, as our algorithmic bot buddies are all around,
and not going anywhere. OK. The bots are watching. You know what’s coming. This is where I need to ask you… To like… comment… …and subscribe. And bell me. And share on the TweetBook. The algorithm is watching. It won’t show people the video… unless you do this. Look what you’ve reduced me to, bots. What do you want? Do you want watch time? Is that what you want? Fine. (sigh…) Hey guys, did you know I also have podcasts you can listen to? Maybe even just in the background while you’re tidying up your all room for hours? Or whatever? There’s hours of audio entertainment for you,
and watch time for the bots overseeing your actions. Go ahead and – and take a click.
Entertain yourself. Help me. Help the bots.

Comments (100)

  1. Well this means if we do a enough complex robot brain that can store so much memories that means we can put it in a hyper realistic fast fowarded life sim and make it as clever as a 3 year old


  2. This is basically similar to school. 99%
    Except the slaughtering 1% though….

  3. Wow its…….fun

  4. What happens if the bots are self aware

  5. Super well made vid

  6. Interesting video.

  7. New idea : program it

  8. Bots are really getting a lot of retention time out of me today

  9. 2:57 Entire American school system explained.

  10. Technological Darwinism

  11. Machine doggo and the troubles

  12. NIce try, A the algorithm. You can't trick me.

  13. Commenting to save cpg grey

  14. Fine sure I love how it’s telling me to do this and I am

  15. ohhhhh so i do them a free teaching with those damn captchas for some reason i am boiling mad because these was not what i though i was doing
    but to be fair its kind of…. well alot of B S having to prove my humanity it is true although it does prevent login trolls but still i do believe …. no… WE SHOULD GET PAID FOR BUILDING OUR DOWNFALL (skynet) DAMN IT IF IM GOING TO GET BRUTALLY BUTCHERED BY MACHINES IN THE FUTURE AT LEAST PAY ME I CAN BARELY AFFORD RENT FFS i want a happy death not dying on the streets ill soon die from diseases than a killing machine say sike right now

  16. *But why do dem bots hav apples on their test benches*?????🤔🤔

  17. Dog in a bee suit
    me: is it a bee?
    bot: well yes but actually yes

  18. Jimmybot: yay I’m on top

    Builderbot: yeah but Harry is 0.0001 better then you * kicks Jimmy into furnace *

  19. So if I just turn off this I kill a bot?

  20. 40% of the time my algorithm bot is high lol


  22. Well this is ALMOST accurate. Good enough for the layman, I suppose.

  23. Very interesting video. We actually did a blog on the subject if anyone is interested in getting more information.

  24. This is a comment.

  25. Great video ! What I enjoyed the most bedside the understandability of it was the Hilarious drawings 😂😂😂

  26. 1:41 somebody caption the image

  27. It would be great if we could make an AI algorithm the physics of warp drive

  28. Humans provide the data, student bots make a decision. Teacher not compares the answer it has with the decision the student bots made. I think.

  29. It's just natural selection system but with bots

  30. if the bots want comments here they go

  31. commenting for the bots

  32. Black Mirror easter-egg 4:02

  33. Basically selective breeding but with robots

  34. Oh boy I'm here yet again. And im sure that i was here before because i left a thumbs up last time.

  35. "Very good."

    "But not good enough."

    Asian parents in a nutshell.

  36. Amazing amazing what a fun way to understand Neural Network.

  37. Every time you don't finish a video a robot child burns.

  38. Great comprehensive visualization of machine learning!

  39. In my days, they used to call it "brute force", not "machine learning"

  40. but AI database and orders was programmed and puted by humans and any other strange behavior or actions will face it the AI wont recognize it so that doesn't make him have a mind of its own its more like its just a bot with a memory filled with specific Orders and commands by humans thats all

  41. This could make an interesting Black Mirror episode.

  42. 5:40 hopefully… Well that would quickly become hopeless, I assure you.

  43. After watching the advertisements I don't feel like watching your video now, cause they served the purpose.

    Especially Simplilearn ones.

  44. As someone who did their master's thesis on ML, this is highly misleading. He is basically describing a genetic algorithm which almost no one uses rather than backpropagation which is common

  45. The bots represent the coding .

  46. Cup gray is awesome

  47. The road signs are probably for the self driving cars, maybe?

  48. Hey so guys this matches domestication and natural selection did you know?

  49. t h e a l g o r i t h m

  50. cough evolution cough

  51. May I have a bot instead of an FBI guy?

  52. So i can make bots in a mincraft server, i will be able to make a stock market!
    Grian: im 4 paraell universes ahead of you

  53. I did em allllllll ❤❤

  54. I must leave a comment LOL

  55. So i got a question, what if we ran an algorithm, over several years, we teach it first basic knowlage, like how our parents teach us things like wearing clothes. Then once they reach a basic knowlage, we start sending it through the a comon core curriculum?

  56. "How to make a Neural Network:"
    1. Press all the buttons
    2. Repeat until it works

    Awesome video BTW 😀

  57. Algorithm comment here

  58. “you clicked on the video” i didnt, autoplay did

  59. You must go into the furnace. For science.

  60. Beautiful, smart, robots

  61. the bots are watching

  62. Cheese

    Are you happy now, bots?

  63. So wait
    Your telling me that the reason I'm told to.

    Like, comment and subscribe.

    is because some random group of jackasses decided to say.

    Like, comment and subscribe

    And that counted as engagement so they got pushed more so it fed its own cycle.

    And now everyone does because someone does.

    Fuck sake.

  64. I’m just commenting to help with the bots

  65. 8:50 "Help me help the bots" DON'T TRUST HIM HE'S ON THEIR SIDE

  66. That moment when I make a comment just because he asked me to.

  67. Blink Twice, if you are the NEW NARRATOR in this show and the previous one was killed by bots!

Comment here