Chapter 18 — Mica’s Kitchen
Rufus waxes lyrical again about some of the prospects machine learning is likely to open up.
Followed by Chapter 18 —— Mica’s Kitchen, in which Saskia explains to Mica how a neural net works.
Listen to full episode :
Hello Friends,
Today is one of the day’s I’ve been promising from the start. Today, we’re going to get our hands dirty and actually get a sense of how some machine learning works. But before Saskia gives us our first lesson in the nuts and bolts of artificial intelligence, there’s a little something I need to get off my chest.
There is, in life, sometimes a risk of being overwhelmed by people who attempt to sound smart by throwing buckets of cold water on delicate new ideas. Happily, those new shoots sometimes absorb the dousing and simply grow stronger. Surprise, surprise, I have a specific something in mind here. Namely, I’m bugged by dystopian rants against AI that claim it’s merely going to take all our jobs and make labor cheap (by which those perpetrators of this line of thinking mean that most people will no longer be able to earn a living). To me, such rants are predicated on the idea that there are a set slew of jobs, that as AI gobbles them up, there will be less left for everyone to fight over, and that, thus, the cost of labor will be lower since it’s so oversupplied. This is called zero sum thinking! Unsurprisingly, it’s not something I subscribe to.
In fact, this not what happens in our world. The world is much more positive sum! Just think back to the advent of the industrial age, or the age of computing. In both cases, productivity shot through the roof. Far less people did the jobs of the previous era, but, lo and behold, new jobs appeared. And, no, wages haven’t fallen through the floor, quite the converse. Indeed, it shouldn’t be surprising that if more is being produced per person that everyone does better in the end (notwithstanding that I do personally hope there is a rebalance in the distribution of wealth).
For those interested in diving a little deeper, this blog post by Noah Smith (who often has insightful opinions) offers a similar but more fleshed out perspective.
As I also mentioned about a month ago, I’m optimistic that personal tutors for everyone will, in the very near future, be a reality that shows itself to be a force multiplier for good. Indeed, last week saw an amazing demonstration by Sal Kahn (of Kahn Academy) with his son and ChatGPT tutoring him in some trigonometry. Already, as Reid Hoffman suggested towards the end of this fascinating interview a great use of ChatGPT right now is as a sparing partner for ideas. Ask it for the pros and cons it sees against any idea you have. It’s a great pressure tester. As I always felt when I was directing films: you can always reject ideas you don’t like, but it’s hard to use ones you don’t hear. In the context of LLM’s that means: don’t fear hallucinations, instead listen for ideas which you can evaluate yourself and let the machines prompt more.
Thanks in part to this opinion, I’ve been accused of being a dogged optimist on occasion——though I’ve also been accused of being a pessimist for other perspectives. I prefer to think of myself as a realist, though don’t we all? Perhaps it’s just another instance of my buddy Todd’s view of the world: we all get more of what we’ve already got a lot of, applies to money, applies to friends, applies to perspectives.
By the way, if you’re inspired by Saskia’s explanations of how a basic neural net works in this chapter, and you’d like to delve a little deeper, Grant Sanderson, who runs the splendid Youtube channel Three Blue One Brown has a lovely introduction that puts some flesh on the bones Saskia has described. I should mention here that Grant is one of the greatest mathematics communicators of our time and if you want to get a feel for what great mathematical taste is like go check out one of his videos.
Well, that’s probably enough recommendations for one day. As always, I’d love to hear your thoughts about today’s chapter, and given it’s potentially didactic nature I’d specifically love to hear how the “lesson” landed. Did it feel part of the flow of the story? Did it make sense? Was there something I or Saskia missed? Are you looking forward to more?
Until next week, be kind to someone and keep an eye out for the ripples of joy you’ve seeded.
Cheerio
Rufus
PS. If you think of someone who might enjoy joining us on this experiment, please forward them this email. And if you are one of those someone’s and you’d like to read more
And now, without further ado, here’s chapter eighteen in which Saskia explains to Mica how a neural net works.
— 18 —
Mica’s Kitchen
Mica’s apartment had a pokey little kitchenette, though she preferred to think of it as efficient. Standing at the bench——there was only one——she could reach back to the fridge, right to the sink, and pirouette to the rickety old gas stove that she’d had such ambitions to use when she’d rented the place. On the bright side, regarding the gas stove, her lack of culinary commitment had been an environmental saving grace.
Mica topped, tailed, and halved a red onion. “So, when will your AI start making me dinner?” she asked Saskia.
“Mine just sorts the trash, remember?”
“How does it do that again?”
Saskia cheekily pinched a cube of the avocado that Mica liberated from its skin and gave Mica a saucy grin. “You promise not to run away, this time?”
“I’m cooking you dinner.”
“Fair enough,” Saskia conceded Mica’s good intentions. “First it’d need to watch how you work in the kitchen.”
“Does your machine have hands like mine?” Mica deftly flicked the peel of her red onion into the old porcelain sink that doubled as her own temporary trash receptacle, and turned her knife to dicing the core.
“Maybe not hands, but it’s going to need to be able to wield a knife.” Saskia paused in reflection. “You zeroed in on an excellent point, though.” She noted that even professional designers often fell into the trap of copying the way a human does something.
Mica furrowed her brow. “Seems like an obvious mistake.”
“You’d be surprised. But given that AI systems all use neural nets, which are essentially modeled on our minds——and they’ve been pretty successful——you can see where the prejudice comes from. Besides, sometimes just seeing the the job to be done, the machine finds an alternative path.”
“How long does it take to learn? Actually, what are neural nets? I’ve heard the term so many times.”
Saskia faltered. Not everyone that asked you to explain neural nets really wanted an explanation.
But Mica looked at her expectantly. “Come on, you must have a go-to metaphor you trot out. I can’t be the first to have asked you to explain what a neural net is.”
“No, you’re not,” Saskia agreed.
“They make it sound like the computer has a mind.”
Saskia decided to go for it. “That’s not a coincidence. As I said, the way our brains work was the inspiration for the architecture. It’s the idea that there are all these dials and knobs in our head——our neurons——and every time we experience something we firm up some of those neural connections, and let others slacken.”
“Okay ....”
“Alright. Let me give you the iconic example. Did you ever wonder how ATMs read your checks when you deposit them?”
Mica stopped chopping to listen more carefully. “I mean——”
“They started with single digits. Hand drawn on say a one hundred by one hundred grid. Think of each cell as a pixel. Either it’s black or white——we’re using a black pen on white paper.”
Mica nodded, to indicate she was following.
“So each of those ten thousand cells——that’s the one hundred by one hundred cells——is zero or one; black or white. Now those cells are connected to another thousand cells via a bunch of dials and knobs. It’s a bunch of linear combinations with some squishification functions——”
“Squishing functions?”
“Forget the squishification functions. Think: dials and knobs. Those are the neural connections we’re going to firm up or slacken. Now, that middle thousand cells are connected to ten output cells with more dials and knobs.”
“That’s a lot of dials and knobs to keep track of.”
“Yep.” Saskia nodded. “And they are the heart of the machine. But that’s it. That’s the machine. Problem is: it’s like a baby that’s just been born. It doesn’t know one from two. So we have to train it.”
“Train it?”
“You draw me a number.”
Mica obliged, drawing a two in the air between them.
Saskia gestured taking the number. “Now, I feed it into the machine, via the zeros or ones on the ten thousand cells that you were drawing it on. But the machine is initially set randomly and it spits out ten answers for our ten final cells. Oh, and the squishing here makes those answers add up to one. Think of them as probabilities.” Saskia could see that she was at risk of losing Mica. “Bear with me. You drew a two. So ideally, we’d like the machine to have spat out: zero, zero, one, zero, zero, zero, zero, etcetera. The first two zeros would be telling me that your number wasn’t zero and wasn’t one. The one would tell me that it was a two, and the rest of the zeros say it isn’t a three, four, five, up to nine.”
“Okay, it should have given me a bunch of zeros, with one one in the cell that represented the digit two.”
“Fantastic!” It normally took Saskia a couple of runs through to cement this idea in whoever she was explaining neural nets to. “The problem is our baby machine just spits out ten random numbers. Because it’s never seen any handwriting before and we set the initial dials randomly. But here’s the clever bit,” Saskia paused for dramatic effect, “we go back and look at our dials and knobs, and with each one we ask: what could we have done that would make the answer a little closer to what we actually wanted? And then we tweak that dial.”
“What do you mean closer to what we wanted?”
“Well, simplest version: take the difference between the value in each output cell and the corresponding ‘correct’ value, and add them all together. More generally, just imagine we have an error function that encodes how badly our machine guessed. Quantifies how badly it guessed. And we have you draw a thousand sample numbers and each time we run one through the machine we go back and adjust all the dials and knobs.”
“Whoa! All of them.”
“Every one. The miracle is that that’s all it takes. After a thousand runs through our baby will be pretty good at recognizing numbers. Give it a hundred thousand examples and it’ll do better than a human.”
“That’d take forever. You hand draw a hundred thousand numbers?”
“Remember those CAPTCHA things you used to have to fill out on the internet, to prove you were a human. You know: tick all the pictures that have cats in them.”
Mica nodded.
“The machine learning people used those as training data. Now, not only can machines recognize hand written numbers, they can identify cats, people, planes. You name it.”
Mica let out a whistle. “That’s pretty cool.”
“I think so.” Saskia had been unsure if she was more pleased by the simplicity that underpinned her field, or the fact that the woman in front of her appeared to share that appreciation. She realized, though, the value of the adage: “quit while you’re ahead” and she pointed to the painted blue octopus on the side of the bowl that Mica had emptied corn chips into. “Have you ever seen a real octopus? In the ocean?”
Surprised by the turn of topic, Mica turned the bowl around, checking Saskia’s eye-line. There were many threads in the tapestry that made up this conversation. A new color had just been added and it had nothing to do with AI or whether Saskia could really time travel. “No,” she answered Saskia’s question, “I haven’t.”
“Scuba-dived?”
Mica shook her head and squeezed a lime onto her salsa.
“You live by the ocean, but you’ve never tried scuba diving?”
Mica shrugged. She’d also never slipped in time despite living a lifetime within its regular flow, always on the precipice of a change of speed, but like a fish inside an aquarium, she was never able to access the surrounding room within which it stood. She picked up the thread Saskia was playing with: “I’ve never had the opportunity.”
“Well, we’ll have to fix that.” Saskia grinned at her. She described the joys of gliding through the water. Being delighted by an eel that poked its head out through a crack in the barnacle covered rocks, perhaps nervous that the receding tide might un-flood its temporary home. Surprised by a school of coral fish flitting through the kelp and seaweed, calling attention to the gentle flow of the flora’s drab structures with their splashes of color. “It’s like flying.”
“So you can fly now?” Mica winked at Saskia. “Yesterday it was time traveling.”
From her perch on the bar stool Saskia admitted a grin. “I can teach you to dive.”
“I imagine floating underwater is like slowing time down.” With the lottery story pushed aside, Mica was suddenly a lot more intrigued about the time travel Saskia professed possible.
Saskia bobbed her head from side to side, looking for a way to agree. “I mean, the way everything moves in water does look like it’s been slowed down.”
“Then suddenly you see your doppelganger?”
Saskia balked at the mention of her double. For a few minutes she’d been blissfuly lost in other worlds. The nagging thoughts about her twin sitting back at her house had been vanquished and she’d simply enjoyed sharing her hopes and dreams——what made her tick——with the gorgeous young woman in front of her.
“Doesn’t everyone look the same in a wetsuit and face mask?”
“Oh,” Saskia understood Mica’s meaning.
“Wait, how come you don’t arrive naked?” Mica asked abruptly.
