Chapter 22 — Descent on a Dish Towel
Rufus describes how mathematicians like to describe their work.
Followed by Chapter 22 —— Descent on a Dish Towel, in which Saskia elaborates on her earlier description of neural nets while she and Mica have breakfast.
Listen to full episode :
Hello Friends,
Today, Saskia is going to describe some geometry to Mica. Actually, Saskia is really fleshing out her description of the neural net she described a few chapters back, but the most fun thing about it is that in doing so, she will give a sense of how mathematicians think and talk in visual analogies.
The way mathematicians describe their work in metaphors is something that isn’t well hinted at in school expositions of mathematics … but it is one of the fundamental elements that makes mathematics fun. It’s the difference between presenting the instruction manual to someone and sharing what tickles you about the product. It has always confused me that English departments have the sense to read Winnie the Pooh before asking kids to learn to spell while Mathematics teachers typically do it the other way around, metaphorically speaking.
Let me give an example: Suppose I wanted to describe to you just how close the earth is to being a perfect sphere. One way might be to tell you that the diameter around the center is 0.3% larger than that at the poles. And that is true and accurate, but it doesn’t really capture what’s happening. At the risk of disparaging the occupation of some readers, I would call this the accountant’s answer. A true mathematician is more apt to impart some sense of what it would all mean. More interested in helping you understand if you’d notice the difference, for instance.
The mathematician might say something like: if the world were a billiard ball, there’d be a slight bulge at the center. It wouldn’t affect the roll of the ball, in fact the bulge is smaller than the individual imperfections around the ball, it’s just that they would all be collected at the equator. It would be about the equivalent of a super thin strip of paper, softened with water and pasted around the center, but softened so much that you could imagine it sort of dissolved in your hands and spread thinner and thinner as it goes out to the poles, until it’s so thin at the top and bottom that it literally isn’t there. But that probably misrepresents it, because you’re now thinking about the different texture that the paper and the billiard ball have. Better might be to imagine that the whole billiard ball has been covered by a layer of paper, and that the difference is that at the equator we’ve added a second sheet. No, even that is at risk of misrepresenting the roundness of the sphere since the difference between the diameter at the equator and that at the poles is more than four times how far Mount Everest differs from sea-level nearby, which is to say that in our paper analogy individual paper fibers would massively overshadow mountains. Put another way: if we just returned to the billiard ball and imagined it represented a world of ocean, then the invisible imperfections on the billiard balls’ surface would represent waves bigger than the tallest buildings in the world; and the earth is just that, but with all those imperfections stacked at the equator.
Anyway, we’re going to see a bunch of examples of mathematical metaphors over the coming chapters, so keep your eyes open for them.
One last thing to mention if you’re listening to this: today’s chapter has a simple diagram in it. Probably simple enough that you can imagine it from the description I give. But, if you want to see it, it’ll always be up on the episode website (which I’ll link to in the show notes). That said, there will be a few more images in the coming chapters that might be a little harder to visualize from a description. I will, of course, always include links to them in the episode notes, but perhaps it’s worth reminding you all that you can always sign up for the weekly newsletter version of this podcast on my website www.writtenbyrufus.com. That newsletter will arrive in your inbox every Monday and include the relevant images in a text-based version of this podcast.
Until next week, be kind to someone and keep an eye out for the ripples of joy you’ve seeded.
Cheerio
Rufus
PS. If you think of someone who might enjoy joining us on this experiment, please forward them this email. And if you are one of those someone’s and you’d like to read more
And now, without further ado, here’s chapter twenty two, in which Saskia elaborates on her earlier description of neural nets while she and Mica have breakfast.
— 22 —
Descent on a Dish Towel
“I’m not surprised your chest is sore. You scared the living shit out of me, and I’m pretty sure my reaction did the same to you.” Mica took Saskia by the cheeks and kissed her nose.
Saskia savored the touch of Mica’s lips; contact still charged with the thrill of unfamiliarity. But when Mica pulled away, Saskia noted that the pain in her chest still lingered. “I wouldn’t get that from a fright alone,” she countered Mica’s explanation.
Mica sucked on her lower lip and bobbed her head from side to side, acknowledging Saskia had a point. The point was not enough, though, to interrupt the task she’d started, and she emptied a can of wet food onto a saucer. “Psstpsps. Fish!” Her cat came skittering into the kitchen, purring loud enough for Saskia to hear as she returned to the other side of the kitchen counter with her coffee. “Tomato is okay with just dry food?” Mica asked.
“He’s partial to wet food too, but he’s fine with kibble,” Saskia said, eliding over the fact that her double was probably opening a sachet of tuna snack as she spoke. Saskia wasn’t sure why she had continued to omit the existence of her double. Fear of complicating the moment? Apprehension fueled by the way Mica had previously responded to seeing two of her? Or simply a hope that her double might disappear?
Mica pulled out two of the sea creature themed plates and touched the octopus. “You know all those dials and knobs you were telling me about yesterday? The ones in your AI machine. How do you know how much to turn each one?”
Saskia was delighted by the switch of topics.
“Wouldn’t some dials and knobs make a bigger difference than others?” Mica hadn’t just been listening, she’d been genuinely engaged.
“Absolutely,” Saskia confirmed. “And we tweak the knobs that have the biggest effect the most.”
“Right, but how do you know how much to turn each knob?”
“Well, if it was just one dial, you could turn it left or right, and you just pick the direction that improves your model. But you’re right, with two dials, you can go left or right with each dial. Instead, though, think of the second dial as up and down.”
Mica nodded.
“Now, imagine playing the dials off against one another. Do you have a pen and paper?”
“Hold on.” Mica ducked out of the room and retrieved a notebook from her desk. When she returned, she plucked a pen from the cracked old mug on the shelf above the sink.
Saskia was tickled by Mica’s curiosity. She took the pen from Mica and drew a pair of axes on the sheet of paper. She then added an arrow extending from the center:
“The vector coming out of the center——the arrow that isn’t one of the axes. It shows: the left–right dial is turned a long way to the right, call it clockwise, and the up–down dial turned down down just a little bit. Down represents counter-clockwise,” she clarified.
“Right, but how do we know how much to turn each dial?”
“Some simple calculus——the idea is simple.” Saskia scanned the kitchen. “Pass me that dish towel.”
Mica cocked her head at the odd request, but complied with its essence.
Saskia took Mica’s hands and directed her to hold the cloth above her axes. “Imagine the dish towel is our error function.”
“Our error function?”
“Remember we had a magical function that compared how well our model predicted your chicken scrawl number with what it was supposed to be. Something like the sum of how far the likelihood of our model’s predicted output was from the sequence of nine zeros and the one one of the correct answer.”
“What’s that got to do with the dish towel?”
“The height of the dish towel represents how big the error function is given every possible setting of the dials.”
“Wait, where are all the dials?”
“This model only has two dials,” Saskia clarified, indicating the paper beneath the dish towel. “The x dial and the y dial.”
“Ok.” Mica didn’t sound entirely confident, but Saskia persisted.
She pointed to the place where Mica’s towel hung lowest above the counter. “Obviously, the dial settings that gave us this result are the best possible settings.”
“Great. So we set the dials to that setting?”
“Well, it’s not quite that simple. Remember, we only calculated the error at the one setting of the dials we used to pass your hand drawn sample through our model. Say here.” Saskia pointed to a specific spot on the dish towel. “The thing is, it turns out, that it’s easy to calculate the direction the dish towel is sloping at this point——much easier than calculating what the whole dish towel looks like.” Saskia held her left hand parallel to the towel at the point her right hand was indicating. “Now, and this is important, we use this ‘tangent plane’ as an approximation of our error function. And when we do that, there’s an obvious direction we could step in on the paper below that would make this ‘tangent plane’——think the error function——go down the quickest.” She indicated the direction that her tangent plane hand pointed down the steepest.
“Wait,” Mica waved the dish towel aside, “what’s that got to do with the arrow you just drew?”
Saskia moved her tangent plane hand directly above the center of her axes. She aligned the steepest downward slope with the direction of her diagonal vector. “So, if we only get to take one step on the paper, then the error here”——she indicated the point on her tangent plane hand directly above the center of her axes——“goes down most if we go in the direction that my unit vector is pointing.”
“Okay.” Mica paused digesting Saskia’s explanation. “Wait, you just added the word ‘unit’ to your vector.”
Saskia beamed. “You’re sharp!”
Mica grinned back at her. There was something very satisfying about Saskia’s exclamation. It felt like she’d nailed a round of Charades with her first guess.
“Alright! This is where we get to the bang for your buck. The why we don’t just turn every dial the same amount.” Saskia indicated the direction that was equal parts down and across on her drawing and then the direction that that would correspond to on her plane. “Turning the dials the same amount would be like going a bit across the tangent plane. Down, sure, but not straight down. The unit just means the total length of the step we take in that direction is one unit——think of it as: in total we’re only going to turn all the dials so much. It doesn’t actually have to be one unit. In fact, typically it’s a pretty small step, because you don’t want to over compensate based on one piece of data——your one scrawled sample digit. The important thing is, you don’t want to change your model too much each time.”
“And where was the calculus again?”
“Finding the tangent plane.”
“Hmmh,” Mica sighed, satisfied.
Not everyone Saskia had given this explanation to in the past cared. More often than not, they didn’t make a real effort to follow what Saskia was trying to make clear. She was flattered by Mica’s interest.
But then, behind Mica, Saskia caught a glimpse of the clock on the stove. She suddenly felt her work day tugging at her. She had called in sick yesterday, something she never did, and she wondered if there was time to stop back at home before heading in to the sorting center. That question brought her double back to the front of her mind. She’d been self-indulgent staying the night with Mica——what if she had to start divvying up her life? It was an absurd proposition, but it was a complicating prospect not completely detached from her new reality.
“Hey there,” Mica prompted Saskia. “What’cha thinking? Something big?”
Saskia shook her head dismissively. “No. Just the day ahead. I’ve got to get to work.”
“Me too.”
“And I’m wondering if I can make it home first.”
Fish rubbed himself against Mica’s leg and drifted out of the kitchen. “Tomato might like to see you.” Mica grinned.
But Saskia was still in her own world. “The other day, at Cleo’s. The restaurant. When you saw two of me in the restroom ...” Saskia wasn’t sure how to finish her question, and it hung in the air long enough that Mica corrected the foundation upon which it stood.
“Outside the restroom, and I only saw one of you there. Your twin——”
“I don’t——”
“I know. Which freaks me out. Can you explain how you were outside the restroom and sitting at our table?”
Saskia couldn’t. Worse though, she was preoccupied with reconciling what her double had told her about her experience inside the restroom. Mica had clearly not experienced that any more than Saskia had. Had the time travel addled her double’s mind? That seemed unlikely. At least given the specificity of her story.
“Have you seen another you?” Mica asked point blank.
“No,” Saskia lied. It was too complicated to face. “But I’m glad you gave me a second chance.”
Mica rounded the kitchen counter and stepped behind Saskia, who was still sitting on the barstool. “Maybe I was wrong. Maybe this superpower will get you everything you really want.” Mica gave Saskia a kiss on the back of Saskia’s silky black hair. Then, to Saskia’s surprise and joy, Mica slipped her hand up under the lip of Saskia’s borrowed T-shirt. She gently circled Saskia’s abs with the tips of three fingers, her middle finger slipping in and out of Saskia’s belly button.
Saskia stroked Mica’s forearm twice before her caress floated past Mica’s elbow and she softly pulled Mica closer.
For a moment, Mica let herself be pulled in. She even wrapped her other arm around Saskia, engulfing her from behind. But then she unwound herself. Her index finger grazed the underside of Saskia’s breast as she retracted her arms and straightened up.
Saskia felt a warm a little electric shock ripple through her body.
“But I’m not letting you off that easy,” Mica broke the moment. “I still want to know who that other you was.”
“So do I.” Saskia’s voice carried less defiance than wistful yearning. “So do I,” she repeated.
“You really don’t know who that other you was?”
Saskia shook her head. Sure, she had met the other her, but she really didn’t know who she was. What could she say? Was there something she could actually disclose?
Mica leaned in again and snuffled Saskia’s hair. “Your new ability has some weird consequences.” She then stood upright and returned to the kitchen galley, “You want some toast to fuel an investigation?”
“Into the other me’s?”
“Sure, but there was also your super strength back at the race track. When you tried pulling me back in time.”
Saskia felt a pang of guilt. “I really hurt you?”
Mica shrugged. “You didn’t mean to.”
