Kobuk the Destroyer

I live in Northern California and when it comes to camping in the backcountry up here,

bears are always something you plan for.

“The best way to avoid bears is to not attract them in the first place.”

And that means putting your food in any fragrant items into a bear canister. But how do we know that bear canisters actually work? For decades, product testers would assess the bear resistance of a canister, by dropping a weight on it from a set height, or by trying to puncture it with a spike that simulated a bear's tooth.

But they kept running up against a problem. These tests weren't close enough to reality. They weren't accounting for a bear's creativity and resourcefulness. So then, in the early 2000s, they hit upon an idea of testing the canisters by using something slightly more realistic. Bears.

A manufacturer would say, I have this new design for a bear resistant canister that I want campers to use.

They bring it to this bear sanctuary. They fill it up with bear treats, like fish and honey and all sorts of goodies. And they just throw it to a grizzly bear. This is journalist Alex Davies. He says that these bears will bite, chew, claw, and jump up and down on these products.

If a product gets through one hour of contact with a bear and isn't opened, it gets certified as bear resistant. Some bears are more successful than others. And the most famous of these bears, his name is Kobak. Kobak became Kobak the destroyer because he was the best.

As a cub, Kobak was dubbed a problem bear for getting into human food.

He was removed from the Alaska wilderness and brought to a sanctuary, where he and his fellow problem bears were put to work. And there, instead of becoming problem bears, they became problem solving bears. Because the work of the bear sanctuary wasn't just to keep bear safe, it was actually to have those bears test out supposedly bear resistant products.

Kobak above all other bears became known for his ability to break into almost anything. If your products survived Kobak, he might even get written up in a local paper. Kobak was more a destroyer of ego and of product design than anything. He was just really good at it. Alex Davies is the author of the new book, Kobak, the destroyer,

and other tales from the wild and unseen world of test engineering. His book takes us through the long history of how things are tested, and all the behind-the-scenes work that goes into making sure things work. And I constantly felt like an emotional pendulum, where one day I'd feel like everything is crap.

Nothing is tested properly. They're all cunning corners. They're just trying to make a profit. And then the next day, I'd read a different example and go, "Oh my God, these engineers are heroes!" The work they put into figuring out to make this thing safe for my kid

is incredible.

“And obviously, like all things, the truth is probably somewhere in the middle.”

But I do think that for the most part, you can come away with a sense of awe that there is all of this work that goes into not making something, but making it work. I think when people, I certainly, when I picture a product testing,

I don't picture a bear. I picture like a man in a white lab coat, maybe with eye protection on, and he's dropping a bowling ball on a hard hat to see if it indoors such treatment. In your book, I think you roughly classify this kind of testing as limit testing. Could you describe limit testing and its origins?

Limit testing is probably the most basic kind of testing. Your examining a product usually for its strength and for its durability. So it's everything from how far can you bend the wings of a 777 bowing airplane before they snap

“to how many times can you hit the spacebar on your computer before it breaks?”

The understanding of a product's limits has been around as long as people have been building stuff. Everyone knew that if you're building a new branch, it has to be really strong. But the actual work of limit testing of trying to figure out in a scientific way with a rigorous method of evaluation, I traced that back to the foundation of the underwriter's laboratories,

which came about in the very beginning of the 20th century. And that was started by a guy named W.H. Merrill.

This was an age when entire cities were still burning down on a fairly regular basis. And also at the same time, a lot of electrical appliances were coming into the home. Before you had an electric iron, an iron was a chunk of iron. But you got really hot and you pressed it on your clothes. That's why it's called an iron.

And now all of a sudden electricity was coming into the home and Merrill saw this business opportunity to say, "Hey, what if I actually tested these things to make sure they're not liable to set your entire home or your entire city on fire?" And he created a whole business out of this.

Underwriters' labs has always been about testing a product against the manufacturer's claims.

And so within 15 years or so, they were testing millions of products a year. And the products that passed got the underwriters' lab seal of approval physically stamped onto them. And that became a real mark of consumer trust. At a time when electricity coming into the home, all new kinds of products coming into the home, created a new kind of risk. And test engineering rose to meet the challenge of that risk.

Yeah, and this mark of underwriters' laboratory that it really mattered to consumers and into retailers. It became a huge deal.

“There were entire cities, I think Los Angeles was one of them.”

And store chains like JC Penny, which existed at the time that said, "You can't buy an electric appliance if it's not UL approved. It has to have that stamp on there." I mean, to do this type of testing, you need a lot of standardization, right? How did underwriters' labs go about making sure everything was tested in a way that was considered fair and trustworthy? They kept incredibly detailed handwritten reports at the time.

I've just exactly what it did when they were testing safe, for example, which the big thing with safe at the time was one so that they could keep burglars out out. But they were also really important for resisting fire. Because if you're building cod fire on all of your business papers when you're safe, it was hugely important that all of your business records didn't burn up.

So a lot of the testing they did on safe was fire resistance. It wasn't just, we're going to heat this thing to 1500 degrees Fahrenheit and see what happens. They had a very specific heat over time gradient. And it was this chart. And it said after three minutes, it should be 90 degrees.

After four and a half minutes, it should be 115 degrees.

“And that's how meticulous they were in their testing.”

Yeah, it brings up this interesting dilemma. I mean, I just wonder how manufacturers and testers decide how safe is safe enough, like how reliable it's reliable enough. How does this line get determined for each product?

If you look at different products, it's always a sliding scale.

And airplane has to be more reliable than your laptop's keyboard, right? A medicine bottle that could get in the reach of a two year old has to be more child-resistant than a jar of jelly. Yeah. And it's always this kind of negotiation between things,

asking, what's the product, what's the user, what's the cost? Yeah. And what's the risk? Right. Right. The designing of these tests is, you know, they can get pretty ingenious.

And there's ways in which you're trying to sort of balance this idea of repeatability versus real world uses. I wanted to talk about the bird ingestion test in this regard. Yeah. When it comes to aviation testing, can you describe what is going on there and what

they're testing for? Sure.

“So if you remember the miracle on the Hudson story or Captain Sully had a bunch of”

geese, take out both engines of his planes and he miraculously, although real pilots will say, well, he just did what, anyway, that's going down a rabbit hole. Captain Sully had a bunch of geese, take out both engines on his plane, and he very impressively safely landed on the Hudson.

And that's a bird strike. Bird strikes are super common in aviation.

The first bird strike was recorded by the Wright brothers.

And, you know, a seagull got tangled in the cables of their plane. And so what we do about it is that we test jet engines against bird ingestion, which is a very sanitized version of, we have a cannon,

And they're very specific rules around this. They're different versions of this test for different jet engines. And it's all very carefully calibrated.

But basically, the idea is, how do you know whether or not a jet engine

can either keep running or shut down safely? I mean, not catch on fire or explode.

“If a bird flies into it, the best way to do that is to turn it on”

and have a cannon that shoots chickens in. And it shoots them in at around 200 miles an hour, because that's roughly the speed of a jet at takeoff, because takeoff and landing are the most common times for birds, strikes because the rest of the time they planes fly higher than birds do.

So for a test like this, how do they strike this balance between making sure that the test is conducted? Similarly enough that it's repeatable,

but also that it replicates real-world conditions

as closely as possible. Yeah, you want a test to be to recreate a similarly as possible the real-world conditions that the plane will face, right? Bird, jet engine, cannon. The cannon is there to recreate the speed of the plane,

the bird is there to recreate the real-life bird who's going to go into it. But you also want your test to be very standardized and very repeatable so that, say, you're making a new jet engine and you had this iteration and it did okay, but you've tweaked this thing and you're going to test this new version of it.

You want to know that the test results you're getting are exactly the same. The reason we use chickens for this test is because they're highly available, they're pretty similar and they almost exactly meet the FAA's requirement that you test the jet engine against a four-pound bird. But the thing is that chickens don't fly.

chickens don't end up in jet engines. And the FAA at one point did like a 20-year study of all the different animals that have hit been hit by planes in various ways. And there are more plane on bearded seal encounters than plane on chicken encounters. That's one seal and zero chickens.

But again, chickens fit the test, chickens are easily available. And basically, this is something where we shrug our shoulders a little bit and go, well, this is pretty safe. And so you have the repeatability or the workability of the test

and the similarity to real world conditions are always intention.

“And just like picking a standard, it's this thing where you have to go,”

well, this makes it a little more doable even if it takes away from the real world just a little bit. And it's finding that balance. So another element of test engineering is something that you call foolproofing, which is essentially making sure that a product not only stands up to what it's supposed to do, but also if someone were to use it wrong, it would still be somewhat safe.

So how do you go about trying to foolproof a product? So the first thing you have to do to foolproof a product is think really carefully about the user. And this is a thing that I don't want to disparage engineers. But what I'll say is, I'll based this on a real fact, so that I talk to a guy named Ken Dontramal who teaches product safety engineering.

And he tells me his students don't naturally think about this stuff. Engineers are often geared toward making a product that works, not making a product that works for a person. So to actually foolproof something as an engineer,

“you have to think about what is someone going to do with this thing.”

Despite the fact that I tell them not to, despite the fact that it's nowhere in the instructions, despite the fact that it's an obviously stupid thing to do. So for example, you make a screwdriver. A screwdriver is made to turn screws. People also use them to open paint cans.

Everyone who's open to paint can as probably use the screwdriver to do it, because that's how you open a paint can. It is the tool for the job, yes. It is the tool for the job, so it has to be made so that even when you're leaving something that might be tough, it doesn't shatter and throw plastic or metal shards into your eyes.

A screwdriver has to be made to open paint cans effectively. But you can imagine someone sticking it in their ear to clean out earwax. And puncturing their ear drum, but you don't have to make it so that it won't puncture your ear drum

there's no real pattern of people doing that because people are at least that intelligent.

“And so it's always foolproofing is, again, finding the balance between”

what makes something safe and what makes something usable. I go camping a lot. I hate our bare canister because I find it almost impossible to open and get hurt, it drives me crazy. But at least I can get it open and I'm pretty confident that a bear

here in the Sierra Nevada mountains can't open that. So, okay, that's a fine trade-off. It's interesting when it comes to foolproofing when you really can't design the hazard out of the product or maybe like you can, but it means that the product becomes very difficult and annoying to use

or becomes very expensive. Yeah, so one great example of that is sauce stop, which is a table saw safety device created by a guy named Steve Gas, who was an amateur woodworker and by day a patent attorney,

“which, if you go really deep into the story, becomes quite relevant.”

But he was using his table saw and he had the very correct thought, "I'll probably eventually hurt myself using this thing." And is there a way to make a table saw safe? Because there are these things you can get that are kind of guards over them that maybe stop your fingers putting in,

but they also make the saw somewhat less usable. Yeah. And there's a very clear pattern of people pulling them off and throwing them away. I don't know, this thing just gets in my way. Yeah, he created a thing called sauce stop where he ran an electric circuit

through the blade with a micro processor. And he put what looks like essentially an aluminum break below the blade. And the idea is if your finger touches it, touches the blade, you will complete that electric circuit because we conduct electricity. And the moment that circuit is completed,

it's very much like putting a stick in the spokes of your bicycle. It just stops the thing from running. And the result is that he created a way to stop a table saw in a fraction of a second. Before you could even actually do damage to your finger. You might need a bandaid.

But you won't need stitches and you won't need someone to sew your finger back onto your hand. Yeah. It was an incredible invention and the consumer product safety commission. Like a lot of it, him said, this is amazing. This should be attached to every table saw ever made.

The problem is it was expensive.

It added hundreds of dollars of cost to the price of a table saw, which otherwise is a pretty affordable product. And all of the table saw manufacturers go, we don't want to do that. One, because this guy owns the patent.

And we'll have to pay him money for every table saw. But two, the real big reason is that it's going to make it to expensive for our consumers. And they won't want to buy these things.

“I mean, you could still go buy one of those things, right?”

Yes. Yeah, what Steve Gas ended up doing was saying fine, I'll just sell a line of table saws called Saw Stop that have this thing built into that. Yeah. And they're more expensive.

They're safe. And actually, some of his best clients are the schools that still have shop classes. Totally. Which is, again, that risk calculus of if you're just one person, you're going, do I really want to pay a couple hundred extra bucks?

Can't I just be careful versus the school that's like, these are teenagers. And we don't want to get sued by their parents. So they're going to pay the extra money. So it's, again, it's that sliding scale.

And what I love is I talked about this with the product safety engineer Ken Dontramal. And he goes, what I buy one. Now it's too expensive.

You know, we're basically so far talking about products.

You know, being used as intended functioning, but there's a whole class of testing. It has to do with the fact that products fail. Cars crash. Pay planes fall out of this guy. You term this disintegration testing.

What do I, what that is and how they test for it? Desintegration testing is based on the acceptance of the fact that things fail.

Dams burst.

“And it's all designed with the idea of how can I make something fail safely.”

And car crash testing is actually one of the most important places for this.

So disintegration testing for cars started with a guy named Hugh Dehaven. He wanted to fight as a pilot in the first World War. He had bad hearing or something, so he got rejected by the US. He ended up flying with the Canadian Royal Air Force. And during training, his plane collided with another plane in midair.

There were four people between these two planes. And he was the only one who survived. I mean, he very nearly died. He burst organs, he broke his legs. The doctors just assumed he would die, so they didn't bother to set his legs. They were broken, but he lived.

And he spent a lot of time in his convalescence. Just wondering, well, why did I survive?

“What happened in my plane that didn't happen in other planes?”

And he started researching this after he got out of the hospital. Actually, just started thinking about it. And he would do things like drop eggs onto foam mats to see what broke them. And he studied suicidal jumps by people who had survived. And he'd look at the conditions of the people who jumped from a certain height that should have killed them.

And why they survived effectively.

And so he came up with this term that he called the second collision.

Within a car or an airplane. And he said specifically within vehicles, it's not the first collision. It's not the plane hitting the ground or the car hitting the other car that hurts the occupant. It's the occupant hitting the inside of the vehicle. Right.

As they're propelled forward by Newtonian physics.

“And he said, that's what we actually need to pay attention to.”

And looking at things like hit the eggs that didn't break when he dropped them onto certain materials. And the people who jumped from great heights but survived because they landed on a freshly killed lawn. Or they hit a wooden structure before hitting the ground that kind of slowed their fall. He said, basically we need to cushion these people. And we need to restrain them.

And that's where we get ultimately airbags and seat belts, which are the things that.

They had cars less incredibly dangerous. But he also said, well, if you think about car design in the 1950s when his work was really taking off. He said, how about fewer metal spikes. Inside the car. And how about even before the age of.

Airbags, how about a steering wheel that will compress when you hit it rather than. In pale, yeah, yeah, which was actually pretty common and very gruesome. And but it took this whole different way of thinking because. Really up until his work and for years and years after it until the federal government managed to. Bludge in car companies into obey essence.

Car companies were saying, it's the driver. All we need is driver education. We just need to educate drivers about how to be better drivers. And it's those nuts behind the wheel who calls car accidents. It's basically the I told you so argument.

Yeah. Yeah. And it comes up a lot in foolproofing to. Found a quote from an underwriters lab engineer who said, why should I bother to design a toaster that doesn't get hot. When the same result can be achieved by not touching it.

And it's like because. It's now people are going to touch hot toasters and not because they're idiots or because they're trying to hurt themselves. Because it's a moment of inattention because you're holding your baby in one arm and you're trying to move something like. Things happen failure happens. Yeah.

And you have to design against it. You know, we talked about infrastructure a lot on this show and I was really interested in the aspect of test engineering when it comes to making things. Safe to fail. Like it's one thing to be testing a product where you're producing millions of copies of the thing. You can, you know, keep testing it and taking it through iterations where it gets better over time.

But when it comes to infrastructure, I mean, you get one chance to build a bridge or a dam like how do you design it infrastructure with failure in mind. The baseline common place way to design infrastructure is to look at the conditions, which are usually historic climate conditions. Let's say we're talking about a dam. You say, okay, wind speeds hit this, you know, storms hit this level, temperatures hit this. So we'll build a big thing out of concrete that's strong enough to exceed all of those conditions.

Right.

The problem is that with climate change, especially historic this idea of what's called a design storm, which is the storm you have in mind when you're designing a piece of infrastructure.

The design storm isn't a very helpful gauge anymore because weather's getting so much wilder saying, you know, more wildfires, more intense hurricanes, faster winds, etc. And so there's a school of thought now in civil engineering that says, well, let's stop just trying to make things stronger and stronger and piling more and more and more concrete. Let's instead design things with failure in mind.

“And that doesn't necessarily mean that you just, you're just resigned to the fact that infrastructure will crumble, right?”

Like you talk about an example of infrastructure in the Netherlands that was designed with failure in mind. Um, could you say more about that project? Yeah. It was a project called room for the river where they found that over time their rivers kept flooding and they kept flooding more and more and more and the damage was becoming worse and worse. It was flooding cities.

It was destroying crops and they were building levees, taller and taller and taller and they were building dikes and all these ways to try and rain in their rivers. And eventually they said, this is not working anymore.

Ultimately, they decided to look at it from a completely different perspective, accepting that their human made designs to stop the rivers from flooding would eventually fail.

And instead, they created room for the river. It's actually several rivers that I can't pronounce. Fair enough. They said, we're going to build these big fields around them farmers who are right next to these rivers, we're going to buy them out and we're going to help them by land elsewhere. And the farmers who refuse to stay, we're going to create compensation programs for them.

They said, we know this will flood.

“How do we accept these and how do we make the costs not catastrophic rather than how do we stop this from happening in the first place?”

And it's classic disintegration testing. You accept that the bad thing will happen and you look for ways to mitigate the damage. Right. It sounds like a much more complicated approach because it's not just like you're building something and you just make it stronger and better. You have to make all sorts of concessions and make sure everyone's on board. It's more complicated because if you're just saying, we're going to stop the river from overflowing.

Everyone says, great, build a big lefty-tolerable. Yeah, it's a good wall. No one has a problem with that. That makes sense. It doesn't affect anyone. But if you're saying, actually, we're going to let these things run wild. You have to think about who is that going to harm and you have to bring more stakeholders into the conversation.

“You have to think like, is it okay that this field floods, but this one doesn't?”

Isn't okay that this series of buildings gets water in their basement, but this series of buildings doesn't? And so, on a societal level, it's much harder, but it works a lot better if you can get it right. More with Alex Davies after the break. We're back with Alex Davies. Something you're talking about in your book is the idea that, no matter how much you test something in a lab,

the real world will ultimately throw way more things at it than any lab could possibly devise.

And you use this example of the Golden Gate Bridge, which I love, where a new design element was added to the bridge. But it ended up having a major unintended effect, which is that it turned the bridge into a gigantic musical instrument. Can you talk about what happened there? So, in 2014, the owners of the Golden Gate Bridge decided to add nets under the bridge to mitigate or prevent suicidal jumps from the bridge. They knew that adding nets would increase the wind load on the bridge, and they wanted to make sure the bridge would maintain its structural integrity under higher winds.

And what they ended up doing was taking the slats that are along the railing of the pedestrian walkway on the bridge, which were originally parallel to the bridge, like the slats on a picket fence, and they took them, and they turned them 90 degrees. And so that way, they're not catching the wind, and so we're balancing it out. And so they said, "Of course, we need to test this properly, so they took a scale model into a wind tunnel."

No problem. So they went, and they turned all of the slats on the thing.

“And then almost immediately, you get a windy day, because it's San Francisco, and you get this, "Oh, oh, oh, oh, oh!"”

So nice, the people on the other side of the San Francisco Bay are hearing, and they're going, "What on earth is this eerie humming noise?"

It's like the noise you get in a horror movie, the first time things start to go a little bit wrong.

And what they didn't realize, they didn't think about the acoustics of the bridge. They thought about the wind load on the bridge. And this is what I call dynamic testing. You're effectively testing how does something affect its environment, and how is something affected by the environment in which you put it?

“And so what they ended up doing was, so they went back a little bit to the drawing board, they said, "Well, we still want the nets here. We still think turning the slats perpendicular is the best way to reduce wind load, and even that out."”

But what they ended up doing was, they put these little rubber-lined clips, a U-shaped clip on every single flat to basically stop it vibrating.

Because what was happening was the wind was coming through, and like a giant harmonica, all of these things were vibrating a little bit, and that's what was making the humming noise. This time they actually tested the acoustics of it, and they found that in almost all wind conditions, the bridge was silent, in some very particular gusty conditions, it would make a little bit of noise. But they said, "This is acceptable. They put all the clips on, and they painted them all, the iconic orange of the Golden Gate Bridge."

One aspect of testing that I found really intriguing in your book was this idea of intention, basically how to test for a product, when the user is not just using the product wrong, but is using it maliciously. And I think a great example of this is the air tag, which is a little tracing tag that's meant to help find lost personal items. And if you thought about testing an air tag, the first things you might think of are, like, "Okay, so if I drop it, it doesn't still work. How long does the battery last? If I move it, 30 feet, does it still connect? How about 100 yards?"

But what I don't really think about is how you test it, keeping in mind that it might be used maliciously. Um, could you talk about that? Almost. I think maybe possibly even before they were on the market. When people knew that Apple was making a thing called an air tag that you could attach to your keys or even attach to your dog or put in your kids backpack to keep track of where things are, people immediately said stalkers are going to use these two track people. Because they're really small, they don't make any noise. It's really easy to slip into something.

And eventually Apple got hit with a class action lawsuit by people who, a lot of, it was almost exclusively women who were being tracked by abusive, either partners or former partners.

And this was a genuine problem for a lot of people. And Apple ultimately went back and thought a little bit more about it.

They did put some measures in from the beginning to give them credit. There was a thing that after a couple of days it would say, "Hey, there is an air tag that's matching your location almost all the time, but it's not on your account." Should you, should you check that? Like, and I get those alerts sometimes, even like, I travel a lot with my husband and sometimes it says, "Hey, this case of air pods has been going everywhere you're going." I'm like, "Yeah, because we're on vacation, we're going to all the same places."

But it's true. It's the kind of conventional testing we've been talking about, tends to fall short when you're talking about, not just is someone going to use this in a stupid way, but is someone going to use this in a malicious way.

“It's easy to take test engineering for granted, I think. But so many things we interact with have gone through rigorous testing and we have benefited tremendously from it.”

The vaccines we get, the planes we fly in, the bridges we drive across, the refrigerators that we have at home. In your book, you also write that we're in an era where it feels like test engineering is maybe more important than ever, especially now that we're kind of in a real world scenario where it feels like we're being tested all the time. And I'm thinking about algorithms and AI technologies that are being tested in the real world on us, learning from us in ways that are faster than we can possibly understand.

I think knowing how these tests are done is important because these tests underpin how basically everything you ever buy is made. And I think it's becoming more important because testing is breaking out of the laboratory.

“And this sample here is self-driving cars, right? If you've seen a self-driving car, what basically means you've been to San Francisco or Austin or the artist different places, if you've seen a self-driving car go by, you are part of that test.”

This is, is this thing, isn't it? I've up onto the sidewalk and mow you doubt. Fortunately, like they don't put these things out in the real world until they're very confident about that.

But testing is happening in all of these different ways around you, right? A new Facebook algorithm is being tested in real time on real people.

“But I think there's a lot to be said for being more knowledgeable about how these engineers work, how these tests are run.”

You can start to question what are the trade-offs between safety and reliability and progress and profit. You can start thinking, how close is this to the real world, what trade-offs, what did they sacrifice to make this a workable test?

I can't test the hidden backstory of everything in your life. It lets you see that backstory, or lets you imagine it with a lot more clarity.

“Well Alex Davies, thank you so much for talking with me. I had such a fun time and I really enjoyed your book.”

Thank you so much for having me.

Ninety-nine percent in visible was produced this week by Lashman Dawn, additional production by Jacob Medina Gleason, mixed by Martin Gonzalez, music by Swanry Al. Kathy 2 is our executive producer Kurt Colestade is the digital director, Delaney Hall is our senior editor, resident includes Chris Berube, Jason Delion, Emmett Fitzgerald, Christopher Johnson, Vivienne Leigh, Kelly Prone, Joe Rosenberg, talent and range stradly, and me Roman Morris. The 99% of visible logo was created by Stefan Lawrence. We are part of the serious XMP podcast family now headquarters 6 blocks north in the Pandora building, in beautiful, uptown, Oakland, California.

You can find this on all the usual social media sites as well as our own discord server. There's a link to that as well as every past episode of 99PI at 99PI.org.