“I walked back in the room, and she’s like “The site’s down.””

Note: Some language has been edited out of the transcript that has not been edited out of the audio. 

“Has there ever been a time where you were able to stay calm in a high stress situation? What happened and how’d you respond? One of our team members had their car spin out on black ice, but they avoided an accident because they stayed calm and didn’t try to overcorrect the car.” Damn. I’m really glad that that had – that worked out for you, team member. Stay calm in a high stress situation. Again, at work – so I’m a software developer. And I was doing a deployment, this was maybe a month or two ago. And it was low risk, it was just purely a front-end change, wasn’t supposed to change anything about the data model or anything, and I deployed. It takes us like five to eight minutes to deploy. And it was fine. It completed, and there was a small data change that I had to make. I made it. I asked my teammate to verify that it looked good, ’cause I was collaborating with her on that deployment.

I walked out of the room to get coffee, and I was back in, you know, less than two minutes. And I walked back in the room, and she’s like “The site’s down.” I was like “What?” And I just looked at her screen and it was just a white screen. I was like “F***.” I was like “Oh man,” and then… So then it’s basically firefighting, like “What did I do? How can I get this back up?” And thankfully, we did have sort of a rollback strategy that I utilized within five minutes, if not within three minutes. So the downtime was minimal, and there were no complaints from customers about it. But yeah, I was – I just kinda get in the zone, kinda just blinders. Like I just see the screen, and the environment just melts away. And then after that. Like, after I got the site back up, I was like just a huge sigh of relief, like my heart was pounding. I was like, “I gotta take a walk, just to calm down, clear my head.” And as it turns out, I don’t think it was my fault per se, that the site went down. I deployed during the day which we try to avoid doing, we try to deploy during non-peak hours. But turns out that there was an issue with sort of like caching. I sorta hand-wave caching just because that can be a problem for like a whole ton of things, just like holding on to stale data. But anyways, yeah, as far as trying to stay calm in a high-pressure situation for me, the thing that jumps to mind is dealing with prod support issues at work.

Recent Stories

Archive