Jul
11
2009
Some encounters are almost too strange to believe. That doesn’t make them any less real.
I was walking down the street in San Francisco at lunch time Friday afternoon. As I came up to a busy street corner I saw a paper grocery bag sitting on a bench with no one around it. I walked up to the bag and peeked in to find three external hard drives, one Maxtor and two brands I didn’t recognize. The drives looked like they were either well used or the product of a dumpster dive. I knocked on the door of the one business nearby, but no one answered. After a few minutes someone came out who worked in the building; he said there’d been a break-in recently but that he didn’t know anything about the drives. I tried to call Rich for advice, but he was busy so I decided I’d finish my walk to lunch and think on the situation for a little while.
One burrito later, I walked up on the scene again. This time a homeless man in dirty, ripped slacks was surveying the bag of hard drives. He looked around much like I had done thirty minutes earlier, then scuttled up to the bag and pulled out one of the external hard drives. After sniffing it for a second, he licked one side of the drive and put it back in the bag. He then ran over to a parking meter and licked it, licked the taillights on both sides of an SUV and vanished from my sight behind the car.
I lost any interest in the hard drives at that point. That takes mom’s caution of “you don’t know where that’s been” to a whole new level.
Saliva incident aside, what would you do if you found a bag of hard drives in a park or public place? Calling 911 didn’t seem appropriate, though there is a slim possiblity of explosives. Taking the drives home and performing some forensics research on them crossed my mind; I have the technology if not much skill in the area. I tried to turn them in to the business, but there was no one there. I guess the gentlemen with the inquisitive taste buds saved me from a moral dilema.
What would you have done?
Nov
10
2008
Congratulations to Jason, the winner of the free pass to CSI. Here’s his story about how a minor change to a script almost caused a major disaster. I have my own war story about scripts I’ll share later this week. Here’s a hint: Always make sure you’re in the proper directory when running your scripts.
This happened when I was first learning to admin UNIX boxes. Another
SysAdmin and I were working on a shell script to lowercase the file
names of 30-40 million image files. They were on an NFS mount that was
used by several servers. These images were part of detail listings of a
relatively busy web site and we were right in the middle of the day.
Now that the background of the mess are fully explained, the story
gets going. We went through several revisions and were testing against
a directory on a desktop system. Nothing destructive happened during
testing and we were getting fairly comfortable with the “safety” of the
script.
We finally thought we had a working script, so we moved it to the
prod server. Then we noticed a “minor” change that needed to be made on
it. We made the change then decided that since this was a such a small,
little tweak we could run it on the live NFS mount without any further
testing. Fire in the hole!
The script took off and we watched it run. All was well. Then my
phone rang from the NOC. A panicked operator was on the phone saying,
“Hey what’s happening with listing images from xyz.com? They are all
coming up as 404s!” I killed the script while thinking some thing like
“oh crap, oh crap, oh crap!” Sure enough the script had wiped out about
50% of the images. Amazing how fast a shell script can delete when it
goes haywire.
We pointed the web servers to a backup copy of the images, then
started to recover to the production mount. The backup was a couple
days old, so our image processing guys had to re-upload the missing
work. I was lucky that the online backup was there. I had taken it for
reasons unrelated to this event. The next day I got to explain to the
CIO what had happened.
The moral of the story was backup first and test your script until
it is golden before going live. Then test it again and again and again.
Make sure you are doing at the proper time, then go to production. We
didn’t have change control, so I’d add get all the approvals now too.
Cover your butt.
It was a good lesson. I’ve never done anything like that again in the last 7 years.