Courting Disaster

Iowa Caucus results recently were delayed due to the meltdown of an untested app. Should we be surprised? I think not. Truth to tell, my own career as a programmer can best be summed up as a series of episodes courting disaster. “Whaaat?” I hear you say. Yep.  Read on, if you dare.

 

 

WARNING: The following stories are true.  But they are filled with acronyms. If you don’t like acronyms, stop right here. Go outside. Take a hike. There.  You done been warned.

 

********

 

My first assignment after I got out of Systems Engineer (SE) training with Electronic Data Systems (EDS) was with the California Student Aid Commission (CSAC) in Sacramento. They’d bought an off-the-shelf solution from another vendor to upgrade their aging mainframe student loan tracking system, called FAPS (Financial Aid Processing System).  The new one didn’t work.  Our mission was to implement this new code immediately in production and then try to fix it on the fly.  It took us two years, but we finally did it – though not before the head of CSAC got sacked and a lot of EDSers (like me) spent a lot of sleepless nights up babysitting a batch cycle that didn’t fit inside the available 12-hour window, frequently crashing, and eliciting muttered curses from the programmer on-call .

After 5 years I moved on to the long-distance telecom business which was then in the throes of consolidation.  This was just at the beginning of the cellular revolution that made long-distance land-line telephone service obsolete.  The company was called MCI, short for either Microwave Cellular International, or Many Confused Individuals, depending on who you talked to.  Later it became known as WorldCom. The former CEO of WorldCom, Bernie Ebbers, died recently after a long stretch in prison for his role in bogus accounting tactics that propped up WorldCom’s stock price while defrauding investors and employees (like me) who had company pensions. Thank goodness I got out of there before the stock price hit zero. But not before my entire pension became worth somewhere around the cost of a nice meal for two at McDonald’s.  Ah well.  Live and learn.

 

********

 

From there I went to work in the State of Colorado’s Child Support Enforcement (CSE) division, maintaining a system that tracked deadbeats shirking financial obligations to their offspring. The Automated CSE System (or ACSES) interfaced with the State’s Welfare system.  At the time, Welfare’s system was undergoing an upgrade from an ancient legacy application.  The project was called CBMS (short for Colorado Benefits Management System).  Once again, a vendor’s off-the-shelf package was chosen to help contain costs. Again, the implementation went on far longer – and ended up far costlier – than anyone ever imagined at the outset. Of course, when the dust finally settled, upper management heads again rolled. And lots of programmers showed up each morning bleary-eyed after many sleepless nights on-call. So – again – we’d learned our lesson. Right?

Over the next few years I leveraged my Child Support expertise to do some contract work for other states (Illinois, Louisiana, Wyoming) before finally landing back in Denver with a teacher’s pension outfit, TIAA-CREF.  Eventually, in a streamlined triumph of marketing genius, they decided to shorten the name to just TIAA, which stands for (deep breath) Teachers Income Annuity Association.  But they were still running big-iron mainframe legacy systems by the time I left 7 years later. There weren’t any huge disasters, at least none I can still recall. But I do know for a fact they are still hiring DBAs to manage databases that are now many decades old.  So how successful at modernization can they really be here at the dawn of the 21st century? Oh, no, wait.  We’re already 20 years into the new century.  My how time flies when you’re having fun.

 

********

 

Back on the road again for me: With the Feds for a time at Interior, working on their Federal Personnel and Payroll System (FPPS). Thence to Topeka, armpit of middle America, for more State Child Support fun.  Then, returning full circle to Sacramento for another go-round on a teachers’ pension app, this one called CalSTRS  (don’t even ask).  Finally I landed somewhere completely different: Universal Music Group in L.A.   UMG is my first foray into the music business.  But it’s certainly not my first experience courting disaster – not by a long shot.  Hence, the real reason for this post.

There’s a recent article in Rolling Stone, and you can read it here. It’s about the meltdown at DirectShot Distribution (DSD) that has disrupted physical supply chain operations at each of the Big Three record labels – Sony, Warner, and UMG, where I now work. Conspiracy theories aside, the same stories of attempted technological modernization, short-sighted cost-cutting, and management turmoil are in process of playing out yet again.  Will we ever learn? For the answer to that question and more, stay tuned for details on your late local news @ 11….

 

But in the meantime, please remember to test your code.

BEFORE it gets to production.

Pretty please?  With sugar on top?

Trust me: You’ll be glad you did.