When I started working in systems, one of my first client was a major bank.  Yes, this was back in the mainframe batch processing days.  They never did any system updates when they ran the month end, quarter end and especially year end.

I always thought that they just weren’t confident in their system folks and scoffed at this policy as it always made our deadlines shorter.

I think this story convinced me that doing production work these days on the bussiest web days is not a good idea.  Maybe microsoft should have borrowed a page from the mainframe policy manual - don’t do system updates on black monday or black friday as it may cause system outage.

This story: Microsoft Says Sorry For Black Friday Cashback Outage

For Internet users, Black Friday was supposed to be about buying and cashing back, but Microsoft’s Live Search cashback machine apparently broke down just as customers “barged in” to make some early morning purchases.

According to a blog posting, the unexpected outage occurred due to a significant spike in traffic, which caused the system to .html”>Buy Propecia go down for several hours. It took quite a while for it to come back to life, but apparently that was related to investigating the issue and rebuilding and deploying the databases and indexes that support Microsoft Live Search Cashback.