Wednesday 23 October 2013

Cert, Then and Now

 
This Friday my team hits code freeze - which, in case you don't work in software (looking at you, Mum & Dad) is kind of a big deal. It means we've only a day or two left to make any fixes or changes to our app before we branch - or "freeze" - a Release Candidate build.

This "RC0" build then gets tossed over the fence to the certification and regression testing processes required to sign it off for launch. If all goes well (pro tip: things never go well) this build will be the final 1.0 software that gets released to users when the new Xbox launches on Nov 22.

More likely, Cert and QE will find a shipstopping issue that we've somehow missed, and the team will frantically fix and spin up new RCs until one is deemed worthy of public release. (We've started an office pool on what RC number the Xbox final OS will ship with - but that's probably not something I ought to say on the Internet!)

Cert is always a weird time in the lifecycle of a product and it's one they rarely teach you about in school. Having a build in Cert means you are essentially on-call - it's limbo-like but you have to be on your toes ready to fix a shipstopper at any moment. I have fond memories of my first ever Cert on the Madden game back in 2008 - I was a lowly junior engineer so none of the really tough ship-blocking bugs came to me. So for me, life was pretty damn chill. Our entire dev team of 11 engineers piled onto one sofa together and spent 12 hour days playing Wario Ware and Boom Blox on a retail Wii kit while we waited for bugs to roll in.


Of course that was the old days - because this industry moves so quickly I'm basically a granny. Now the world has shifted from box products to digital delivery, there is usually a 1.1 release to start work on immediately. No time to take a big holiday or comp leave while the discs get manufactured. Users expect you to push them an updated version at least once a month! And that's not a bad thing.

The larger software industry's paradigm shift to Agile has also helped. Hopefully, if your team has been following Scrum methods they'll be working at a sustainable pace and won't be burned out in need of a rest come launch. They'll also have been releasing every sprint (even if just to soft launch or internal dogfooding) and have automated testing so the build quality should be high - meaning fewer issues to find in a certification pass.

In fact, it's possible to make the argument that Agile precludes the need for Cert and that it's a sad old relic of the console waterfall era. I'm not going to make that argument, as there are enough people discussing "bug sprints" and "stabilization" or "integration" sprints to indicate it's still a common enough problem at the end of a project. And certification continues to be required in one form or another on all closed platforms from game console TCRs to the Apple App Store's submission process.

So while I miss the weeks on weeks of Wario Ware, I think dev is generally changing for the better. Our Cert period beginning next week will be a interesting time, with attentions divided between 1.1 development and urgent fixes on the 1.0 RC. Cross fingers there aren't too many of the latter!

No comments:

Post a Comment