A good customer is a disgruntled customer. It’s these squeaky wheel customers who find value in their vendors’ enterprise software products, impatiently waiting for new features. The adoption of Continuous Integration/Continuous Delivery (CI/CD) pipelines enables ambitious companies to release production software rapidly – sometimes several times a day. Yesteryear’s customers anxiously awaiting new releases now benefit from obtaining hot-off-the-presses features. In an ironic twist, customers now complain about getting blindsided by new features that appear out of the blue.
Those who build software and are old enough may recall pre-internet releases; yes, there was software before the internet. Software teams cut floppy discs or CDs. Marketing teams created catchy slogans for the new versions, passed out catchphrase-emblazoned swag, and issued press releases. Documentation teams produced shrink-wrapped paper references, either bound or hole-punched for three-ring binders. Startups had parties to assemble packages and mailed them to customers. These laborious, late-night events occurred two or three times a year. Anxiety built in the post-shipping lull because days would pass before the first, brave customers received and installed their software. It all feels quaint in today’s hurly-burly world of Software as a Service (SaaS) and Cloud hosting.
Romanticizing the past is a mistake. Committing a software release to physical media and sending it via the mail invited despair when the first bug was discovered (sometimes moments after the packages were mailed) that necessitated shipping patch discs. Customers who desperately needed features were displeased to wait for semi-annual releases, sometimes to discover the functionality didn’t adequately address their pain. Disappointment morphed to fury when customers learned their features didn’t even make the cut and were relegated to subsequent releases. The glacial speed of releases gave dissatisfied customers ample time to brood and switch vendors.
Unquestionably, today’s technology for delivering and releasing software is a boon for engineering teams. These are some of the benefits:
- SaaS deployments eliminate the need to build and ship error-prone and time-consuming installers.
- Avoiding big-bang semi-annual releases invariably results in more robust software.
- Releasing software a soon as it’s ready provides developers with immediate feedback before they undertake new projects and forget past work.
- Canary releases expose new features to a limited set of customers and mitigate risk.
- Developers take satisfaction seeing their work used immediately.
For the most part, customers reap the same benefits as engineering teams. However, enterprise customers may also experience some of these pitfalls:
- Preexisting, relied-upon functionality may abruptly change with little or no notice.
- Management must scramble to understand new, unexpected features to educate their employees.
- Confusion and errors result if workflows or user interfaces change without time for workers to absorb the changes.
Interestingly, the aforementioned downsides are largely communication failures, not technical failures. Countless tomes detail the nitty-gritty of trunk-based development, automated testing, pushbutton deployments, and myriad other technically forward topics. Sadly, there’s little written about customer success in a CI/CD world. The compressed timeline of continuous deployment requires at least a well-oiled Product organization. Moreover, tools like Pendo, which provide in-application feature education, are good, customer-facing adjuncts to harried Product teams.
Customers may demand runtime toggles to hide/show new features in response to the lack of lead-time. This is an antipattern that complicates testing in direct proportion to the number of toggles. There are 2 ** n combinations of toggles where n is the number of individually toggled features. In the case where there are two toggled features, there are 2 ** 2 = 4 possible combinations:
- Feature A ON, Feature B ON
- Feature A OFF, Feature B OFF
- Feature A ON, Feature B OFF
- Feature A OFF, Feature B ON
Feature configurations exceeding two or three toggles presents a testing challenge. Customers may discover that two features are incompatible instead of an automated test suite.
Even worse, if customers experience the pain of too-frequent releases and ensuing quality issues, they may demand a stand-alone instance, another antipattern that’s anathema to a SaaS model. Implicit in the request for a stand-alone environment is customers’ ability to control when to apply features and patches. If customers choose to stack up changes for a long time, it resembles the bad old days of shipped software.
Part Two of this Quality series presents the state of the CI/CD art and how to avoid some of the pitfalls that result in destabilizing antipatterns. The next post also examines the question, Just because we can publish software multiple times a day, should we publish multiple times a day?
