Yes, it is important to ship often, get feedback, and make the software better. Especially if you’re developing a service shipping to customers, it is a great way to get the code executed. Shipping improves quality.
It’s given that first couple times, there will be failures, more bugs, fire-drills that would make you look bad in front of your management and customers. Especially if you’re transitioning from longer release cycles. But it will be a short period of time, and benefits will show themselves pretty quickly.
All good bugs are found when the service goes live and running in front of customers with their data. Those bugs cannot be found in the lab, while testing. A test cannot match to a customer’s mind or behavior, and the test infrastructure cannot match to the combined force and configuration of phones, tablets and computers out there!
How do you know you’re ready to ship?
Most often, you won’t. You’ll feel ready, with an expectation of a learning opportunity coming up. You can increase the confidence level, by having regular automation tests, component level tests, scenario level tests (for defined scenarios). And these will cover a lot of paths in the code, but not all unfortunately, and enough if you had just these.
The code can tell you more with its data. Data that comes out from your service, telling you how much the failure rate is per scenario, and where the failure is, and provide enough information in numbers so that you’ll be able to fix it in a quick short amount of time. Logs may not be be as affective as is development stage. Real usage would be high, and there will be a good amount of logs collected due to the usage. Tracing where the failure is and debugging live would be a big time spent for too small.
So if you have a habit of shipping often, preferably in small chunks and gather data about how things are going from the code that goes out in front of the users, the quality of software increases. Because the increment goes out is tested by real users, running with real data and if a bug comes up, it will be fixed rather quickly because the change went in the other day is probably small.
