Is your product ready to ship?
In the early days of Synapse, we had a client who would come to us 6 weeks before an industry trade show every year with a list of product concepts that they needed working prototypes of. We would spin up a team of engineers and make 6-8 prototypes in a few short weeks. The team motto was “Done is Beautiful”*. We weren’t constrained by product cost or manufacturability, so every decision we made was first and foremost about what would get us working parts on a near-impossible schedule. Since then, Synapse has worked on a lot of products that go into mass production, and while done is still beautiful, it’s more difficult to define. You never have the time or budget to reach perfection before shipping, so how do you decide how good is good enough?
You never have the time or budget to reach perfection before shipping, so how do you decide how good is good enough?
With software, you can usually make updates after you’ve released your work into the wild. Hardware is much more difficult to upgrade. Consumer products get used for years, and industrial products can be in service for decades. Releasing hardware that’s not mature enough can be bad for your brand and cost you lots of money in customer service and product returns.
Before we get into the details of how to tell if you’re done, we want to remind you that the cost and complexity of making changes to a design rises substantially as you get closer to manufacturing. This blog post isn’t a checklist of items that you should consider the month before you’re supposed to ship; it’s a set of things you should consider throughout your development process. Spending time up front to understand your users, define exactly what your product is and isn’t, and prototype areas of design risk is always a good investment.
Have you tested prototypes with real users?
The best way to preview what will happen once you launch is to get real users using your product. Give a prototype to one of your potential customers and try not to hover or explain too much. What users initially say is often telling about what design features they will gravitate towards in the end. They might not even notice the things you’ve decided are most important. Every feature you have in a product takes time to specify, design, and test and a great way to ship products faster is to only design in the features that have sufficient value to your users. User testing can also save you valuable time in re-design of features after they’re implemented. For example, one product we built had a “trigger” button that was so seamless in the industrial design, users often had to be taught where the button was. A slightly larger reveal space between the parts solved the problem and the button became intuitive.
Leaving prototypes with users for days or weeks can tell you a lot about product robustness. You know how expensive your pre-production units were to build, and you treat them gently. Real users are not gentle. On many products we’ve worked on, particularly wearables, simulation and fixtured cycle tests rarely replicate the failure modes we see in real-world use. If your users cause failures that you didn’t catch with your standard set of accelerated life tests, you can update your test plan to make it better. In certain cases, Synapse has even built specialized test equipment to mimic user failures, like our dynamic water ingress tester.
If you’ve spent months or even years working on a particular project, it can feel like you know your mission inside and out, and user testing isn’t the best use of your time. If you can humbly accept that you don’t know all the things about your user (who knew people use electric toothbrushes on their chihuahuas??) this step will feel easier, and like the wise investment that it is.
Were you as deliberate about reliability and verification testing as you were with the design?
Design engineers tend to focus on design, but testing is the step in the product development process where you really understand and improve your product. Do you have a realistic understanding of how reliable your product needs to be? Do you have a verification plan that enables you to see how well you’ve met your design goals? When you write that plan, don’t just test to your product requirements, test to failure. If a button needs to survive 10,000 presses, it’s useful to know whether it fails at 11,000 or 100,000 presses. If a sensor needs to work over an operating temperature range of 0-60C with a certain accuracy, it’s useful to know whether it goes out of specification at -5C or -30C. Design margin helps you scale products to volume production, and good verification testing tells you how much design margin you have.
Design engineers tend to focus on design, but testing is the step in the product development process where you really understand and improve your product.
Fear of uncovering a design flaw is so entrenched, even the most seasoned engineers can want to ‘ostrich’. It helps to think of each failure you see in verification testing as something you won’t have to fix once the product is shipping. Redesign is expensive, but we build it into schedules intentionally because no one designs a perfect product in the first attempt. Redesign after launch can be ruinously expensive, so use your test data to redesign at the right time.
Does your product meet regulatory requirements?
You can’t sell an electronic product without going through regulatory certification. If you’re selling a wirelessly connected product, this certification process is more complicated. Unfortunately, different countries have different regulatory certification requirements, and the process can quickly become expensive and complex. You’ll want to make sure your engineers and marketing / sales team are on the same page about launch plans. Regulatory testing also frequently gets ignored until right before launch, when the costs of redesign are high. When you get your first form factor prototypes, you should consider a regulatory pre-screen to look for issues. Regulations aren’t just for RF and EE types either. For example: if you have a project where biocompatibility is required, it is much better to consider that from the beginning, and work the testing into the schedule, rather than to have to re-run a design cycle to make parts in better or different materials.
Do you have production testing in place?
A successful pre-production build with your chosen manufacturer is a big, thrilling milestone. High fives and champagne all around! But how can you ensure that the first 10,000 units you ship will all work exactly like these prototypes? As manufacturing ramps up, it is quite common for processes to change or procedures to be streamlined. Parts get second-sourced, and operators doing the assembly change. Do you have sufficient production line tests in place to catch any issues that these changes can cause? In order to design a production test system, you’ll need to work with the manufacturer and refer to your product requirements. What are the most important things to test? Do you need to test them on every unit or are you comfortable with batch testing?
How good is good enough to ship?
By now, you might be saying “OK, OK, I get it, testing is important. But you said you’d tell me how to know when I’m ready to ship.” We’ve spent so long talking about testing because that’s how you understand where you are relative to “done”. It’s very likely that you’ll go through the testing we describe and find out your product isn’t perfect. So what do you do then?
The reality is that imperfect products ship all the time, and sometimes shipping an imperfect product sooner is the right business decision.
The reality is that imperfect products ship all the time, and sometimes shipping an imperfect product sooner is the right business decision. If you’re running a startup that needs a product on the market to reach a funding milestone, shipping a product that meets most but not all of your requirements is probably the right answer. Maybe you’re part of a larger organization that’s trying to launch a completely new type of product and the value of getting something on the market to get real consumer insights is worth the tradeoff of implementing a no-questions-asked returns policy and replacing failed units. If you’ve done good testing, you have the tools to make an informed decision when you find yourself faced with this choice. When you get a test failure, you can go back to your product requirements for context about why the requirement was included, how critical it is, and what else it impacts. Even making a strategic decision to skip certain types of testing to save time gives you more information – you know a little bit more about what you don’t know. In the end, we can’t actually tell you when you’re ready to ship, only you can decide that based on the realities of your situation.
To engineers, it can often seem as if “done” is when the design is done – cross that finish line, and relax. In reality, the design isn’t “done” until it has been designed, tested, re-designed, re-tested, certified, scaled, and re-tested. After all that, done IS beautiful.
*Martine brought this saying to Synapse, and she got it from the Bucknell University Department of Theatre and Dance. If you want to feel better about your tight timelines, see someone who works in costumes or set design.
Podcast
