Many years ago, I was invited to an interview in Boston by a company who wanted to bring on additional QA staff. After the usual back-and-forth banter of the interview, the interviewer paused and asked me this question:
What is an acceptable defect?
I immediately replied "There's no such thing. It's an oxymoron, something that is inherently self-contradictory."
The interviewer persisted, wanting to know what an "acceptable defect" was. I replied that there is no such thing as an "acceptable defect" as once the defect becomes "acceptable", it ceases to be a defect. Or, as one of my favorite snarky QA quotes says: "It's not a bug, it's a feature!"
The interviewer continued to press the point; and yes, I understood that what he was asking was not about defects being "acceptable", but rather he wanted to know at what point does a QA effort say it's time to stop? When have you reached the level of "diminishing returns" and decide to release anyway, even though there may still be some bugs left to be resolved?
I mentioned this understanding to the interviewer, said that this was an entirely different question; whereupon I proceeded to answer it. However I persisted in my original position that there is not, and there should never be, such a thing as an "acceptable" defect.
Needless to say, I didn't get the job. And quite frankly, I wasn't too upset about it either. As far as I was, (and still am), concerned, the idea of an "acceptable" defect presents a warped and possibly disastrous mind-set on the part of any QA team.
Why is this a problem?
Now I can hear everyone saying that I'm being too picky and pedantic; that I'm splitting hairs. And maybe that's true, but I don't think so.
And why not?
It has always been my opinion, and my position within the greater QA community - not just "software" QA, but any kind of QA effort - that "defects" must never become "acceptable". Because once a defect is categorized as "acceptable", it ceases to be an annoyance or an irritation in the back of our minds. That nagging irritation goes away and we don't give it a second thought.
I have seen this time and time again, in hardware QA, software QA, process or manufacturing QA, or any other QA efforts I have been involved in. And it has always, invariably, been the fast lane to disaster.
I go into the consequences of this kind of complacent, devil-may-care attitude in another article I wrote called The Cost of Complacency. Go read it.
Again and again I return to the idea of defects never becoming "acceptable". It is the responsibility of the QA community to ensure that defects, even seemingly small defects, never get "lost in the sauce", so to speak.
Even if we need to push on toward an ultimate release date, these defects should always "get in our craw", or be that annoying little pebble in our shoe. They should always remind us that they are there, and we should continue be on the lookout for ways to mitigate them now, or if that's not possible, ways to avoid these issues in the future.
It's only by letting the seemingly "little things" get to us that we remain vigilant. It is only by this continuing attitude of eternal vigilance that we in the QA community earn the respect of those around us.
By doing this, by always letting the "little things" bother us, we ensure that the people we work with, and those that depend on us, never sink into that abyss of complacency that has been the graveyard of those who have not heeded this call.
What say ye?
Jim (JR)
Monday, November 17, 2014
4 comments:
Thanks for sharing your thoughts here at the QA Tech-Tips Blog!
Note:
This blog will not, repeat NOT, publish comments that contain ANY KIND OF HYPERLINK.
If your comment contains ANY KIND OF HYPERLINK it WILL BE DELETED.
Please read the article at How To Get Comments Approved On This Blog for additional information regarding this rule.
Thank you for understanding.
Jim (JR)
Subscribe to:
Post Comments (Atom)
Hello Diogenes, uh, Jim,
ReplyDeleteThe ethical part of me supports, actually applauds your position. The business reality part of me says, "We gotta get thing to market and start making money o n it." I've seen too many pieces of you-know-what get to market, sell a lot of product, only to have it break down a day after the warranty expires.
Let's face it. QA has come a long way since environmental testing of a simple piece of hardware. The complexities you guys have to deal with are mind-boggling. And yet, from a business point of view, how much time can a company devote to testing? How many test beds can they invest in if the product is a software package?
While I understand the company's point of view, I wish there were more QA people with attitudes like yours.
Alan,
DeleteYour points are all valid. More to the point, it is absolutely not reasonably possible to eradicate every possible bug in a piece of software - that is unless you have NASA's software budget.
I also know that there IS a point where you have to triage remaining defects and decide when to release anyway. And I am not denying that. It's a matter of simple business prudence that you eventually have to ship anyway. Hopefully you've caught all the catastrophic, serious, and significant bugs prior to release!
The point that I am making is that, though we may have to deal with the defects and ship anyway, this does not make the defect "acceptable" - it's still a defect, and it should still get under our skin.
Hopefully this kind of attitude will help convince the development staff and management that the QA engineer should become involved with the development cycle early on, when defects are easier to spot, and much less expensive to fix.
What say ye?
Jim (JR)
This poem I learned in grade school comes to mind...
ReplyDeleteOur choicest plans have fallen through, our airiest castles tumbled over,
because of lines we neatly drew, and later neatly stumbled over.
from Grooks, by Piet Hein
The reason this come to mind is because absolutely rigid thinking on any subject eventually impeded progress. Granted you may cite situations where that should not be the case, but you are wrong about NO defects being acceptable. I know that like me you have been involved with producing devices, circuits, and products. In doing so, at least in my experience on this planet, compromises always become necessary. I will grant that there is no such thing as a defect that should be deemed acceptable to the person or department whose job it is to identify them. If its your job to set aside all defects for further scrutiny, then you set them aside. But once identified, the choice of acceptability is ultimately a management decision. If, as a QA engineer, you insist your findings must override management judgement, you won't get the. But worse IMHO, if you ARE the manager in such a case, you may miss opportunities to successfully market your product or compete.
Some acceptable defect examples (1) functional flaws that do not SIGNIFICANTLY impeded the usability or safety of the system or product, and are deemed too expensive to correct (2) Non functional flaws such as INSIGNIFICANT Cosmetic issues that would cost SIGNIFICANTLY more to correct, but would only be noticed by an INSIGNIFICANT number of users, (3) defects discovered INSIGNIFICANTLY to far into the life cycle of a product to be worthy of correction, when there are no safety concerns.
I could add to the list, but you get the idea. The definition of significant is a management one, and a good manager will not be absolutely rigid to the point where INSIGNIFICANT must = "0", or SIGNIFICANT anything less than 100%
Randy,
DeleteAs usual you are a fountain of wisdom. However, my original point still stands - "defects" should never become "acceptable".
Perhaps I should clarify that a bit, as there is a wide gulf between "we can't hack this thing forever" and the "Aww F*** it!" attitudes.
Even with the cases you describe, though releasing the software/product/whatever might be absolutely appropriate, these issues - though seemingly "insignificant" - should still put us on our guard.
If I had one Russian Kopek, (right now, about 60 per U.S. penny), for every "insignificant" defect that came back and bit us, I'd be a bazillionare. In U.S. dollars.
We should always be asking ourselves if there are ramifications to this "insignificant" defect that might not be so "insignificant"? Classic case in point: The use of two digit years in dates. It was seemingly "insignificant" at the time, but cost zillions of dollars to fix.
Another thought:
What may be "insignificant" or "cosmetic" to YOU, might be a glaring, shameful defect to ME.
How many times have you purchased a product and looked at the documentation, only to discover that it looks like it was created by someone in the Pacific Rim who used Google Translate to generate the English language version? Or that had no idea how to use a spell-checker?
Maybe it doesn't matter to you, but it makes ME wonder.
If they care so little about the content and presentation of the product's documentation, how do I know they gave a damn about the quality of the product itself?
Here is an example of this same thing in Scripture: Luke 16, verse 10
"Whoever is careful with little is careful with much also, and whoever does evil [is careless] with little also does evil [is careless] with much."
What say ye?
Jim (JR)