Lately there seems to be a lot of press coverage about the evils of test certification from uTest’s series of “Testing the Limits” interviews.  uTest seems to be interviewing all of the “cert-slammers” in a row.  While I understand that they are just printing what their interviewees have said, the email notifications going out to subscribers sure seem to stress the cert-slammer rhetoric; that has moved me to write this response to all those cert-slammers out there in the test world.

In one uTest interview James Bach states, “You see, certification is promoted by frightened people who feel they need elaborate content-free ceremonies in order to feel competent.”  Personally, I think Mr. Bach’s stance on certification stems from his deep seated fear that someday he might be forced to take a test and fail it miserably.  Test anxiety is not a valid reason for so vehemently putting down a program that is helping fledgling testers find jobs!  The job market today is vastly different than it was back in 1987 when Mr. Bach first entered the SQA field.  Today a candidate has to have more than “self-taught high school drop-out” on his resume.

As a recruiting organization, ProtoTest is watching the software test job market change drastically.  Not only is Software Testing the #12 career with the fastest growing salary in the nation, but it is also one of the most difficult to break into.  Gone are the days when reading a book would convince a hiring manager that you can hit the ground running in the software testing industry.  A bachelor’s degree is almost always required and sadly there are very few college level courses in testing.  Without accredited training at a college level to give a person interested in software testing the necessary knowledge and skills, what do they do?  Many of us currently in the industry stepped into our roles completely by accident and learned on the job, sink or swim!  So what are we to say to aspiring testers when they ask, “How do I become a software tester?”

Another cert-slammer, Michael Bolton, was also interviewed by uTest and continues his colleague’s short-sighted propaganda stating, “No group should be dictating to other people how they must test unless there are compelling human health and safety reasons for it.”

Let’s step back and look at this objectively.  First, in no way do certification programs state that by going through their program or taking their test a person’s brain will automatically become a cloned robot that has no ability to think for itself!  All certification programs are trying to do is give people the basics in order to make their own best decisions using a standard basis of test techniques.  Saying that certification programs dictate testing is like saying that degree programs dictate how students must work for the rest of their lives!  Getting a degree from a university only gives a person the basis of information they might need in order to enter the workplace.  Then they have to go through a lot of on the job training and hopefully mentoring with more senior colleagues.  The same is true for certification programs: they give people the basic materials needed to enter the workplace or understand the techniques and the vocabulary in order to communicate with the rest of the industry. 

Which brings me to the fact that certification programs were not created for people like James Bach and Michael Bolton who obviously do not need to be educated in the ways of testing.  Certification programs are designed for people who want to get into the field of software testing or have recently moved sideways into the field and want to get a foundation in test techniques.  And that is all certification is, a foundation from which their own experience and their company’s context will help solidify their ability to effectively test software.

Just because there is a new path into our field does not make it the “wrong” path.  It may not be the “right” path either.  In a perfect world, we would have a degree program at Universities and Colleges across the country for aspiring software testers, or at the very least there will be a course or two required in the Computer Sciences degree program.  I would really love to see that happen.  But, until then, how do test neophytes learn the techniques, the terminology and the theory behind what it is we do?

That’s right, certification.  Sure, they could go read the copious numbers of software test books or take a free, self-study course online.  Sure, they could mentor with James Bach on Skype.  But, do they have a kite’s chance in hell of getting a job from just having read a book?  No.  Does a free, online class have any clout on the market whatsoever?  No. 

Please do not mistake what I am saying here.  I am not saying that a test manager should base their hiring practices on certifications.  Having been a hiring QA Manager, I can tell you that there is a lot more to consider than just a piece of paper.  However, if I am handed resumes for an entry-level test position and see a certification on one in a stack of 10, I will look at that as a person who cares more about their career choice than the others.  I am also not saying that certifications should become a requirement for applicants, either.  Certifications do not prove that a student has the ability to DO the work, just like degrees do not prove that a student can do the work either.  There are no universities that guarantee hiring companies that the graduating class has the ability to work.  I think it is unfair and ridiculous to expect such from a certification program.

The fact is that certification programs are educational.  Personally, I find that education is never a bad thing and is never inherently evil.  What every student needs to know walking into a certification program --ANY certification program-- is that what you learn and whether you pass that test is not the end of the learning experience.  In our industry, every employee must be prepared to adapt to the context of their company’s policies and projects.  Technology, practices and tools change too rapidly to assume that what you knew even a few months ago still applies.  What you are taught in a certification program is not intended to dictate how you or your company must do anything; it only gives you a starting point and a vocabulary with which to communicate the possibilities.

-- Charity Stoner

Just over a year ago I blogged about the poor quality of software and the general apathy the general public has about it. Now it is 2010 and it continues.

Let me relate an experience I had last week that illustrates an example of poor software and the underlying reason why I think poor software exists and why we, as the public, are used to it.

Last week I needed to take my 2001 Toyota Tacoma in for an oil change. I love my Toyota tuck. I have only driven Toyota trucks my entire driving life. I think they are the best vehicle for my lifestyle. So I do my best to take care of it. Go Toyota Scion on Arapahoe (they used to be John Elway Toyota) in Centennial, CO used to offer a QuickLube service. They guaranteed they would change your oil in 30 minutes or less or it was free. I loved this service. Like most people I am busy and this was a great way to get a needed service and not have it take ½ my day.      Then they did away with that service. I persisted in taking my truck there since they are close to my office and generally they would still get it done fast. The last couple of times I took my truck in, it took a very long time. So I was exploring other options for getting my oil changed.

Then I get a post card in the mail telling me I can now schedule an appointment on-line and they even give me a couple for a discounted oil change. I thought to myself ‘this is brilliant! I can schedule the time to come and be in and out quickly’.

However my joy quickly turned to disappointment. Last week I went on to the sight to schedule an appointment. It appears they have a third party that created their appointment booking system – Xtime.

The app leaves much to be desired. The spacing is horrible. There is a scrolling message that is cut in half as it scrolls by. See the screen shot below.

Anyway, I go use their web interface to schedule my appointment. I pick the time and service I need. I get a confirmation email of my time and service. I then drive over with my printed out service request and appointment reminder. I hand it to the service advisor and he tells me they don’t accept appointments for oil changes. I stand there flabbergasted. I say ‘no, you do. See your application did exactly that’. He said ‘yeah, that is a problem. It lets you do that but it doesn’t mean anything’. I tell agree with him that it is a problem. So I ask ‘what am I supposed to do?’ I have appointment that is worthless that your system let me create. He said there is nothing he can do about it. All I can do is wait the hour+ to get my oil changed. Doesn’t that defeat the whole purpose of an appointment? I tell him to keep the appointment and my coupon and drive off in disgust.

As I drove back to my office I began to ponder about what I wrote last year about poor software. I don’t know if Xtime created a poor piece of software or if Go Toyota Scion on Arapahoe gave them poor requirements. Why allow your software to do something that is not honored. When an airline makes a mistake and offers tickets at an incorrect price and people buy them, the airline has to honor the ticket it is their mistake. But so often today companies make the customer pay for their mistake.

I tried to contact the service manager – I believe it is Tim Moorhead – but their site lists no email address for the Service department. You can either call or chat with a representative. I wanted to email him to let him know my experience. However, I could not. So I chatted with someone and had to cut and paste my entire email piece by piece into the chat window. Of course, no one from Go Toyota Scion on Arapahoe ever contacted me. But it is interesting that they now have a scrolling message (which you can only partially read because of very poor formatting) that informs people scheduling appointments that they do not need to schedule an appointment for minor services.

Do I think Go Toyota Scion on Arapahoe or Tim Moorhead is the spawn of Satan? Do I think they get up in the morning and hatch diabolical schemes to create poor software and anger their customers? I can’t find any evidence that points to that. But in general customer service and quality customer care have degraded over the last 30 years. Companies make calculated decisions based on bottom line financial numbers. They know that they can spend a little more time making their appointment scheduler work correctly (I have build software for nearly 20 years I know a little something about that) or they throw something up there and have it mostly work. %95 of companies will choose the latter option because it costs less. And in the end they don’t care about customers (even though they all pay it lip service) they only care about customer wallets. Do they care about my $30 for an oil change? No, not really. If I was a customer that was going to pay $1,000 for a major repair then Mr. Moorhead may have contacted me and apologized for the mistake. But I am a little customer. Not worth his time. This is the true test of whether a company values it customers or not. If they treat all customers, big and small, with the same courtesy and respect they deserve. But that costs money and companies won’t do that. And if you have a person that wants to offer that level of service they are often told my upper management to stop that because they don’t have time for that. Bring your numbers up and nothing else matters.

It isn’t just Go Toyota Scion on Arapahoe that suffers from poor customer care and bad software. It is symptomatic of our society. Greed rules. The last year has taught us that. What person needs a bonus of millions of dollars? No one needs that. Yet we see upper level management across all industry strive for the absolute most they can suck out of a company. Do they live by the motto of ‘leave something better than you found it?’ Hardly. We as a society have lowered our standards to just accept it. You often hear people say ‘what are you going to do about it’ or ‘it is what it is’. But I would love to us start to hold each other and companies to higher standards. Maybe, just maybe, some might actually start to change.

~ Lawrence Nuanez

ProtoTest has partnered with KnowledgeUnited to offer the first real-time virtual instructor-led (RVI) ASTQB accredited course.  Take a Certified Tester Foundation Level prep course from the comfort of your own home office while still having access to a certified instructor to ask questions, get help to complete test technique exercises and interact with other course attendees from around the United States. 

ProtoTest’s “Fundamentals of Software Testing” course is taught using KnowledgeUnited’s Adobe Connect Pro virtual classroom environment.  You provide the high-speed internet and we will provide a training experience for you or your whole team.

Recent Post-Course Survey Responses:

“I enjoyed the class very much and it added knowledge.”

“Instructor [ProtoTest] was excellent! She was very organized and stayed on schedule. Very responsive when asked questions. Clearly explained topics. I would definitely take another course like this (online, instructor-led) and especially if [ProtoTest] was the instructor.”

“I passed my exam today, so I am happy!”

To find out more about the class, upcoming training dates and to register please visit: http://knowledgeunited.com/course/detail/Development/3743

About ProtoTest:
ProtoTest is a consulting firm dedicated to Software Quality Assurance and Testing.  Courses offered are software test related including the Fundamentals of Software Testing (ASTQB accredited) and Applied Agile Testing.  Based in Denver, CO ProtoTest consultants serve software companies from Fort Collins to Pueblo, CO.

About KnowledgeUnited:
Knowledge United provides organizations customized learning solutions built to your specific needs. By combining our unique approach to organizational learning and our global reach through specialized education partners our suite of services is virtually endless.

Back in March we blogged about low cost load and performance testing options. Back in November 2008 we blogged about load testing in the cloud. It is amazing how quickly things can progress when you have very smart people charging the way. LoadStorm has in less than 1 year grown to over 2000 customers. Other cloud based load testing tools are also experiencing a rapid increase in customer base.

Other tool vendors have not kept up. LoadRunner is as expensive as ever. I can’t even recall a single time over the last 5 years I have recommended LoadRunner to a client. Borland had been making nice progress with offering their tools at a reduced price with the Testing as a Service offering. But now that they are Micro Focus, that went out the window. They still offer the service but the prices went WAY up… they are now at the end of line with regards to tools I recommend. (HP LoadRunner doesn’t even get in the line)

So it was nice to see that another company isn’t scared by the cloud based offering. No… in fact they embraced the technology and are adapting. They are displaying the trademarks of an agile forward thinking organization. They are Web Performance. They have adapted their product to make use of the Amazon EC2 cloud. I was already a big fan of the Web Performance Load Tester application. I think it is one of the best out there for load testing web applications. Now that it can be deployed over the Amazon EC2 cloud it just makes them even better. I have a client coming up that this may be a perfect fit for. It will be exciting to see how this moves forward.

It has been almost a year since we at ProtoTest have started to make use of these new, exciting, and lower cost options. We have seen a dramatic spike in the number of customers doing load testing now because the cost is so much less. What you seeing? Have any of you used these tools? What do you like about them? What do you hate about them?

~Lawrence Nuanez

LoadStorm is  running a campaign right now where you can enter a contest on their web site to win gift cards to Best Buy or the Apple Store. According to them "all we ask is for your thoughtful input, complaints, or suggestions regarding our load testing tool".

It cost nothing to enter and you might even get to do a little load testing done at the same time. You probably have a much better chance of winning a $250 Best Buy gift card than of hitting Publishers Clearinghouse or the Lotto. Give it shot  - LoadStorm Contest

Let us know if anyone wins anything

~Lawrence

Check out this URL - http://loadstorm.com/2009/load-testing-lawrence-nuanez-interview

This is an email interview Scott Price of LoadStorm conducted with Lawrence Nuanez. He shares his insights and testing expertise. In this email interview, Lawrence talks about his views on software testing, load testing, test automation, and off-shoring.

If you think I am way off say so. If you think I am right about at least a couple of things.. say that too. These new cloud based load and performance tools are an exciting offering and offer great flexibility. Check them out and see what you think.

~ Lawrence Nuanez

Over the last few weeks we have been talking a lot about load and performance testing. On January 23, 2009 we blogged a lot about low cost load and performance testing and how even in a bad economy smart companies make wise investments in such things as load and performance testing when it is necessary.

Over the last month we have really been on a war path to make sure people know about all their options when it comes to lower costs load and performance testing. On March 6, 2009 we host a breakfast roundtable in Denver, CO. It only last about 2 hours but it was a great experience. Those that attended were able to see about options they did not previously know about. I don’t think there was single person who attended that did not walk away with a new thought they could take with them.

We talked about three different options

•1) Using a proprietary tool on a temporary basis. More companies are realizing that buying a 60K – 150K+ tool isn’t going to happen for most people. So they are willing to let people lease their tool for an hour, week, or month. Most of the time you have to go through a partner or at least someone who has a relationship with the vendor but it still can save you 90%+ on the cost of the tool

•2) Open source was another option. Right now there are 39 open source load and performance testing tools listed on opensourcetesting.org. Although that number should drop by 1 since Radview has tomb stoned the open source version of WebLoad. Not a big lost for most since it was fairly weak –purposely so – compared to the pay version. OpenSTA just came out with a new version and Apache JMeter and Grinder are also excellent options.

•3) Most of the excitement was reserved for cloud based offerings. Two that we talk about extensively are LoadStorm and BrowserMob. Each is different in how they approach utilizing the load cloud to offer a load testing tool but both are very good early adopter tools. We will be utilizing these options for more of our clients to help drive down costs.

Next month at the month SQuAD meeting (4/14/09) we will be presenting a slightly expanded version of the breakfast seminar series on Low Cost Load and Performance Testing. All are welcome!

~ Lawrence Nuanez