Blogs

Gary's Blog

Does Testing Make You a Bad Programmer?

     

If you’d asked me that question last week, I’d have said no, of course not. Actually, I’d probably have laughed at you and then said no, of course not. But today, I’m not so sure. So what’s changed? Simple really, last Wednesday I went to my son’s parent’s evening at the local Army Cadet Force Detachment, where he’s a member. There’s an obvious connection there, right? Well no, there’s not, and it took the lesson a few days to congeal in my mind. Let me tell you what I saw and why it made me form the question in the title:

Army Cadet Force – The Way it Was
First let me set the scene for you. I was a member of the ACF for 4 years, between 1984 and 1988, rising to the rank of Company Sergeant Major. At that time the assault course was completed by teams of cadets, against the clock and wearing battle order webbing. It featured in every military skills competition I ever competed in. The assault course at Barry Buddon, our weekend training area, is a third of a mile long and is a severe test of physical strength, endurance and team work. It is one of the longest and most demanding assault courses in the UK. No matter how introverted a cadet might be, you could always tell his character by the way he tackled the assault course.

The Modern Army Cadet Force
Fast forward 20 years and things have changed. A “risk aware” culture now permeates the ACF. The first thing that went was the battle order webbing, cadets are no longer allowed to wear it whilst crossing the assault course, I mean if you fell it could trap you in one of the water obstacles, couldn’t it? Next the name, assault course is a little “war-like” isn’t it? It should be changed to something a little more soft and cuddly, and so it is now an “obstacle course”. Then helmets. Helmet should definitely be worn, cadets might bump their heads whilst going through the tunnels or be hit in the head by a returning rope on the rope swing. Next the clock was removed. Cadets are no longer allowed to race across the assault course. Trying to beat a time, may cause them to take unacceptable risks. Finally, all of the water obstacles should be drained of water down to just below knee height, anymore and a cadet might drown if they couldn’t regain the edge. There is even talk of cadets having to wear safety harnesses and have a safety rope when completing the high obstacles. I haven’t been to the assault course since I left the cadets. On Wednesday night, I eyed the beast of yesterday, only to find that it’d had it’s teeth pulled and it’s claws blunted. It was a little sad to see if I’m honest.

Diminishing Returns on Health And Safety
Still, you have to balance that with the safety of the cadets, and they are much less likely to hurt themselves now, right? Well, no actually. In the 4 years that I was a cadet, I didn’t see a single bad accident on the assault course. In fact, I saw no accidents whatsoever, excluding the odd grazed knee and the like. But on Wednesday, there were a couple of cadets who hurt themselves quite badly, despite the ridiculous health and safety measures. Why? Simple, the cadets of today have no respect for the assault course. They are used to operating in an environment where every danger has been removed or blunted that they fail to take even the most basic precautions for themselves.

In my day, apart from the odd shout of encouragement or instruction, there wasn’t a word uttered – everyone was concentrating on their role in the team, squeezing out every wasted second, everyone was giving every ounce of strength they had to finish in a fast time. Not now. Cadets were strolling along, exchanging the gossip of the day, totally oblivious to the dangers that might still lurk – because it is impossible to remove every danger from an environment. Consequently, a couple of the cadets took a tumble and hurt themselves, one of them pretty badly.

You see, it’s clear to me that there is a diminishing return on health and safety measures. When helmets were introduced, I’m sure there was a reduction in the number of head injuries sustained. Now that number wasn’t big to begin with, because we were all aware of the danger, but reducing that small number was certainly a good thing. However, as each successive layer of health and safety was added there was a diminishing return because for every one person you protected, another was injured because they had become oblivious to the danger. On Wednesday night I saw more cadets injured taking a leisurely stroll across the assault course, than I saw during the whole of my cadet career.

The Danger of Working in an Over-Tested Environment
It occurred to me that this was not a function of something that the Army Cadet Force was doing, but instead it was a function of human nature. The more safe an environment is perceived to be, the less careful we humans are. Naturally then, this scenario is applicable to software engineering. With the current focus on TDD and automated unit testing, it stands to reason that the modern generation of engineer perceives him or herself to be working in a safe environment. I mean all the tests pass, so the code must be okay, right?

Unfortunately, as the above shows, that is not the case. Just as the ACF can't remove all of the dangers from the assault course - so each cadet must take responsibility for their own safety whilst completing it - so tests can’t verify that your code works under every possible execution scenario. As a programmer, you must take responsibility for your own code’s safe execution. Before the fashion for TDD and automated unit testing, programmers were adept at writing defensive code, just as my generation of cadets were adept at ducking when they saw a heavy rope coming. The newest generation of programmers must not allow the presence of unit tests to lull them into a false sense that their code operates in a “safe” environment. If that happens, if they rely only on tests and do not learn how to code defensively, then the answer to the question posed in the title of this blog post will be yes, testing does make you a bad programmer.

Published Sep 13 2010, 06:28 PM by Gary Short (DevExpress)
Filed under:
Technorati tags: Outside DXperience
Bookmark and Share

Comments

 

Michael Thuma said:

I have 3 Points

a) Kathy Sierra's class ->> The more you use your reins - the less they use their brains (little adopted).

b) Overtested means - quick quick and we will get the bug later. This is the false thinking. This happens and from a bad habit something is pushed into the calculus of the application of a methodology.

c) Unit testing is strong, when you have lot's around your logic generated. Many people use it for a certain kind of coverage testing ... this is not much.

Automated testing can make one lazy...this is true. Not a bad programmer but not better and over the times you could be right - not getting better is making you worse.

Everything that has to do with batch operations are risky if not applied correct. Regression testing too. The good it is ... taking things into account and being little more preceise earlier ... is hell over the times.

September 13, 2010 2:43 PM
 

John Prideaux said:

While one can't be exhaustive in unit tests, tests that demonstrate graceful failure in the event of bad input reduces this risk substantially and are just as important as tests that demonstrate correct behavior when given valid input.

September 13, 2010 2:54 PM
 

Gary Short (DevExpress) said:

Just to be clear, I'm not saying tests are bad. I'm saying human nature, which causes us to focus less on danger when we feel safe (no matter how real or false that feeling is) is bad when it comes to programmers and testing.

September 13, 2010 2:59 PM
 

Kirsty Busfield said:

I agree with the sentiment that unit tests can create complacency

Just as a thorough regression pack can...

Or a large testing team.

Unfortunately it's human nature, we strive to do better to make it easier for ourselves. Everything is engineered faster, better, stronger. All so we essentially can do less, or get more done in the same amount of time.

I heard a great analogy once; think of the attitudes in a department as a swinging pendulum. Everything's fine, the software runs well, we're all happy, people start getting complacent, the unit tests slip and begin to offer poorer quality coverage. Still the application is performing, gradually unit tests become outdated and unmaintained and the pendulum effect is swinging the other way. Suddenly all hell breaks loose, something was missed, released into production and everyones working every hour to get the app back on track. People need to answer for the mistake, massive amounts of processes and red tape are introduced to try not to let this happen again. The unit tests and test packs are finally overhauled. Everyone starts to relax, the application begins to stabilise again....and what's this? The pendulums right back over the other side again and it all starts again.

I feel the only way to try to manage this is be aware of either extreme. When all is quiet, be wary, book time in for someone/people to review testing, rather than see it as an opportunity to fit in more  project deliverables. 

In the meantime, give your son a spade, get him to dig a series of deep ditches in your garden, fill them with water, and get him to assail them with his feet tied and weights on his back. Tell him he's not to come back inside until he's realised that it was much harder "back in the day"

;) poor lad      

September 13, 2010 6:34 PM
 

Gary Short (DevExpress) said:

Hi Kirsty,

Thanks for the feedback and the analogy, it's a good one. I particularly liked the last paragraph, I almost took out the phrase "In my day..." but I knew it would make some people smile. It's true though, these people do not pack the gear to serve in my beloved battalion. :-)

September 13, 2010 6:42 PM
 

Steve Sharkey said:

Interesting about the assault course err obstacle course - this is mirrored throughout society: take formula 1 it is now "so safe" that you get people taking silly risks like Michael Schumacher on Barrichello and the numerous similar occaisions where drivers try to force others off the track - when the consequences of this behaviour would most probably have resulted in death it wasn't done. Now it is barely commented on for many of the occaisions. I'm not saying the drive towards more safety is wrong but it has got out of balance - like the argument that a spike rather than airbag on the steering wheel would save more lives due to more careful driving!

September 14, 2010 4:20 AM
 

Christoph Brändle said:

some programmers tend to think that unit test make their work better, but even unit test are just as good as the programmer. so probably the not so good programmers think unit tests are really very important (because the unit test ist the best part of their work?). i think unit tests are rarely that useful

September 14, 2010 8:28 AM
 

Felipe R Machado said:

Before Jigoro Kano created Judo, there was jiu-jitsu. Jiu-jitsu was extremely dangerous to practice. Kano removed the dangerous technique so that "jiu-jitsu" could be trained with full force and commitment. The result is simple: people training Judo were able to evolve faster and reach higher standards than people training in traditional jiu-jitsu, that couldn't actually fight each other, they had only "staged" fights due to the inherent dangers of their "uncontrolled" and "deadly" art.

So, using this analogy, TDD can make to the uncontrolled and dangerous art of programming the same that Judo did to jiu-jitsu.

So, beware of analogies, ultimately ALL analogies are fundamentally flawed. I can prove anything right (or wrong) through a careful choice of analogies.

September 15, 2010 2:16 PM
 

Gary Short (DevExpress) said:

Hello Felipe, yes you are right, one should beware of analogies. In the case of yours, I happen to have studied Ju-Jitsu for many years and utterly refute your assertion of the origins of Judo. However, analogies do help to illustrate valid points, no matter how flawed they are. :-)

September 15, 2010 2:37 PM
 

Felipe R Machado said:

Hi Gary, you must be more precise when stating you've studied ju-jitsu for many years. Currently there isn't a single Koryu Ju-Jutsu school, outside Japan, that you could have studied to be able to have the knowledge to refute the (very simplified) origins of Judo I've posted, which is, although simplified, historically accurate. So, in which school of ju-jutsu you studied under? This can bring some light into why you "refuted" my assertion. I'm not exactly without knowledge of the many traditional Japanese ju-jutsu schools, their ramifications, masters, the lineage of modern Aikido and Judo, and even Brazilian Jiu-Jitsu, which I can follow very closely since I live in Rio. Since this is completely off-topic you could reply to me via email, I will love to talk about ju-jutsu (and maybe Japanese) history with you, or you can follow me on the judoforum.com forum, my nick name there (and almost everywhere) is "loudenvier".

On an "on-topic" note, after re-reading my post I found it may have sounded that I disliked your article, which is far from the truth. I can pretty much understand and visualize most of the things you talked about. I too don't think testing is any substitute for defensive programming, just like airbags are no substitute for defensive driving (wow! here we go with another analogy!)

September 20, 2010 1:59 AM
 

Gary Short (DevExpress) said:

@Felipe I have studied the Hontai Yoshin Ryu school for the best part of 25 years, but that is not at all important. The reason I refute your assertion regarding the origins of Judo was that you stated that Dr. Kano founded Judo to remove the dangerous moves from Ju-Jitsu, when everyone knows this was not his motivation at all. He was, in fact, motivated to find a more competitive element to the art and this was his foundation for Judo.

As you say though, this is totally OT, but please feel free to follow up with me on this fasinating topic at garys@devexpress.com

September 29, 2010 6:32 AM
More from DevExpress
Live Chat
Have a pre-sales question?
Need assistance with your evaluation?
We are here to help.
Chat is one of the many ways you can contact members of the DevExpress Team. We are available Monday-Friday between 8:30am and 5:00pm Pacific Time.
If you need additional product information, require pre-sales assistance, or want help with your order, write to us at info@devexpress.com or call us at
+1 (818) 844-3383.