Elementary, my dear Watson

ctodx
25 August 2007

We are constantly striving to improve the quality of our software, be it the controls, the frameworks, or the IDE tools. Obviously we do so by writing test cases and continually running them, and the support team are instrumental in identifying bugs that our customers run across. The issue tends to be that customers use our controls in ways we never thought of; tsk, tsk, them wacky customers, eh?

Sometimes though we even manage to identify and fix bugs after release but before our customers report them. ESP? No, just Watson.

I'm sure you've run into the dialog: you're working away when suddenly, bam, the program you're using crashes and the operating system pops up this dialog telling you that there's been some problem, and would you like to send the diagnostic information about this problem to Microsoft? This dialog is known as Watson, after the sidekick to Sherlock Holmes (and in fact the narrator of the stories).

I wonder, what do you do? Do you automatically click the Don't Send button? When I was at Microsoft, I got into the habit of clicking Send instead. Why? Because the dialog is not sending Microsoft intimate details about your setup as many would lead you to believe, but instead is sending details of the crash that just occurred.

What happens to this data? It's collected and analyzed. Watson was originally developed by the Office team for Microsoft Office and proved invaluable at enabling the team to identify bugs seen out "in the wild" and then to fix them. Watson was so successful at finding intractable and hard-to-find bugs that the subsystem was appropriated by the Windows team and is now part of the operating system.

The thing is Watson does not differentiate between crashes that occur within Windows or Microsoft products; it's a collector of all crashes, including programs that you or I would write. And Microsoft makes this data from all the crashes over all the world available to third parties as well.

A third-party software company, like, say, Developer Express, can register with the Watson team to receive the crash information that pertain to its EXEs and DLLs.

This happened just recently in fact. We released version 2007.2 of DXperience and within a day, we had a seven or eight incidents of the same event. The Watson data we get indicates the DLL inside which the crash occurred (obviously one of ours), the stack trace, and the address at which the crash occurred. It was the matter of a couple of hours for us to set up the debug information, and then work out at which line the crash occurred. Using that and the stack trace information we were able to work out that the crash occurred in a finalizer: we were assuming that an object was non-null when in fact it was null due to finalization. The fix was easy to do and appeared in the next version.

I cannot stress this enough though. Because seven people did, in fact, click Send meant that we were notified of the crash and were able to fix it. Without the Watson dialog and crash data collection, we would not have seen this problem as quickly, and, because it was in a finalizer, it would have been extremely difficult to reproduce. We can also provide some help text as well for the crash, so that someone who clicks Send can also be notified that, for example, there is a release available that will fix the problem.

The Watson data is always worth sending to Microsoft. By doing so, you are ensuring that companies like ours will improve the quality of their software. So please click Send.

9 comment(s)
Jan Rosendahl
Jan Rosendahl

We are currently planning to do the same, but have not found the link to sign up yet. Searching for "error reporting" on the MS site yields a lot of broken links :-(

Since all of us customers could benefit too, please post the link where you can sign up.

26 August, 2007
Anonymous
Ahto T.

I usually press Send if I think that the crash info of this event will be useful (like I do often when Flash crashes IE7). However, many times Watson tells me in exchange that this is a known problem and more informaiton can be found "here", but the link that is provided usually does not display any meaningful web page, usually some Microsoft page with generic information about somehting else. I guess that can stop many people sending crash input.

26 August, 2007
Anonymous
Daniel Danilin

We are planning to use Watson in our program. And we are already sign up.

But I don't find any information of how to send a stack trace. Do we need to change something in our program?

26 August, 2007
Anonymous
Tracy McClarnon

Here is the information you need to get started:

www.microsoft.com/.../StartWER.mspx

26 August, 2007
Ben Hayat
Ben Hayat

Julian, that was a very helpful info to share with us. I had no ides that third parties can sign up to Watson data.

Thanks for sharing!

26 August, 2007
Anonymous
Caesar Frederick Qüeb Montejo

I alway do a click in the Send button. Specially in the latest days that i working using Visual Soursafe.... this applications causes that VS2005 crashes. Others times i have fixed the issues, thanks to the link that Watson supply to me.

Thanks for the info

Regards

26 August, 2007
Rollie Claro
Rollie Claro

i must admit, when my boss encounters errors that has send, he clicks it in an unlimited fashion.

but i think its good that we can sign up

26 August, 2007
Anonymous
David Gardiner

I'm curious as to whether you get information from Microsoft about crashes that are indirectly caused by your products?

Say that one of your Visual Studio addins causes VS to crash, I suspect the main dump would be tagged as VS.NET rather than your DLL.

Do you still end up with the dump info after MS triage it?

-dave

29 August, 2007
drew..
drew..

Aweseome post Julian.. funny how our habits form and become second-nature... i was a staunch "Do Not Send" kinda guy.. i will definitely rethink this now. While i don't have the tech skillset to sign up an use this info, it is certainly good to know.. cheers, drew..

29 August, 2007

Please login or register to post comments.