Feedback requested: Test Review Tool

Hi folks.

We are always considering new ideas that will make unit testing, better unit testing, easier and safer for developers. One such tool we are working on as a proof of concept is a test review tool that will analyze your tests and give guidance and automated fixing ability on issues found in tests.

please do not share the following with anyone else.

the good news is that we have something you can play with that actually works with 7 “common” issues to find in unit tests. Currently, it is based on a tool called “CodeRight” and it is a set of custom rules for that tool.

to get started playing with this thing you need vs 2008.

download this document for full details on how to download and install the tool and custom rules for it.

Introduction: CodeIt Right for Tests

i need to know if you’d want to buy such a tool, or what you’d want to see added or changed that would make you buy it.

Please email royo@typemock.com with any questions, thoughts or ideas. We’ll looking to you to make this thing a reality if you believe it has merit.

CodeItRight for test review is like having an expert unit testing instructor sit next to you while you code. At any time you can ask CodeTitRight for a test review, and it can detect (for now) 7 common unit testing problems.

What problems can it find?

• 1) Missing asserts in a test (if the test is not names to expect an exception)
• 2) Calling a test from another test
• 3) Integration tests in a non integration project (using file system, registry access, random, threads, date and others)
• 4) Logic inside tests (if, loops, switches)
• 5) Multiple asserts against different objects in a single test
• 6) Usage of multiple mock objects to verify (instead of one mock and others as stubs)
• 7) Using random numbers in tests

Installation:

• · Install COdeItRight by downloading from http://api.typemock.com/files/CodeItRight/CodeIt.Right_v1.8.zip
• · Download http://api.typemock.com/files/CodeItRight/SubMain.CodeItRight.Rules.TEST.20091116.rar
• · unzip the zip files
• · put SubMain.CodeItRight.Rules.Test.dll at : C:\Program Files (x86)\SubMain\CodeIt.Right\Rules\ (remove "(x86)" for 32 bit systems)
• · put Test Profile.profile.xml at C:\Program Files (x86)\SubMain\CodeIt.Right\Rules\ (remove "(x86)" for 32 bit systems)
• · load visual studio
• · you should see a "codeitright" menu item as well as a combo box selecting which "profile" to use.
• · select the "Test" profile

Using it

Note: currently, as this is a proof of concept: it ONLY works with NUNIT

Right click on a test file or click on the “start analysis” button:

You will then see the results:

Some rules have a “correct” action or more associated with them. You can see those actions by either right clicking on the rules and choosing “correct” or opening the combo in the rule line:

CURRENTLY: FIXES DO NOT WORK. They are for demo purpose only to show possible ideas for correction.