So here's something we started preliminary work on.
Let's say you already have an integration test. This integration test works, it calls the database, a service or any other thing, that replies with results. The test passes, and now you want to create an isolated unit test.
With the current toolset, you start from writing the same code in the test, but then you start working backwards. You run the test, it fails, you add a faking statement. You run the test again, but then it brings a wrong result from your database, so you add a statement to return a fake result, and this is an iterative process. You continue until the test passes, and it can take a lot of time until it works.We want to make it shorter, by a lot.
What we plan to do is record the integration test. By record I mean run it in a certain context (I imagine doing a right click on the integration test, "Generate unit test from this", but it can be in another way) and recording all external calls (which we currently defined as outside of the code-under-test assembly) and what they return. Then we generate code for the faking statements (or the entire unit test code).
We've just started working on that and I'll keep you posted on how we're proceeding. It's one of the ease-of-use features we're thinking of, and in advanced stages can be done just from running your app. I'll be happy to hear what you think.
This is a brilliant idea!
ReplyDeleteI know that many people start from unit tests but I myself prefer to do integration tests first.
However, I'd like to control somehow the mocked boundary. One problem is, how do you define the code-under-test assembly just looking at the test? Second, my initial test could be spanning several classes in the same assembly, in that case there's no "outside".
But I see that even in this basic form it would be very useful.
Another suggestion: your system could not only stub out the external dependencies, but generate a test for each internal class involved in the interaction.
Artem,
ReplyDeleteYou've caught on to some of the early problems. First, the boundary. We've started with external assembly, but in the future I guess in some cases to move the boundary. When we'll get there, I'm sure we'll have a solution.
As to what's the code under test - currently, it's stack based, so what's removed one frame from the test is either a code-under-test. we'll need to find something better, though. Work in progress.
Interesting suggestion about generating a test for the internal classes. If ClassA uses ClassB, as part of the interaction, you can create a test that makes sure that given the input in the interaction, there's a certain output. Cool.
This is a very, very good feature.
ReplyDeleteOne thing I would like is, it would be great if you can generate unit test from when you are debugging an application. The whole process goes like this.
1. A dev launches an application, steps into the code
2. When he steps into method A, he feels that he needs to add a test on this method A.
3. The only problem with method A is that it has too many dependencies ( either via method parameters or the object states) and he can't possibly write every parameters and dependencies down with his own hand.
4. So he clicks on a "generate unit test" option, a unit test that tests method A, along with all the method parameters and object states at that time will be created.
What do you think about this?