CThru and TypeMock.Internal.Hooks

I've been playing with CThru trying to understand how it can be used in our tests. There is one part of it that seems to me both powerful and limiting. CThru engine is based on TypeMock internal hooks, and its ability to intercept method calls depends on TMDuringCallEventArgs. This event (and it base class HookEventArgsBase) has constructor that take "typename" and "methodname" parameters. So if you want to intercept certain calls you must be able to filter them by respective method and type names.

At first glance this looks sufficient. However it is quite common when there is a need to set interception on base class level: a WCF service contract interface, an abstract class etc. And if the method name is too generic (e.g. "Name" property), it looks like it's impossible to intercept such calls. How can I intercept IDataReader.Read()?

So my question is if it's possible to build a CThru-alike engine so it can be hooked into method calls with filters that can use rich information available via reflection, e.g. when class is a subclass of a certain class or implements certain interface. Such ability would open for more generalized approach when setting filters for call interception.

Uses cases for a new user

Hi folks.

We want to rethink the ease of use features we talked about in previous blog posts. first we need to think about the Person and use cases we want to address.

Persona:

• Name: Suzy Rockshire
• Age: 30
• Occupation: Team Lead\project lead
• Team size: 3
• Place of work: large university computer dep.
• Experience: Started in VB 6, moved on to .NET and C#, lots of work with classic ASP and ASP.NET. Sharepoint webparts. Standard Microsoft developer.
• Unit testing experience: very little. mostly from hearing. thinks it’s a good idea, has no time to try it with her busy schedule. She finally foudn some time to install isolator. she has a free our. No TDD experience.

 

Possible scenarios where we can make things easier for her:

1. After she has finished running the installer.
2. The first time she opens visual studio after installing isolator
3. When she decides to she wants to see what Isolator looks and feels like (visual studio already open)
4. When she is writing a simple NUnit or MSTest unit test (no Isolator in it)
5. When she wants to create a new test from an existing piece of code
6. When she wants to write a sharepoint test
7. when she first runs the test she wrote and it succeeds
8. when the test she wrote fails
9. when she uses Isolator’s syntax wrongly
10. when a test using isolator has failed

which scenario would you choose first to show her the most value as quickly as possible if she has never used the product before?

are missing we missing scenarios where we could be of help?

Very Late News: Our New Site is Up!

Yeah, it's been more than a week already, but it is still worth mentioning. Our new site is up!

We're doing it like in our product - incrementally. So we're adding pages on a weekly basis. So expect more pages to display the new look, and more information. For example, we have our ASP.Net page, along with articles from Artem Smirnov on unit testing ASP.Net with Ivonna, and Vagif Abilov on mocking and isolation in unit-testing..

Cool stuff. And we're continuing both on adding content and modifying the looks. And obviously, we'll be happy to get feedback and suggestions. Let us know what you think, and what you'd like to see more of!

Need your help: Ease of use for new Isolator users

Hi gang.

We’ve been considering features that will enable out of the box ease of use for the first time Isolator developer. The big question is -

“How do you make that first time after-installation experience usable, discoverable, and actually make the devloper use and “grok” Isolator in 10 minutes or less. just to get the essence of it. The rest can be then handled by docs and forums and blogs.

Here are the top five suggestions we came up with and we need to choose one to start with that will bring the most value:

1. “Fake this” : Be able to select a piece of production code in the code editor (usually a line or two) and then right click and select a new menu item “Fake this”. This will generate Isolator code that fakes that specific call(s). Then put it in a new test or put in the clipboard.
2. “Generate test from Breakpoint” : you are in a debugging session and you arrive at the method where a bug exists. You right click in the method during debugging and select “Recreate this scenario using Isolator” or something of that nature. You will get an isolator test that has all the code needed to get to that piece of code with all the needed parameters and the system state in that matches what the code in the method exists. the test also invokes the method (alternatively – it invokes the first public method in the call stack that leads to this scenario)
3. “Quick Fix for tests”. This will use resharper style quick fixes for your tests for test conventions as well as isolator conventions (like “you’re better of faking a static method here”)
4. “Dependency walkthrough” (we need a much better name for this). The developer writes a standard unit tests calling the APIs they need, which could may well be an integration test but they don’t know it yet. Then right click on the test and select “analyze and generate dependency behavior”. This makes isolator repeatedly run the test in question. everytime isolator catches an exception, it will:
   1. show the current callstack to the developer and highlight the last call that threw the exception with the parameters sent to it
   2. allow developer to decide on one piece of the callstack what it’s fake behavior will be/ (for example – they will select that call and right click it, and select “ignore call”
   3. After this isolator re-runs the test with the new behavior saved in memory. (go to step 1)
   4. if the test finishes OK, the developer can then “save” the behavior they defined in the current test (the code will be generated)
5. “Debugger fake notification” : When running an isolator test, during debugging, everytime you debug through a faked method a popup icon will say “you just stepped over a fake method” with some details about the method. maybe this only happens the 2nd time they debug through that method.. we don't’ know yet.

• If you were to choose just one of these, which do you think would add the most value to new users? (please answer the poll below)
• Can you think of other features which could be even more valuable?  (feel free to add comments!)

The poll below is in Flash. If you can't see it, here is a direct link to the poll page.

 

Version 5.3 release

Hi All,

As promised, albiet with slight delay, we released version 5.3 today. The major features are:

• Exact argument matching when setting behavior
• A new DoInstead() completing statement allows setting dynamic behavior
• Better control of constructor behavior when creating fakes
• Solves many issues with generics support (for "object" and others...)

This is the first version we've released while using this blog as a sounding stage for our ideas, suggestions and plans. Your input has been invaluable - thank you all!

Please let us know of any feedback on this version. I'll continue updating this blog with our feature plan for upcoming versions.

Gathering Resources

I want to do a couple of things regarding blogs and resources. First, everyone who has a blog and wants it on our blog-roll (which I added on our blog here), please leave your blog address in the comments. I'll update the blog list accordingly.

I also would like to start accumulating a list of resources where you guys get your information. This could be blogs, articles, whatever you think is relevant to this community, and anyone interested in unit testing.

Thanks in advance!

Version 5.3 is postponed to next week

Now, I know we promised to release these features this week, but we didn't make it. Gearing up for release we found a couple of issues and took the decision to postpone it so we'll have better confidence in the version's quality.

We will work to release during next week - sorry about the delay :(

The New Typemock Slogan

Well, we've got a winner, and you heard it hear first:

"Unit Testing, Plain Smart"

We're launching the new site on Sunday, and the slogan will be right up there. I want to thank the people who participated.

What are you missing today in Isolator

I'd like to start a small tradition, which I intend to do every so often. I'm going to ask you what you feel is most needed today in the product.

It doesn't need to be a feature, it can be a bug, a documentaton, something on the missing on the site. Everything that annoys you, prevents you from doing your job, or even explaining what Isolator does to someone new.

The more votes (or "me too!"'s, or "I second that", or any other way you can express empathy:)) we'll give it the greates consideration. For example, Travis raised the test attribute issue, with two more people on the same week. Must be a sign, and we've already started thinking about a solution.

By the way, you don't need to wait for me to ask you. This is the place to post your request, and what you think will make our products better. But I will come back to ask once in a while to see what's missing.

So sound off. What's missing for you today?  

Test Attributes and Leaking Expectations

I noticed this post over on the forums talking about "protecting the developer from him/herself" with respect to test attributes and mock setup/cleanup.

One of the biggest issues I've found when working through tests using Isolator is that if a developer forgets to mark a test with [Isolated] (in AAA) or [VerifyMocks]/[ClearMocks] (in Natural/Reflective), expectations can leak from one test to the next, causing tests to fail. It's really tricky to debug these issues because the tests, by themselves, will pass... but when run in context of the suite, will fail.

I've personally spent hours on fixing up other peoples' tests because things that were passing will start to fail and it's because some test that got checked in a month ago forgot the attribute and suddenly something in a new test caused a leaked expectation to fail.

Just as bad is the confusion folks see in using the [Isolated] or [VerifyMocks] attributes - they don't know which to use, so they'll mix and match, sometimes even using both attributes at the same time.

All of this adds up to frustrated developers who rail on Isolator as the tool at fault. "Other frameworks don't require this," I'll hear, or, "Why does it have to be so confusing?" It sometimes surprises me that folks who can figure out high-level cryptography and complex federated authentication scenarios can't figure out when to use a particular attribute, but that's another story.

What it boils down to is that it's a pain point for some and it'd be nice not to have to use them, or to have some sort of mechanism that programmatically checks for these things so folks who forget can get some automated help and save me the time of fixing their tests and/or re-justifying why Isolator is so vastly better than these "other frameworks."

New site pages: ASP.NET, Silverlight Unit Testing

We have uploaded 3 new Learn pages to our site.

2 pages for ASP.NET unit testing:

http://www.typemock.com/ASP.NET_unit_testing_page.php

http://www.typemock.com/ASP.NET_unit_testing_Article.php

And a page for Silverlight unit testing with SilverUnit.

http://www.typemock.com/Silverlight_unit_testing_page.php


A big thanks goes to Artem Smirnov, creator of Ivonna, for writing the ASP.NET unit testing article.

Moran

Unit Testing Articles

Hello

First let me introduce myself, my name is Moran, and I am from Typemock's marketing team. You've probably seen the various sections in our Learn page, well, we want to add more pages for different verticals to our Learn section.

We're looking to expand the unit-testing information on our Learn section and we thought of asking if you can help us out with articles (and/or code examples). Of course, you'll get full credit, a mention in our Insider hall of fame (we're working on that too), and we'll link to your site / blog, so our visitors can get more info on your site as well. More information on unit-testing means everyone wins.

If you're up to the challenge, please leave a comment with the topic you are interested in writing about, or drop me an email .

Looking forward to hearing from you,

Best regards,

Moran

Marketing Department

ReturnRecursiveFakes should/ should not fake private members?

Do you think ReturnRecursiveFakes should or should not fake private members? Or, do you think that this feature is important?

Personally I think that this is important, especially when I am dealing with legacy code that contains a lot of private members which I don't really care when I am unit testing my code. I think that private members should definitely be faked, and that should be high on the priority list. 

Any thoughts?

test 123 please ignore

Typemock and NerdDinner Tests

If you’ve tried to run the tests in the new ASP.NET MVC NerdDinner sample, with Typemock enabled, some of you may have received exceptions, which would disappear when Typemock was disabled.(Several Generics related bugs)

In the latest version (coming out this week) this seems to be fixed. The latest nightly build can be gotten directly from here (download x86 , download x64).

Please let us know if you have any issues.

PS

This version already includes .WithExactArguments"() as well as partial Do() support.

Count number of times method is executed

Most mocking API's have the ability to do verifications against the number of times a method was executed, it'd be nice to have this available within the Typemock AAA mocker.

I know that Isolator must have some kind of tracker of method call counting, my extensions (for those who haven't seen them - http://www.aaron-powell.com/blog/march-2009/a-nifty-typemock-extension-on-steroids.aspx) indicate that Isolator has taken into account the number of times it was called.

Something like this would be handy:

Isolate.Verify.WasCalledWithAnyArguments(() => fake.SomeMethod()).NumberOfTimes(5);

Sandbox's RSS not working (at least in my Google Reader)

Hi, am I the only one who hasn't received the latest three posts? I'm using Google Reader, not sure if it's ok with other readers.

A quick view of CThru

here’s a quick walkthrough of CThru – based on Typemock Open AOP I made. Still in rough stage, but you’ll get the idea.

here’s a link to the video if you can’t see anything.

Follow Up: Our Suggested API for Checking Arguments

Vagif's blog post and ensuing discussion online did not go unnoticed in the office, and one of the first things we did this week was to discuss possible API designs for this feature. Basically, we are trying to do two things:

• Provide the user with an intuitive way to say "I want to match all values I passed into WhenCalled()" - this means the behavior set after WhenCalled() will happen only if all arguments passed to the actual call exactly match those passed to WhenCalled(). We feel this is what most users want and need.
• Provide the user with the ability to define customized arguments matching - this means the behavior set after WhenCalled() will only happen if the custom argument matching evaluated to be true.

We brainstormed several ideas and suggestions around the office, and came up with the following API:

Exact argument matching: Return 1 only if the arguments passed to Foo() exactly match those passed to WhenCalled() (i.e are "hello", "world" and 9)	Isolate.WhenCalled(() => fake.Foo("hello", "world", 9)). WithExactArguments(). WillReturn(1);
Custom argument matching: Return 1 only if the arguments passed to Foo() behave as follows: - the first string contains the world "hell" - the integer value is between 7 and 10 - ignore the second string	Isolate.WhenCalled((string s, int i) => fake.Foo(s, "", i)). AndArgumentsMatch((s, i) => s.Contains("hell") && i <= 10 && i >= 7). WillReturn(1);
Custom and exact argument matching: Return 1 only if the first string contains "hell", other arguments match exactly to those passed to WhenCalled()	Isolate.WhenCalled((string s) => fake.Foo(s, "world", 9)). WithExactArguments(). AndArgumentsMatch(s => s.Contains("hell")). WillReturn(1);

As you can see we have two new methods following WhenCalled(), and one change to WhenCalled() itself:

• WithExactArguments() - this method changes the behavior of WhenCalled() from the default of disregarding arguments to exactly matching arguments
• WhenCalled() will be changed to have overloads that receive Func<Res, T1,..,Tn> (and Action<T1,..,Tn>). This way the user can pass in typed parameters - this borrows heavily from Vagif's suggestion. In our example these are (string s, int i), which replace the arguments passed to the method in WhenCalled().
• AndArgumentsMatch() - this method receives a Func<bool, T1..Tn>, meaning it receives the same typed parameters WhenCalled() received. In this delegate the user can implement any argument check he needs on these parameters; in our case this is (s, i) => s.Contains("hell") && i >= 7 && i <= 10.



• The two extensions may be used together i.e WhenCalled().WithExactArguments().AndArgumentsMatch() - the use case here is exactly matching all arguments to the values passed in WhenCalled(), except those passed in as typed parameters which are handled in the custom matching in AndArgumentsMatch()

I think this API builds on the existing one without changing its default behavior and breaking older tests, and adds the capability to behave conditionally. What do you think? Your feedback and suggestions are highly appreciated.

My two cents about forthcoming WithExactArguments syntax

Like many others I am waiting for a possibility to set exact match on fake method's arguments. So for me this is a good news. However I can't say the way it is implemented looks perfect. 

First, indexers where exact match has been supported for a while will be treated differently from regular method calls: if you have an indexer in your class, then depending on whether you access some value using square brackets or parenthesis, the arguments will be matched exactly or disregarded respectively. 

Second, you can choose between exact match on all argments or no match at all. What about partial match?

And third, I believe adding few more overloads of WhenCalled can eliminate the need of WithExactArguments.

I spent some time playing with the code trying to imagine how extended syntax might look. It resulted in a blog post.

Feel free to criticize :-) But the more I think about notation to distinguish between matched and unmatched arguments, the more it looks to me that using lambda expressions is a way to go.

Vagif Abilov a.k.a object

Our forum

We've run our forum as PHPbb 2. We're thinking of upgrading to 3. What we would like is something like Stack Overflow, but we'll settle for a forum that at least let's you edit and present code normally.

Do you have any suggestions, based on your experience with other forums?

Install-less Deployment

We’re looking into finally solving the auto-deploy problem Typemock has.

Here’s a quick tool : TMDeploy.exe that you can download. Just put it where TMockRunner.exe resides and run it before running the tests.

Try to run it in both 64 bit and 32 bit machines. It will need admin rights, because it writes to the registry and GAC.

Let us know if you had a problem. (royo@typemock.com)

remember – this is a “works on my machine” binary. do not share it. just try it at your own risk.

Roy

quick look at upcoming features for next week

Here are the major features between 5.2.3 and the next version)should be out sometime next week). Your feedback is welcome

- New API that enables the user to run a custom method when a method is called:

• Isolate.WhenCalled(() => fake.Something()).DoInstead(CallContext x => …;);

- New API that enables the user to run a custom method when swapping a future instance:

• Isolate.Swap.NextInstance<Logger>().With(fakeLogger).SwapConstructorWith(x => …;);

- New API that enables setting behavior on a method only when encountering an exact argument match in the code under test:

• Isolate.WhenCalled(() => fake.Foo(“”, 1)).WithExactArguments().WillReturn(5);

- Users can now create a fake object with CallOriginal without invoking the original c’tor

• Isolate.Fake.Instance<Logger>(Members.CallOriginal, Constructor.WillBeIgnored);

Welcome!

Hi everyone!

If you can read this, this means you're also an author on this blog. So don't hesitate to post here.

We'll do our part in order to show you what we have in plans, discuss our pains and solutions and get your opinions. 

We count on you to help shape Typemock products, so please don't hold back.