This question on StackOverflow pointed me to autonose. It wasn’t that easy to install using easy_install since one of its dependencies (snakefood) failed to install so I had to do that manually.

Unfortunately, autonose has a few issues, especially when running on Windows. Since it doesn’t appear to be updated anymore, I went searching for an alternative and found pyautotest, part of the Modipyd project. It doesn’t use nose, but that’s OK, because I wasn’t using anything that required it.

I had to install Modipyd by downloading its source from this GitHub repository and running python setup.py install.

Pyautotest is exactly what I was looking for–simple and works right out of the box without a nest of dependencies. The only part I was missing was support for Growl for Windows.

Ian Lewis and his co-workers released some custom test runners that can be used via pyautotest. The Growl version uses the growlnotify tool, but that didn’t work with the version of growlnotify.exe that works on Windows.

I started modifying their runner to work with growlnotify.exe, but it really bothered me how they copied and pasted the entire contents of the unittest.TextTestRunner.run method into their derived class so I threw together my own version which doesn’t contain such a flagrant disregard for object-oriented principles. You can clone/fork it here.

For the icons, I used Jamie Hill’s pass/fail smilies which I found here.

I hope this helps others trying to do TDD with Python. I absolutely love saving in Vim and seeing a green smiley face immediately appear. =)

I used ILMerge to merge the Growl for Windows assemblies into the executable so it’s basically a single file now.

By the way, last night on Twitter, Steve Bohlen pointed me to this port of the original autotest to .NET. I took a look at the code and it was much more complex than I was looking for. It actually builds and runs your tests every time you save which is much more often than I want.

I had AutoRunner turned on all day at work today and was loving how it would catch me breaking the tests when I wasn’t expecting it to. =)

Besides enjoying and learning from his actual performance, I was really impressed by his use of a Ruby tool called autotest. (I’m not sure, but it looks like it has become autospec.)

Not being a Ruby developer, I wanted the same thing for .NET. I did some searching, but my Google-fu failed me so I spent an hour hacking together my own.

The result is called AutoRunner (I know–way creative) and its source is available on GitHub.

If you want it, you have to download the source and compile it yourself for now. (UPDATE: You can now download it here!) Run it from your favorite console (PowerShell, right?) without any arguments to see what options it accepts.

AutoRunner is a little more general purpose than autotest/autospec is. Basically, it can run any executable when any file changes.

What I wanted it for was to run nunit-console.exe whenever my current tests assembly was rebuilt. To do that, I just invoke it with the right arguments.

If you have Growl for Windows running, it will send it a notification which is pure eye candy and not necessary to actually get it to run your tests.

It’s not a Visual Studio add-in. It’s just a plain old console application. Using Visual Studio’s External Tools feature, however, it’s almost as good as an add-in. I set up an external tool with the appropriate arguments and it’s good to go for all of my projects.

To set this up for yourself, you’d create a new external tool with its command set to the path where you built AutoRunner.exe and its arguments set to something like the following (I’ve separated the options on their own lines, but you wouldn’t do that in Visual Studio):

--target $(BinDir)\$(TargetName)$(TargetExt)
--exe C:\path\to\nunit-console.exe
--pass "$(TargetName) FTW!"
--fail "Oh noes! $(TargetName) is FAIL!"

You can use whatever test runner you like, of course. Please note that you must have a file from your tests project open or selected in Solution Explorer when you activate the tool or your AutoRunner instance will be watching the wrong DLL!

It doesn’t support plug-ins the way autotest does and most of its functionality is hard-coded for now. If anybody finds it useful, let me know and maybe we can work on improving it together.

He then went and moved on to a new project called SharpTestsEx, which he intended to be framework-agnostic, but currently only worked with MSTest which prevented me from being able to use it (since I never saw a compelling reason to switch to MSTest).

Fabio was kind enough to let me make the changes necessary to remove the dependency on MSTest. A also made framework-specific versions of SharpTestsEx for MSTest, NUnit, and xUnit. The framework-specific versions aren’t really necessary, but they make the error messages a tiny bit prettier if you use the right one for the test framework you’re using.

You can read Fabio’s announcement here.

I plan on updating my personal project to using SharpTestsEx next.

I just realized I never posted about that project; I’ll have to get around to that soon. If you’re curious, it’s a behaviour-driven development framework for .NET called BehaveN. I’m using it at my work and we’re loving it.

How come nobody told me that TestDriven.NET came with keyboard shortcuts? They have to be manually mapped, but once I did that, I’ve found them indispensible.

• Alt+1 : TestDriven.NET.RunTests
• Alt+2 : TestDriven.NET.RerunWithDefault
• Alt+3 : TestDriven.NET.Debugger
• Alt+4 : TestDriven.NET.RerunWithDebugger

I especially love the two shortcuts that repeat the most recent test run for me. Yeah, I know you can right-click in any file and select Repeat Test Run, but if I want to alternate between running inside or outside the debugger, I have to go find the test again. With my shortcuts, I can just hit Alt+2 or Alt+4 no matter where I am.