A story about mocks:

A couple of weeks ago I got into a minor argument with someone about whether/when one should use mocks vs. trying to actually call the library in question.  My argument was that whenever you use mocks instead of real code, you're reducing code coverage, so if you're going to use them, you'd better have a good reason (such as a library that has an actual external dependency on the network).

So okay, there was not much resolution to that argument, and I was too depressed to bother looking to see if anyone took my advice.  But a few days later I ran into a problem with a program that I had helped write, related to its handling of HTTP redirects.  I looked at the HTTP client test code and... well, it didn't test HTTP redirects at all, which isn't surprising, because really, who tests HTTP redirects.

But it also didn't use real HTTP.  The http "client" was talking to the http "server" using a mock library that left out HTTP altogether.  Which worked okay, but wasn't going to let me add the HTTP redirect tests.

So I converted the thing to use real http client and server classes instead of mocks.  No big deal.  In fact, the resulting program was slightly shorter.  Excellent.  It even passed, leading me to doubt all this advice a bit, since avoiding mocks had failed to benefit me in any way.  Then I added the HTTP redirect test I came here to add, which after all that, was the easy part really.  It passed too!

Great, right?  Well, no, because I hadn't fixed the bug yet.  Sigh.

What I learned was that the main() program was configuring the http client library to use libcurl.  But the class under test was not doing that.  So my new unit tests were testing a totally different http client library.  Oops.  Fixed that, and now the test failed as expected.  Except it failed in the wrong way; the test just froze upon receiving a redirect.  It wasn't supposed to do that, it was supposed to redirect the wrong way (using GET instead of POST, in case you're wondering; we needed it to redirect using POST).

Oh, why is that? To make a long story short, a really weird race condition in libcurl that triggers pretty easily when using epoll, but is harder to trigger otherwise (but which would have made it dangerously error prone if you use threads).  Word to the wise, upgrade to libcurl 7.31 or higher if you use threads or epoll.  (Explanation: it closed the socket, then waited a while, then unregistered for events on the socket.  If someone opened a new socket and got assigned the same fd number before the event unregistration, bad news.)

So what started out as a one-line fix turned into a whole day of screwing around, and my no-longer-mocked unit tests found a bug in a library that wasn't even mine, which I wasn't looking for, and that I didn't really want to know about.  Also, it was probably responsible for the way our client randomly freezes up in the field sometimes, on a small number of devices, with very low probability.  And now it's fixed.

So here's my updated view on mocks: overusing mocks is the lazyperson's backup strategy to not writing tests at all.  If you don't write tests, you don't have to find bugs you never wanted to find.  Eventually, someone shames you into writing tests.  But you want to find bugs in your code, not bugs in someone else's code, so you write mocks.  That saves you time debugging other people's code, which you never wanted to do and isn't how you get promoted and make an impact, right?

But if you want to fix all the weirdo rare bugs in your product, you try to avoid mocks.  Because you do find edge cases, even in other people's code.  And other people's code is part of your program, like it or not.  And clearly their own tests didn't find all the bugs.

Epilogue:

I checked in my fix (which was upgrading to libcurl 7.32, of course).  It broke SSL and I didn't notice.  I hate computers.