Bug Hunt
Written by Colin Guthrie ,
When developing a Platform like ours, it's not uncommon to find (and squish!) bugs. However, when dealing with a web environment, it's often hard to nail down exactly where a given bug actually lives.
Recently, I happened to notice a problem when uploading photographs to our system. Uploading the first image was fine, but the second image would steadfastly refuse to leave my local machine. What was surprising about this development is that the image uploading widget was relatively new and had not long ago been extensively tested in various different browsers, so I was somewhat surprised to find this problem.
So standard bug fighting protocol was set in motion: I tried in a different browser. Surprisingly IE 8 and 9 (not exactly known for their ability to make weird issues any better) both still worked fine for me. Chrome also seemed not to exhibit the broken behaviour. It must be my beloved Firefox that was to blame! As I had not encountered this problem while developing the widget, I presumed a recent update to Firefox had introduced the problem. I therefore booted up an old Virtual Machine testing system with a much older version of Firefox installed. Strangely, it also exhibited problem, and thus the plot thickened.
I went back to the other primary tool in the bug fighters armoury: I searched for the error message in Google! The error I received was: “Security Manager vetoed action arg 0 [nsIDOMHTMLDivElement.contains]”. Now, if truth be told, searching for this in Google this was actually the first thing I did (maybe I'm getting lazy in my old age?), but as it hadn't previously thrown up any obvious results back then I needed to venture deeper the second time round (i.e. I actually looked a bit harder at the results!).
In the end I found this bug on an HTML 5 upload widget project. It seemed very similar. In this case, the “cause” was found to be in a jQuery UI update. We use jQuery UI in our project and we had updated it not that long ago. With a quick downgrade of the relevant code, I confirmed that our widget started working again! Excellent. I could see light at the end of the bug hunt tunnel. I could almost hear the little critter's six knees knocking together as it quivered in fear of it's impending eradication. But my work was not yet done. I had to find out exactly what change cause the regression.
Finding specific regressions is sometimes a time consuming process, but tools available to us these days really help make the process more painless. In this case I didn't actually use such tools, but years contributing to various Open Source projects meant that I was quite familar with the processes and techniques and I set about bisecting the problem (bisecting is a technique where by you slowly but sure narrow down where a given problem lies by starting with a “good” and a “bad” version and then testing the version in the middle and then throwing away half the changes each time as you decide if the current test point is your new “good” or “bad” version). First I was able to see that the problem occurred sometime between jQuery UI 1.8.15 and 1.8.16. Bisecting further I got down to this commit Effects: Backporting 8108ec8 - Fixes #7595 - Wrapper-creating jquery-ui animations will discard any focus state during the animation. Indeed reverting this change on the latest version of jQuery UI also solved my problem. Wonderful!
But the task of bug hunting does not end there. Oh no! This is just the start. Now we've found what caused the breakage, we now have to understand the why. In this case, the commit helpfully referenced a bug report: Wrapper-creating jquery-ui animations will discard any focus state during the animation. This change solved a real problem! Damn. If it was just a broken change then it could have been reverted. Not so here, so further investigation was still required.
First, I opened a new bug report on jQuery UI's bug tracker detailing what I found so far. I also developed a work-around that didn't revert the initial behaviour, but did avoid the problem for me (even if the focus bug originally fixed resurfaced – which I was quite OK with as I didn't actually need focus – I actually go out of my way to hide that form element!). The jQuery devs quite rightly told me that without a reduced reproduction example, they could not really consider the problem as a bug. This is perfectly fair and while I didn't relish the prospect of cutting up a thousand lines of code to reduce it to the core problem, I set about that task. This is one of the duties of a bug hunter when working with Open Source software. So after a little while of hacking, I managed to create a fairly reduced test case. Once the jQuery developers had a poke at that, they were able to reduce it further and then further still to a case where jQuery wasn't even used at all, just pure JavaScript and DOM methods. Now this was interesting. No jQuery? That means that it's not a bug in jQuery but a bug in Firefox!
So it is my beloved Firefox that is to blame after all - back at square one!! The jQuery UI update exposed it, but it was still there. Some investigation (by the awesome jQuery developers) was able to uncover an existing bug report against Firefox which showed the issue. Sadly no development for over two years, but hopefully with new test cases and appropriate application of a poking finger, we can pique the interest of some Mozilla developers and the bug will be looked at in the near future.
In the short term the jQuery guys will implement some kind of workaround to avoid the buggy behaviour, but really the fix has to be done in Firefox itself too (underpin the foundations, don't wallpaper over the cracks).
This is one of the many things we do for our clients. This process takes time, effort and determination to follow through. This is one of the things you pay for when you use our platform. I've written this article to try and highlight the kind of things we do every day to improve both our platform and the tools we use to build it. This is one of the many reasons I love Open Source. Everyone, no matter how small, can contribute and help out. I'd like to thank our clients for using our products and allowing us to work with the Open Source community in this way. I'd also like to thank all the other companies, groups and individuals out there who are also involved in the Open Source community and from whose work we benefit immensly.