Bug hunting in the public eye

A few days ago, ayende put up a post showing a bug that had been found in RavenDb that was causing the profiling tools to enter an infinite loop.

http://ayende.com/blog/152738/bug-hunt-what-made-this-blog-slow

This was followed by a post about how this bug had come into being and survived so long.

http://ayende.com/blog/153825/ask-ayende-what-about-the-qa-env

So confession time, that’s my bug. I contributed the early code for the profiler.

So since then I have been trying to work out how I missed it. Was I really so oblivious to what was going on while writing that code?

Second to that, how had it sat in production on ayende’s blog for 6 months or so without being noticed?

I am glad to say I have an explanation for both, which I will share now.

First some background on how the profiling actually works.

When you attach your DocumentStore to the profile, the following code is run

Basically we attach to the OnSessionCreated event and stuff a header into the response with the session id.

This is very important, and the key to why this bug was missed. So I will say it again, the X-RavenDb-Profiling-Id header is only added to your response if you actually open a session.

So lets look at that JavaScript again.

It only calls back to the server when it gets an AJAX response that has the header set. So if you make an AJAX request to your server for data that does not come from RavenDb, the profiler will not pay any attention to that request.

The request to get the profiling results does not need a session, and in all my development it did not create a session. So it never set the header. So it never entered a loop.

So that explained to me how I had missed the bug, the way I deal with session management (creating and disposing of them only as needed) simply does not trigger the bug.

So with my conscious cleared, I moved on to the second problem. Just because I handle my session this way, clearly RacoonBlog does not, surely that bug hasn’t been there since ayende turned on the profiler for his blog.

Well let me present a commit to the RacoonBlog source from the very end of December.

https://github.com/ayende/RaccoonBlog/commit/b284efae61108af991221d948eb891e1310bc64b

In this commit, Racoon switched from creating sessions only as they are needed (my way of handling them) to creating the session in the BeginRequest event of *every* request. So even requests that don’t use the session still create one. Which obviously adds the profiling header and tells the JavaScript “this request hit RavenDb, so you should fetch it’s profiling results”.

So until that commit was pushed live RacoonBlog, like my own sites, simply did not trigger this bug, it was sitting there dormant just waiting for someone to take a different approach to session management.

So there we have it. I really don’t feel so bad about it now.

To a certain degree the JavaScript was actually correct. It noticed an AJAX request that said it had profiling information attached and it fetched that information. The fallacy in this argument though is that the JavaScript also has enough information available (the request URL) to know that it is being lied to. It should have used that information to avoid the trap that was laid for it. So it’s still a bug for all that I wish it were not.

Experiments in text adventure

Like many who were into computers “back in the day” I still maintain a nostalgic, warm and fuzzy feeling at the memory of early text based adventure games.

Back when all I knew were the PRINT and INPUT commands in QBASIC, these were the very first programs I created.

Where am I going with this?

Well over the weekend I was learning the Impress.js presentation library and thought to myself that it really was a beautiful way to display text in a web browser. Then a little light bulb went off in my head.

3 hours later, I had forked impress.js and used it’s HTML5 canvas and CSS3 animations to create an engine for a modern text adventure game.

You can see the results of that intense session of programming here http://csainty.github.com/ImpressGame

I have quite a few other ideas for what I could do with this underlying concept. If it eventually comes together as I see it, it will be beautiful.

Getting deeper into Open Source

It’s about 8 months since my first contribution to an open source project. It was followed up minutes later by one to reverse all the accidental formatting changes I made. Damn you Visual Studio!

I am happy to say that it was not an isolated contribution though.

Now I am by no means a prolific contributor and my spare time is more often spent tinkering with my own ideas and projects rather than working hard on open source. However, there has been a steady enough flow of pull requests and projects to keep me satisfied that I am at least involved.

RavenDb and Nancy are where most of my focus has been. Two great examples of modern C# open source development.

In my own right there have been RavenDb plugins for Glimpse and MvcMiniProfiler.

Straying from C# there is even a start being made on a Node.js client for RavenDb. Getting started with Node has been an absolute blast, and my first serious attempt at adding a new language to my skill set since C#. Sure I’ve dabbled with jQuery heavy javascript in the past to add a little life to a web page, but truly learning the object model and scoping rules of javascript is a different beast entirely.

Most recently, as in last week, I have been pitching in at Code52, a new idea for this year which tackles a new project every week. It has been very .NET focussed so far, but that is simply a product of the founding core of developers having .NET backgrounds. At the end of the day, the people who are in the chat room at the start of each project are the ones who decide the technology stack. I hope to get them doing a Node project at some point this year.

The project from last week was IdeaStrike a UserVoice inspired (clone?) site that you can simply deploy to AppHarbor and manage yourself to track ideas and gather feedback about your product. You can see it in action at http://ideastrike.apphb.com/.

It uses Nancy and Entity Framework. I wouldn’t call it a best practice example of either at the moment, it is tough to refactor a project that goes from start to finish in a week. It is however a good example of a non-trivial application, with (some) tests, in Nancy.

Even though the week is up, the project does not stop there. People are always welcome to fork, change, commit and send through a pull request.

A full list of previous projects, that will update over the year, can be found at http://code52.org/projects.html.

This week the focus is a cross-platform multiplayer game. Much further out of my comfort zone, but a great chance to learn some XNA, and eventually Android and iOS when those clients are added.

Now this is a fairly reflective post, not my usual style, but the point is really just to spread the word. There are lots of interesting projects around, big and small, in almost any language you care to use. Just pick one and jump in.

Running a NodeJS server alongside your NodeJS desktop app

The node-webkit project added a new feature yesterday, one which I requested. So I feel I should put up a quick post about it and why I think it could be useful.

The change was simple enough, giving you the ability to pass a callback to node-webkit that is called when the application is closing.

The primary reason I wanted this was to allow you to create, and then clean up, a local web server within the lifetime of your application.

It was previously possible to create the server, but when you closed the app, the process would stay open with the webserver waiting for requests.

The callback looks something like this.

So why exactly is this useful? Well I have a couple of uses in mind.

1. Some libraries and UI controls have only been coded to fetch their data with an HTTP request. This gives you a simple way of using such a library or control. You just point it at the local server and handle the request.
2. If you did all data access across the HTTP layer, then you remove one more piece of uncommon code between a web and desktop application, now the difference between the two is just the URL it is pointing at.
3. It allows you to pre-process previously static files. This means you could compile your LESS or CoffeeScript files as they are being served. It also means you could serve HTML from a ViewEngine such as Jade instead of from static files on disk.

Further thoughts on NodeJS desktop applications

I have been working on a more complete demo of creating a desktop app with NodeJS.

This one will use a full UI (KendoUI) and a Sqlite database to create a basic CRUD style app based on the Northwind dataset. Someone really needs to put together another (and actually well designed) standard database that people can use for apps like this.

While putting this together I have been considering reasons why you might actually write an application like this.

Something that dawned on me is that I am only really using Node in a very thin data layer. My Views and View Models are all standard HTML/JS. Which means the only thing I would need to do to switch this application I am writing to being on the web would be to replace the data layer which is currently using Node to access Sqlite with one that goes across the network to a REST service.

This means I am effectively writing an identical web and desktop app at the same time. Which opens up some interesting offline capabilities.

Add a build time dependency injector to switch between the Node and REST data layers, and a synchronisation mechanism (assuming you want the user to be able to use either) and you have the web and desktop versions of your app running the exact same codebase.

Now that is pretty darn interesting.