As part of my firestarter talk on building applications from the cloud, I talked about decoupling the tasks worker roles had to perform using Queues.

Here is the basic pattern:

Web Q Task Q Task diagram

This is a pattern you should follow when building new applications and services for the cloud.

Why should you think this way?

There are several reasons:

  1. De-Normalizing data for Windows Azure table storage. If you are using table storage, you could well be using a worker role to keep the data de-normalized. As an example, in the Bid Now sample application there is a view of the data that is the most viewed items. This table is updated by a worker. When you view an item on the site, a message is placed on a queue which instructs the worker to update the data in the most viewed items table.
  2. Scale out. Some tasks take longer and/or more resources than others. Some tasks need to be done quicker than others. As an example, it doesn’t matter how quickly we update the most viewed items table, but we had better update the table containing the winning bid very quickly. Breaking out tasks into different workers allows you to scale up and down in the right place.
  3. Failover. At some point, your worker will effectively restart. (could be hardware failure, OS patching etc.) The more work a worker has to do in any one task, the longer it will take to redo this after a failure. It also has a much greater chance of leaving your system in a mid-way state.
  4. Isolation/Layering. You can easily add improvements and deploy fixes into individual instances if they are separated out without any down time.

This has to be the first rule of building cloud apps, but what about a sample that shows it?

If you look at the Bid Now sample app, you will see that although we have a single worker role – the app is in fact implemented using the strategy above. We use a single worker as I didn’t figure you would want to run 6 worker roles for a sample app!

If you want to walk through an example of this, take a look at the following places in Bid Now:

  • Buy/AuctionDetails.aspx.cs in the project BidNow.Web, line 164-168. If the item is viewed, call service.IncreaseAuctionItemViews, which if you follow the link will take you to
  • AuctionService.cs in the project BidNow.Services, line 266-269. Here we add a message to a queue.
  • ViewItemHandler.cs in the project BidNow.Handlers, starting at line 51, reads the message from the queue and increments it up the list. It performs a delete and an add to do this as the view table is ordered by the partition key (more on this in a later post).

You can grab the latest Bid Now Sample from http://code.msdn.microsoft.com/BidNowSample.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS