Tag Archives: Bid Now Sample

De-Normalizing your data

April 13, 2010 7:00 am /

I mentioned yesterday about how we use some trickery to maintain the top views table in Bid Now.

Items are stored in a table, but to generate a quick top views view, we maintain a separate table. This is done using the Task/Queue/Task pattern described yesterday here.

Since TOP isn’t a keyword that Windows Azure table storage deals with, we have to use some trickery.

Items are stored in the AuctionItems table. But when you visit the home page, you get several different lists of items. Each of these items are in fact separate tables, which are kept up to date using the Task/Queue/Task pattern.

image

For the Most Viewed list – we use the MostViewedItems table. The table contains PartitionKey, RowKey, TimeStamp, Title, EndDate, ItemId, PhotoUrl, ShortDescription & ThumbnailUrl. In fact just enough information to display the list and enable a click through to the item details.

The query to return the top 5 items is simple – we simply return the first 5 items from that table – which is super fast. How you say. Well we use the PartitonKey to order the table!

Take a look at the Most Viewed from the home page.

image

Here is the MostViewItems table. Note the partition key is numerical. Every time an item is viewed we read the row, decrement the number in the partition key, save the new row and delete the old one. (since you cannot update the partition key).

image

Note Football is 2nd from the bottom. If we view football, the update works and we end up with this on the home page.

image

And an update to the partition key. Note the number has been decremented by one.

image

You can use this pattern to keep your own data de-normalized and provide super fast queries.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS

Posted in: Windows Azure / Tagged: , , , ,

Web Queue Task Queue Task

April 12, 2010 1:02 pm /

As part of my firestarter talk on building applications from the cloud, I talked about decoupling the tasks worker roles had to perform using Queues.

Here is the basic pattern:

Web Q Task Q Task diagram

This is a pattern you should follow when building new applications and services for the cloud.

Why should you think this way?

There are several reasons:

  1. De-Normalizing data for Windows Azure table storage. If you are using table storage, you could well be using a worker role to keep the data de-normalized. As an example, in the Bid Now sample application there is a view of the data that is the most viewed items. This table is updated by a worker. When you view an item on the site, a message is placed on a queue which instructs the worker to update the data in the most viewed items table.
  2. Scale out. Some tasks take longer and/or more resources than others. Some tasks need to be done quicker than others. As an example, it doesn’t matter how quickly we update the most viewed items table, but we had better update the table containing the winning bid very quickly. Breaking out tasks into different workers allows you to scale up and down in the right place.
  3. Failover. At some point, your worker will effectively restart. (could be hardware failure, OS patching etc.) The more work a worker has to do in any one task, the longer it will take to redo this after a failure. It also has a much greater chance of leaving your system in a mid-way state.
  4. Isolation/Layering. You can easily add improvements and deploy fixes into individual instances if they are separated out without any down time.

This has to be the first rule of building cloud apps, but what about a sample that shows it?

If you look at the Bid Now sample app, you will see that although we have a single worker role – the app is in fact implemented using the strategy above. We use a single worker as I didn’t figure you would want to run 6 worker roles for a sample app!

If you want to walk through an example of this, take a look at the following places in Bid Now:

  • Buy/AuctionDetails.aspx.cs in the project BidNow.Web, line 164-168. If the item is viewed, call service.IncreaseAuctionItemViews, which if you follow the link will take you to
  • AuctionService.cs in the project BidNow.Services, line 266-269. Here we add a message to a queue.
  • ViewItemHandler.cs in the project BidNow.Handlers, starting at line 51, reads the message from the queue and increments it up the list. It performs a delete and an add to do this as the view table is ordered by the partition key (more on this in a later post).

You can grab the latest Bid Now Sample from http://code.msdn.microsoft.com/BidNowSample.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS

Posted in: Windows Azure / Tagged: ,

New Bid Now Sample on Code Gallery

April 8, 2010 4:27 pm /

I’ve just posted the latest version of Bid Now on code gallery at http://code.msdn.microsoft.com/BidNowSample! On code gallery you will find the code, as well as some guidance on how to get Bid Now running on your machine, as well as how you can deploy it to the cloud.

A few days back I presented at the Windows Azure fire starter event here in Redmond. (More on that in a few days when the videos are posted.) This is one of the demos I showed during my talk.

The demo is built on Windows Azure, and uses Windows Azure table and blob storage for data. There are some great things to look at in the code, such as the decoupling of functionality and use of Windows Azure Queues. It is also a great example of how you can build complex data applications using Windows Azure table storage. Think nosql here.

If you look at the homepage you can see there are several “views” of the data. As an example the boots below are shown in both the “Bids ending soon!” and “Hottest” sections. This data is pulled from different Windows Azure tables as all the data has been de-normalized.

image

Over the next few days I’ll be posting more details on how we built this app, some of the do’s and don’ts as well as how you can use it for your own demos/projects.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS

Posted in: Windows Azure / Tagged: , ,