Windows Azure Online Log Reader

Jim Nakashima wrote a great blog post on how to get at your Windows Azure logs via the PowerShell powered Cloud Drive. (see http://blogs.msdn.com/jnak/archive/2008/11/12/using-the-clouddrive-sample-to-access-windows-azure-logs.aspx).

I’ve been working on a project lately, and its required some examination of logs. Copying the logs from blob storage is great, but you still have to parse the logs, and if you are working with multiple instances, this can be time consuming.

I decided that I should have a tool that would parse the logs, and display the log output from all the logs. Enter my Windows Azure Online Log Reader. (and you can download the code at the bottom!)

When you need to view your logs, the first step is to navigate to the portal, then issue a copy logs command. This will copy your logs to the blob storage you name in the portal.

If you examine the folder structure created you will see nodes for WebRole and WorkerRole. Within each of these folders you will see a folder for each instance that is running. These instance folders contain the log files for each instance. Last time I copied the files there were 103 log files to process!

My Log reader is a quick and simple implementation – all you need to do to use it is feed in your storage account name, shared key and the container that you copied the logs too. What the reader does is enumerate all the blobs in that path, using the client storage SDK sample library, its a simple to enumerate all the blobs:

BlobContainer logContainer = blobStore.GetBlobContainer(txtSrcFolder.Text);
var blobs = logContainer.ListBlobs("", false);

Next I read all the blobs into memory one by one, loading the contents into an XmlDocument. The xml for a log file looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<ServiceDiagnostics Service="DemoGroup" Role="WebRole" ServiceDeployment="1a3c641000111111b52388a4183ad693.0" RoleInstance="WebRole_IN_0" xmlns="http://www.microsoft.com/UtlityComputing/ServiceDiagnostics/2008-08-01">
<Events>
<Event Time="2008-12-09T07:41:28.2482737Z" ThreadId="2284" Name="Information" Severity="Info">
<EventProperty Name="Message">Web Role Initalized</EventProperty>
</Event>
<Event Time="2008-12-09T07:41:28.2488483Z" ThreadId="2284" Name="Information" Severity="Info">
<EventProperty Name="Message">Web Role Started</EventProperty>
</Event>
</Events>
</ServiceDiagnostics>

I created a new class to hold each event entry. I have a future idea that involves filtering, sorting and other nice-to-have features. The class today looks like this:

public class ServiceDiagnosticEvent
{
public string ServiceName { get; set; }
public string Role { get; set; }
public string DeploymentId { get; set; }
public int RoleInstance { get; set; }
public DateTime EventTime { get; set; }
public int ThreadId { get; set; }
public string EventType { get; set; }
public string EventSeverity { get; set; }
public string EventMessage { get; set; }
}

Can you guess the rest? Read the properties from each XML node and add a new ServiceDiagnosticEvent object to a list. Finally bind the List to a GridView for instant list action. The only other thing I did was to remove the GetHealth calls to the worker role.

Simple, but effective – I don’t have to copy the logs anymore, and eventually I’ll add some features that allow filtering, sorting aggregation etc.

Enjoy, and if you do make some changes – let me know what you did!

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS