It has been a while since I last posted.

Since my last post, I’ve worked on some of the biggest Windows Azure projects, had 2 vacations, spoken at several conferences, changed jobs TWICE, moved office, moved house, made a doll’s bed & changed cars.

Looking back it was a very good year, but it was also an extremely busy year. There were many things I didn’t get to do as much as I wanted too. Cycling and blogging being 2 of the main ones.

1. Cycling – because riding 18 miles each day actually counts as exercise and its a great way to de-stress and think.
2. Blogging – because I find it incredibly useful to organize my thoughts into words. It also serves as a reference for when I forget things.

So the goal is to start doing these 2 things again. Wish me luck!

The file is used by the diagnostics agent to create the initial profile for collection. You can modify the settings after deployment using the PowerShell cmdlets I’ve talked about previously.

Here is a sample file. It should be pretty straight forward to understand. Note, times for seconds would be in the format PT30S, this representing 30 seconds.

<?xml version="1.0" encoding="utf-8" ?>
<DiagnosticMonitorConfiguration xmlns="http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration" configurationChangePollInterval="PT15M" overallQuotaInMB="4096">
  <WindowsEventLog bufferQuotaInMB="1024" 
                   scheduledTransferLogLevelFilter="Verbose" 
                   scheduledTransferPeriod="PT10M">
    <DataSource name="Application!*"/>
    <DataSource name="System!*"/>    
  </WindowsEventLog>
  <DiagnosticInfrastructureLogs bufferQuotaInMB="1024"
                                scheduledTransferLogLevelFilter="Verbose"
                                scheduledTransferPeriod="PT10M" />
  <Logs bufferQuotaInMB="1024"
        scheduledTransferLogLevelFilter="Verbose"
        scheduledTransferPeriod="PT10M" />
  <PerformanceCounters bufferQuotaInMB="512" scheduledTransferPeriod="PT10M">
    <PerformanceCounterConfiguration counterSpecifier="\Memory\Available MBytes" sampleRate="PT5M"/>
    <PerformanceCounterConfiguration counterSpecifier="\Processor(_Total)\% Processor Time" sampleRate="PT5M"/>
    <PerformanceCounterConfiguration counterSpecifier="\Network Interface(*)\Bytes Sent/sec" sampleRate="PT5M"/>
    <PerformanceCounterConfiguration counterSpecifier="\Network Interface(*)\Bytes Total/sec" sampleRate="PT5M"/>
  </PerformanceCounters>  
</DiagnosticMonitorConfiguration>

All you really have to do is drop that file in the root directory of your role and you are done!

The documentation here, suggests that you would use code to configure diagnostics. If you have ever read any of my blog posts you probably know how I feel about that. (Note: for those that haven’t its just wrong to use code. Really ops should be supplying this file for production. You have an ops team right?)

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS, UNLESS MY BROTHER GAVE YOU THE SECRET CODE.

Doing this creates the need for a tool to encrypt such settings. There are a few posts out there that show how to decrypt the values in code (you can grab some from here), you still need a way for operators to create these values in the first place. I thought a couple of PowerShell scripts should do the trick nicely.

You will need the thumbprint of a certificate in the CurrentUser\My store, which would be the same cert you deploy with you Azure deployment in order to decrypt.

The Encrypt function looks like:

Function Encrypt($stringToEncrypt, $thumb)
{
    $cert = get-item cert:\CurrentUser\My\$thumb
    [System.Reflection.Assembly]::LoadWithPartialName("System.Security") | out-null
    $passbytes = [Text.Encoding]::UTF8.GetBytes($stringToEncrypt)
    $content = New-Object Security.Cryptography.Pkcs.ContentInfo -argumentList (,$passbytes)
    $env = New-Object Security.Cryptography.Pkcs.EnvelopedCms $content
    $env.Encrypt((new-object System.Security.Cryptography.Pkcs.CmsRecipient($cert)))

    [Convert]::Tobase64String($env.Encode())
}

The Decrypt function looks like:

Function Decrypt($EncryptedString, $thumb)
{    
    $cert = get-item cert:\CurrentUser\My\$thumb    
    [System.Reflection.Assembly]::LoadWithPartialName("System.Security") | out-null
    $encodedBytes = [Convert]::Frombase64String($EncryptedString)
    $env = New-Object Security.Cryptography.Pkcs.EnvelopedCms
    $env.Decode($encodedBytes)
    $env.Decrypt($cert)
    $enc = New-Object System.Text.ASCIIEncoding
    
    $enc.GetString($env.ContentInfo.Content)    
}

Usage is simple:

$pwd = Encrypt "TheDavidAiken" "39836617C1A2BBAC6F90C0224C31B019854C6659"
Decrypt $pwd "39836617C1A2BBAC6F90C0224C31B019854C6659"

Enjoy.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS, UNLESS YOU HAVE A NOTE FROM MY MUM

Here is MY list of recommendations to help things run a little smoother:

1. Keep your diagnostics account separate from your production account. This will help with performance of both production and diagnostics since they won’t be competing for the same storage account.
2. Make sure your storage account is in the same data center as your compute. I know. Just saying.
3. Make sure you collect the right set of performance counters AND check that you are actually collecting data. Make sure you are collecting the .Net 4.0 counters for ASP.NET where applicable.
4. Use either the .wadcfg or PowerShell scripts I’ve talked about here to configure diagnostics. Hard coding it will overwrite any changes you make when an instance restarts.
5. Knowing and understanding your baseline workload is important. You should look at your performance on a regular basis, and over a period of time.
6. To troubleshoot, you can enable RDP and perform an upgrade of your service. You can then RDP into specific instances to troubleshoot.
7. When you are working with Microsoft’s product support, try not to delete old deployments. Once you do it makes finding a root cause more difficult. You can always VIP swap them out and leave them running while they troubleshoot.
8. Having more instances running means more people can look at the problem at the same time.
9. Check the status of the service at http://www.microsoft.com/windowsazure/support/status/servicedashboard.aspx
10. Invest in a diagnostics data viewing tool, such as Cerebrata, or grab a free trail of ManageAxis by following the rabbit hole from the Cloud Cover Show.

M5$A)4R!03U-424Y’($E3(%!23U9)1$5$(.*`G$%3($E3XH”=(%=)5$@@3D\@A5T%24D%.5$E%4RP@04Y$($-/3D9%4E,@3D\@4DE’2%13`

What the script will do

The script will:

1. Find out the current deployment id.
2. Connect to blob storage
3. List all of the blobs that don’t have the deployment id in the uri.
4. Delete those blobs.

Note: This assumes you have 1 storage account for each deployment. If you use the same storage account for diagnostics from multiple apps you need to change the script. Also Why?

Setup

This is the same as the previous post, so if you already have done this, you can skip to the next section.

First you need to make sure you have a management certificate created and installed.

Once you have that, you need to download the PowerShell cmdlets from codeplex.

Next figure out what the thumbprint of your certificate is. You can see this in the Windows Azure management portal when you click your certificate. While you are there, make a note of your subscription Id, Storage Account Name and Key and the service name.

Defining the service and storage variables

We’ll start with the same variables used in the previous post:

# TODO: Cert thumbprint of a certificate already installed into the Windows Azure Portal
$thumb = "ENTERTHUMBHERE"
$cert = get-item cert:\CurrentUser\My\$thumb
# TODO: Subscription Id
$subid = "ENTERSUBIDHERE"
# TODO: Storage Account Name
$SAN = "ENTERSTORAGEACCOUNTNAMEHERE"
# TODO: Storage Account Key
$SAK = "ENTERSTORAGEACCOUNTKEYHERE"
# TODO: Service Name
$serviceName = "ENTERSERVICENAMEHERE"

Getting the current Deployment Id

Next, we’ll grab the deployment id:

$did = (Get-Deployment -ServiceName $serviceName -Certificate $cert -SubscriptionId $subid -Slot Production).DeploymentId
write-host "current deployment " + $did

Connecting to blob storage

First thing you need to do is to make sure the assemblies for storage are loaded. You can do this using

add-type -Path 'C:\Program Files\Windows Azure SDK\v1.5\ref\Microsoft.WindowsAzure.StorageClient.dll'

To access storage, we will use the CloudBlobClient class from the storage client library. This requires some credentials:

$creds = New-Object Microsoft.WindowsAzure.StorageCredentialsAccountAndKey($SAN, $SAK)
$client = New-Object Microsoft.WindowsAzure.StorageClient.CloudBlobClient("https://$SAN.blob.core.windows.net",$creds)

Grabbing a list of blobs & deleting them

One of the great things we can do is grab a list of blobs in a container using a “flat blob listing” basically, it will list everything in the container, including all the sub folders. To do this, you need to define a BlobRequestOptions object:

$bro = New-Object Microsoft.WindowsAzure.StorageClient.BlobRequestOptions
$bro.UseFlatBlobListing = $true

You can grab a list of blobs for a container by using the ListBlobs method of a directory reference, which you need to grab for the container:

$wadcontainer = $client.GetBlobDirectoryReference("wad-control-container")
$todelete = $wadcontainer.ListBlobs($bro) | where {$_.Uri -notmatch $did}

To delete the blob, you can simply call the Delete method.

$todelete | foreach{ $_.Delete() }

Awesome.

Full Script

Here is the full script that also handles multiple wad-* containers:

# TODO: Cert thumbprint of a certificate already installed into the Windows Azure Portal
$thumb = "ENTERTHUMBHERE"
$cert = get-item cert:\CurrentUser\My\$thumb
# TODO: Subscription Id
$subid = "ENTERSUBSCRIPTIONIDHERE"
# TODO: Storage Account Name
$SAN = "ENTERSTORAGEACCOUNTNAMEHERE"
# TODO: Storage Account Key
$SAK = "ENTERSTORAGEACCOUNTKEYHERE"
# TODO: Service Name
$serviceName = "ENTERSERVICENAMEHERE"

# deployment Id
$did = (Get-Deployment -ServiceName $serviceName -Certificate $cert -SubscriptionId $subid -Slot Production).DeploymentId
write-host "current deployment " + $did

$creds = New-Object Microsoft.WindowsAzure.StorageCredentialsAccountAndKey($SAN, $SAK)
$client = New-Object Microsoft.WindowsAzure.StorageClient.CloudBlobClient("https://$SAN.blob.core.windows.net",$creds)

$bro = New-Object Microsoft.WindowsAzure.StorageClient.BlobRequestOptions
$bro.UseFlatBlobListing = $true

$containersToProcess = @("wad-control-container","wad-iis-logfiles")
$containersToProcess | foreach {
    $wadcontainer = $client.GetBlobDirectoryReference($_)

    $todelete = $wadcontainer.ListBlobs($bro) | where {$_.Uri -notmatch $did}
    $todelete | select Uri | fl | out-host

    $todelete | foreach { $_.Delete() }
}

Tip: If you want to see what it does before committing, simply comment out the $todelete | foreach{ $_.Delete() } statement.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS, UNLESS YOU HAVE A NOTE FROM MY MUM

Apart from the obvious separation of duties – unless you are specifically debugging a service deployment issue, you don’t need to write code in your service to enable diagnostics. You will get it wrong anyway by collecting 800 counters at 1 second intervals which is not likely to be any help to anyone.

What I typically start out collecting is some pretty basic things, CPU, Memory Usage, Network and if it is a ASP.NET site some ASP.NET specific counters. Basically I’m looking for enough information to give me a good finger on the pulse. These can be collected at intervals between 5 and 15 minutes because we are really interested in the load over time, rather than individual peaks.

Getting Started

First you need to make sure you have a management certificate created and installed.

Once you have that, you need to download the PowerShell cmdlets from codeplex.

Next figure out what the thumbprint of your certificate is. You can see this in the Windows Azure management portal when you click your certificate. While you are there, make a note of your subscription Id, Storage Account Name and Key and the service name.

Tip: While I’m figuring out the right commands to use, I usually set these up as variables.

Rather than dump the whole script on you, let me walk you through what is happening and why. You can do this interactively in a PowerShell window if you want to follow along. The entire script is at the end.

Here are a few lines of the script to set the variables up:

# TODO: Cert thumbprint of a certificate already installed into the Windows Azure Portal
$thumb = "PUTTHETHUMBHERE"
$cert = get-item cert:\CurrentUser\My\$thumb
# TODO: Subscription Id
$subid = "PUTYOURSUBSCRIPTIONIDHERE"
# TODO: Storage Account Name
$SAN = "PUTYOURSTORAGEACCOUNTNAMEHERE"
# TODO: Storage Account Key
$SAK = "PUTYOURSTORAGEACCOUNTKEYHERE"
# TODO: Service Name
$serviceName = "PUTYOURSERVICENAMEHERE"

As you can see, we can grab the actual certificate very easily using the get-item cmdlet.

Getting a list of services

You can check really quickly to see if you have everything configured correctly by using the Get-HostedServices cmdlet:

Get-HostedServices -SubscriptionId $subid -Certificate $cert

This cmdlet should return a list of services you have deployed to that service account.

To get information on an individual service you can you Get-HostedService:

Get-HostedService -SubscriptionId $subid -Certificate $cert -ServiceName $serviceName

Before I get carried away, you can get a full list of cmdlets to explore using:

Get-Command -Module WAPPSCmdlets

Getting the current deployment

Back to diagnostics. When configuring diagnostics, you need to know which deployment to work with. You specify the deployment using the deployment id. You can grab the current deployment using the Get-Deployment cmdlet:

Get-Deployment -ServiceName $serviceName -Certificate $cert -SubscriptionId $subid -Slot Production

One property of the result is DeploymentId. You can grab that and save it to another variable using:

$did = (Get-Deployment -ServiceName $serviceName -Certificate $cert -SubscriptionId $subid -Slot Production).DeploymentId

Once you have the DeploymentId you use it in many of the other commands to setup log collection etc.

To get a list of roles you can grab the RoleInstanceList from the deployment:

$roles = (Get-Deployment -ServiceName $serviceName -Certificate $cert -SubscriptionId $subid -Slot Production).RoleInstanceList

This returns both the role names as well as the instance names.

Note: There is a cmdlet Get-DiagnosticAwareRoles which returns the role names but not the instance names. There is also a Get-DiagnosticAwareRoleInstances which returns the instance names as strings. You could use both commands…

Just Web or Worker roles

Some diagnostics collection you just want to configure on web roles. I haven’t found a way from the PowerShell cmdlets to return a specific role type, so instead I always make sure my role name has the word “Web” in and use the following:

$webroles = $roles | where { $_.RoleName -match "Web"}

I haven’t figured out a better way of doing this, if you do let me know.

Guess how I find worker roles?

$workerroles = $roles | where { $_.RoleName -notmatch "Web"}

Anyway, if you have a list of roles and instance names, along with the storage account name and key and the deployment id, you are ready to go!

Configure the Windows Azure Logs

To configure the Windows Azure Logs you can use the Set-WindowsAzureLog cmdlet:

$roles | foreach { Set-WindowsAzureLog -LogLevelFilter "Error" -RoleName $_.RoleName -InstanceId $_.InstanceId -StorageAccountName $SAN -StorageAccountKey $SAK -DeploymentId $did -BufferQuotaInMB 50 -TransferPeriod 5 }

This will tell the diagnostics agent to copy the logs every 5 minutes. The total amount of data in the local log will not exceed 50MB

To check the configuration was set you have to use the Get-DiagnosticConfiguration cmdlet:

$roles | foreach {Get-DiagnosticConfiguration -DeploymentId $did -StorageAccountName $SAN -StorageAccountKey $SAK -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferName Logs }

Configure the Diagnostics Infrastructure Logs

To configure collection of the Windows Azure infrastructure logs use the Set-InfrastructureLog cmdlet:

$roles | foreach { Set-InfrastructureLog -LogLevelFilter "Error" -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferQuotaInMB 50 -TransferPeriod 5 -StorageAccountName $SAN -StorageAccountKey $SAK -DeploymentId $did }

To check the configuration use:

$roles | foreach {Get-DiagnosticConfiguration -DeploymentId $did -StorageAccountName $SAN -StorageAccountKey $SAK -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferName DiagnosticInfrastructureLogs }

Configure the Windows Event Logs

To configure the collection of the Windows Event Logs use you need to first specify which logs to collect. If you just want to collect the Application and System logs, you first need to create an array:

$logs = "Application!*","System!*"

You can then use this list in the following:

$roles | foreach { Set-WAEventLog -EventLogs $logs -LogLevel "Error" -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferQuotaInMB 10 -TransferPeriod 5 -StorageAccountName $SAN -StorageAccountKey $SAK -DeploymentId $did }

To check the configuration use:

$roles | foreach {Get-DiagnosticConfiguration -DeploymentId $did -StorageAccountName $SAN -StorageAccountKey $SAK -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferName WindowsEventLogs }

Configuring Performance Counters

To configure performance counters to be collected, you need to first create one or more PerformanceCounterConfiguration objects, then pass them into the Set-PerformanceCounter cmdlet.

To create a PerformanceCounterConfiguration object use:

$cpu_perfcounter = new-object Microsoft.WindowsAzure.Diagnostics.PerformanceCounterConfiguration
$cpu_perfcounter.CounterSpecifier = "\Processor(_Total)\% Processor Time"
$cpu_perfcounter.SampleRate = new TimeSpan(0,5,0)

For all roles, you should create performance counters for the following:

• \Processor(_Total)\% Processor Time
• \Memory\Available Mbytes
• \Memory\Committed Bytes
• \Network Interface(*)\Bytes Received/sec
• \Network Interface(*)\Bytes Sent/sec

For a webrole you should add at least:

• \ASP.NET Applications(__Total__)\Requests/Sec

Once you have created each PerformanceCounterConfiguration object create an array then use the Set-PerformanceCounter cmdlet:

$cpu_perfcounter = new-object Microsoft.WindowsAzure.Diagnostics.PerformanceCounterConfiguration
$cpu_perfcounter.CounterSpecifier = "\Processor(_Total)\% Processor Time"
$cpu_perfcounter.SampleRate = $counter_time

Get-DiagnosticConfiguration -DeploymentId $did -StorageAccountName $SAN -StorageAccountKey $SAK -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferName PerformanceCounters    

If you actually want to view the performance counter names and sample times you need to look at the datasources property:

(Get-DiagnosticConfiguration -DeploymentId $did -StorageAccountName $SAN -StorageAccountKey $SAK -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferName PerformanceCounters).DataSources

Configuring IIS Log Files

Windows Azure lets you configure the collection of any log files. Typically for a web role you would want to grab the IIS log files. In order to do this, you have to configure a datasource that specifies which folder the log files will be located in. To figure this out, the pattern is:

C:\Resources\directory\<DeploymentId>.<RoleName>.DiagnosticStore\LogFiles

Which can be constructed using:

$logfile = "C:\Resources\directory\" + $did + "." + $WebRoles[0].RoleName + ".DiagnosticStore\LogFiles"

Other formats are Failed Request Logs

C:\Resources\directory\<DeploymentID>.<RoleName>\FailedReqLogFile

and Crash Dumps:

 C:\Resources\directory\<DeploymentId>.<RoleName>\CrashDumps

To create the actual datasource, you need to create an instance of DirectoryConfiguration:

$iisDirectorySource = new-object -TypeName Microsoft.WindowsAzure.Diagnostics.DirectoryConfiguration
$iisDirectorySource.Path = $logfile
$iisDirectorySource.DirectoryQuotaInMB = 100
$iisDirectorySource.Container = "wad-iis-logfiles"

You can then set the directory sources:

$webroles | foreach { Set-FileBasedLog -DirectoriesConfiguration @($iisDirectorySource) -RoleName $_.RoleName -InstanceId $_.InstanceId -BufferQuotaInMB 1024 -TransferPeriod 5 -StorageAccountName $SAN -StorageAccountKey $SAK -DeploymentId $did }

Phew – made it.

Here is the full script for setting the configuration, followed by a script that will display the configuration:

# TODO: Cert thumbprint of a certificate already installed into the Windows Azure Portal
$thumb = "ENTERTHUMBHERE"
$cert = get-item cert:\CurrentUser\My\$thumb
# TODO: Subscription Id
$subid = "ENTERSUBIDHERE"
# TODO: Storage Account Name
$SAN = "ENTERSTORAGEACCOUNTNAMEHERE"
# TODO: Storage Account Key
$SAK = "ENTERSTORAGEACCOUNTKEYHERE"
# TODO: Service Name
$serviceName = "ENTERSERVICENAMEHERE"

# You can leave the remainder of the script alone, will collect some good counters at 5 minute intervals and setup logging of event logs etc.
# Does not setup Fail Request Logs
# 
# Logs to collect
$logs = "Application!*","System!*"
# configure performance counters sample TimeSpan (h,m,s) - recommended 5 - 15 minutes
$counter_time = new-object TimeSpan(0,10,0)
# Configure default collection time (recommended 5-15 minutes)
$transferPeriod = 5

# performance counters
$cpu_perfcounter = new-object Microsoft.WindowsAzure.Diagnostics.PerformanceCounterConfiguration
$cpu_perfcounter.CounterSpecifier = "\Processor(_Total)\% Processor Time"
$cpu_perfcounter.SampleRate = $counter_time

$mem_available_bytes_perfcounter = New-Object Microsoft.WindowsAzure.Diagnostics.PerformanceCounterConfiguration
$mem_available_bytes_perfcounter.CounterSpecifier = "\Memory\Available Mbytes"
$mem_available_bytes_perfcounter.SampleRate = $counter_time

$mem_committed_bytes_perfcounter = New-Object Microsoft.WindowsAzure.Diagnostics.PerformanceCounterConfiguration
$mem_committed_bytes_perfcounter.CounterSpecifier = "\Memory\Committed Bytes"
$mem_committed_bytes_perfcounter.SampleRate =$counter_time

$asp_app_requests_perfcounter =  New-Object Microsoft.WindowsAzure.Diagnostics.PerformanceCounterConfiguration
$asp_app_requests_perfcounter.CounterSpecifier = "\ASP.NET Applications(__Total__)\Requests/Sec"
$asp_app_requests_perfcounter.SampleRate = $counter_time

$net_received_perfcounter =  New-Object Microsoft.WindowsAzure.Diagnostics.PerformanceCounterConfiguration
$net_received_perfcounter.CounterSpecifier = "\Network Interface(*)\Bytes Received/sec"
$net_received_perfcounter.SampleRate = $counter_time

$net_sent_perfcounter =  New-Object Microsoft.WindowsAzure.Diagnostics.PerformanceCounterConfiguration
$net_sent_perfcounter.CounterSpecifier = "\Network Interface(*)\Bytes Sent/sec"
$net_sent_perfcounter.SampleRate = $counter_time

# deployment Id
$did = (Get-Deployment -ServiceName $serviceName -Certificate $cert -SubscriptionId $subid -Slot Production).DeploymentId
# Role List
$roles = (Get-Deployment -ServiceName $serviceName -Certificate $cert -SubscriptionId $subid -Slot Production).RoleInstanceList

#Web Role List
$webroles = $roles | where { $_.RoleName -match "Web"}
#worker Role List
$workerroles = $roles | where { $_.RoleName -notmatch "Web"}

$roles | foreach { Set-WindowsAzureLog -LogLevelFilter "Error" -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferQuotaInMB 50 -TransferPeriod $transferPeriod -StorageAccountName $SAN -StorageAccountKey $SAK -DeploymentId $did}

$roles | foreach { Set-InfrastructureLog -LogLevelFilter "Error" -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferQuotaInMB 50 -TransferPeriod $transferPeriod -StorageAccountName $SAN -StorageAccountKey $SAK -DeploymentId $did }

$logs = "Application!*","System!*"
$roles | foreach { Set-WAEventLog -EventLogs $logs -LogLevel "Error" -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferQuotaInMB 10 -TransferPeriod $transferPeriod -StorageAccountName $SAN -StorageAccountKey $SAK -DeploymentId $did }

#Web Role Specific Configuration
if ($WebRoles -ne $null)
{
    $logfile = "C:\Resources\directory\" + $did + "." + $WebRoles[0].RoleName + ".DiagnosticStore\LogFiles"

    $iisDirectorySource = new-object -TypeName Microsoft.WindowsAzure.Diagnostics.DirectoryConfiguration
    $iisDirectorySource.Path = $logfile
    $iisDirectorySource.DirectoryQuotaInMB = 100
    $iisDirectorySource.Container = "wad-iis-logfiles"

    $webroles | foreach { Set-FileBasedLog -DirectoriesConfiguration @($iisDirectorySource) -RoleName $_.RoleName -InstanceId $_.InstanceId -BufferQuotaInMB 1024 -TransferPeriod 5 -StorageAccountName $SAN -StorageAccountKey $SAK -DeploymentId $did }
    
    $web_perf_counters = $cpu_perfcounter, $mem_available_bytes_perfcounter, $mem_committed_bytes_perfcounter, $net_received_perfcounter, $net_sent_perfcounter, $asp_app_requests_perfcounter
    $webroles | foreach { Set-PerformanceCounter -PerformanceCounters $web_perf_counters -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferQuotaInMB 10 -TransferPeriod $transferPeriod -StorageAccountName $SAN -StorageAccountKey $SAK -DeploymentId $did } 
}

#Worker Role Specific Configuration
if ($workerroles -ne $null)
{    
    $worker_perf_counters = $cpu_perfcounter, $mem_available_bytes_perfcounter, $mem_committed_bytes_perfcounter, $net_received_perfcounter, $net_sent_perfcounter
    $workerroles | foreach { Set-PerformanceCounter -PerformanceCounters $worker_perf_counters -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferQuotaInMB 10 -TransferPeriod $transferPeriod -StorageAccountName $SAN -StorageAccountKey $SAK -DeploymentId $did } 
}

The display script:

# TODO: Cert thumbprint of a certificate already installed into the Windows Azure Portal
$thumb = "ENTERTHUMBHERE"
$cert = get-item cert:\CurrentUser\My\$thumb
# TODO: Subscription Id
$subid = "ENTERSUBIDHERE"
# TODO: Storage Account Name
$SAN = "ENTERSTORAGEACCOUNTNAMEHERE"
# TODO: Storage Account Key
$SAK = "ENTERSTORAGEACCOUNTKEYHERE"
# TODO: Service Name
$serviceName = "ENTERSERVICENAMEHERE"


# deployment Id
$did = (Get-Deployment -ServiceName $serviceName -Certificate $cert -SubscriptionId $subid -Slot Production).DeploymentId
# Role List
$roles = (Get-Deployment -ServiceName $serviceName -Certificate $cert -SubscriptionId $subid -Slot Production).RoleInstanceList

$roles | foreach {
    write-host "======================================================"
    write-host $_.RoleName 
    write-host $_.InstanceName

    write-host "Event Logs"
    Get-DiagnosticConfiguration -DeploymentId $did -StorageAccountName $SAN -StorageAccountKey $SAK -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferName WindowsEventLogs | out-host
        
    write-host "Infrastructure Logs"
    Get-DiagnosticConfiguration -DeploymentId $did -StorageAccountName $SAN -StorageAccountKey $SAK -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferName DiagnosticInfrastructureLogs 

    write-host "Logs"
    Get-DiagnosticConfiguration -DeploymentId $did -StorageAccountName $SAN -StorageAccountKey $SAK -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferName Logs 

    write-host "Performance Counters"
    Get-DiagnosticConfiguration -DeploymentId $did -StorageAccountName $SAN -StorageAccountKey $SAK -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferName PerformanceCounters | out-host    
    (Get-DiagnosticConfiguration -DeploymentId $did -StorageAccountName $SAN -StorageAccountKey $SAK -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferName PerformanceCounters).DataSources | out-host
    
    
    write-host "Directories"
    Get-DiagnosticConfiguration -DeploymentId $did -StorageAccountName $SAN -StorageAccountKey $SAK -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferName Directories | out-host
    (Get-DiagnosticConfiguration -DeploymentId $did -StorageAccountName $SAN -StorageAccountKey $SAK -RoleName $_.RoleName -InstanceId $_.InstanceName -BufferName Directories).DataSources | fl | out-host
    
    write-host ""
}

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS, EVEN IF YOU HAVE CHOCOLATE

One thing that always annoyed me is that it is a warning and not an error. I may have in the past deployed packages that failed, just because I wasn’t looking at the errors.

However you can actually have these warnings block your deployment by setting the “Treat warnings as errors” option on your Windows Azure project to true. (Tip: right click your Windows Azure project, click properties, then click Development.)

Which changes the warnings to errors:

Awesome.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS OR DANCING CATS

Transient Datacenter conditions

When you have to deal with external services of any type there are times when the service might not respond. This could be due to any number of reasons from network connectivity & service throttling, to hardware failure. Windows Azure is designed to withstand these failures, not by avoiding them, but by taking corrective action when they do occur. Windows Azure auto heals itself. These conditions are sometimes referred to as transient conditions because they are not typically long lasting.

As an example, SQL Azure can give you connection errors, and throttling errors, Windows Azure Storage can give you timeout and throttling errors and Service Bus has ServerBusy and MessagingCommunication Exceptions.

Any other external dependency will also likely have similar conditions. Without defensive coding for these transient conditions, your app will suffer unnecessary outages. Fortunately the problem can be easily resolved.

Retry Logic

Handling these conditions is usually as easy as repeating the operation after a short delay.

The Windows Azure Storage Client Library that ships with the SDK already has retry behavior that you need to switch on. You can set this on any storage client by setting the RetryPolicy Property.

SQL Azure doesn’t provide a default retry mechanism out of the box, since it uses the SQL Server client libraries. Service Bus also doesn’t provide a retry mechanism.

Transient Fault Handling Framework

To provide an easy way to handle this, the Windows Azure Customer Advisory Team have developed a Transient Fault Handling Framework – http://windowsazurecat.com/2011/02/transient-fault-handling-framework/. The framework provides a number of ways to handle specific SQL Azure, Storage, Service Bus and Cache conditions.

The most interesting aspect to me however is the ExecuteAction and ExecuteAction<T> methods. These methods allow you to basically wrap any user code in a retry block. Example:

var policy = new RetryPolicy<SqlAzureTransientErrorDetectionStrategy>(MaxRetries,
                    TimeSpan.FromMilliseconds(DelayMs));
policy.ExecuteAction(() => object.DoSomething());

Retry Pattern

What is great about these methods are they enable you to use the decorator pattern to add retry logic to your service. This of course assumes you built your service with extensibility in mind.

In my example I have a UriRepository which is defined by the IUriRepository interface. I have a SQLAzureUriRepository that implements the interface. This class however contains no retry logic. Instead I implemented a RetryUriRepository that also implements IUriRepository. RetryUriRepository allows you to specify via constructor injection, which UriRepositiory to retry.

Here is a snippet of the RetryUriRepository:

public class RetryUriRepository : IUriRepository
{
    private readonly IUriRepository _uriRepository;
    private const int MaxRetries = 10;
    private const double DelayMs = 2000;

    public RetryUriRepository(IUriRepository uriRepository)
    {   _uriRepository = uriRepository;    }

    public void InsertUri(string shortUri, string longUri, string ipAddress)
    {
        var policy = GetRetryPolicy();
        policy.ExecuteAction(() => _uriRepository.InsertUri(shortUri, longUri, ipAddress));
    }
    private static RetryPolicy GetRetryPolicy()
    {
        return new RetryPolicy<SqlAzureTransientErrorDetectionStrategy>(MaxRetries,         
                     TimeSpan.FromMilliseconds(DelayMs));
    }
}

Using the supplied framework might be overkill, but it should give you an idea on how to implement retry logic in your service.

Too Much Retry

One thing that becomes interesting is when the number of retries increases. This typically indicates either a longer error condition, or you are overloading the services you are consuming. The most likely, and only one we can do anything about, is the later. The more throttling that goes on, the more retires. The more retires the less throughput. The less throughput the slower the response time. Poor response time = disgruntled users (and executives).

Don’t be tempted to turn off the retry logic when this happens. This will just make the problem much worse. About the only solution when dealing with overloading a service is to either scale that service out, or attempt to delay the processing using a queue/worker pattern.

Conclusion

Implementing retry logic is critical if you want your service to keep running. Monitoring the frequency of these retries can be a good indicator you are starting to experience scale issues. Don’t turn your retry logic off to handle scale issues.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS, EXCEPT ON THE THIRD SUNDAY WITH A FULL MOON

We don’t as a rule talk very much about how awesome they are – although there are a few videos out there which are great for a Friday lunchtime watch.

Guided Tour inside the Windows Azure Cloud with Patrick Yantz

A great animation of the ITPAC

Our recent GFS datacenter tour video

You can find more geeky videos on the Microsoft Global Foundation Services site at http://www.globalfoundationservices.com/infrastructure/videos.html.

Just a note, these are videos about our datacenters and not just Windows Azure.

BTW – if you are in the Seattle area, you should join the user group. People like Steve Marx & Vittorio Bertocci might even be there.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS

Storage on-premises

If you think about how you build big applications on premises and how you approach data in general is very different. Typically you have a database or storage standard. This could be SQL Server, or some other relational database. The law of the enterprise is usually “all data shall be stored in <insert your enterprise database here>”. There is no other choice.

The second big difference is how you approach scale. Typically to get scale, you build the biggest most elaborate database server you can. I remember years ago, tuning everything from the hardware, through to how the disks were partitioned and where the file groups in SQL Server were located and used. Machines typically had multiple CPU’s with multiple cores, multiple disk controllers, disks and a bucket full of RAM.

If you also needed high availability, that was easy. You typically built a second monster machine and configured an Active/Passive setup, or hot standby,

How the cloud changes storage

The first thing about using storage in the cloud is that there are zero setup costs. You can just as easily create a SQL Azure database or 7. Create multiple storage accounts, tables etc. You pay for what you consume. Setup is free. This means you have less reason to put all your data in one place. You can start thinking about which data source is right for what data. It is not unusual to have multiple data sources used.

The second big thing is that we don’t have that big huge server with lots of cores and ram and fancy stuff. At least not tuned to the extent you would for your workload. Instead we have “commodity hardware”. To achieve “scale” you have to think about using lots of little storage things, rather than a single huge one. This is the exact same thing you do with compute.

If you have 50GB of data, you may be better served by 50 x 1GB databases than 1 x 50GB. The cost is comparable too. Except you now have 50 of those servers running your database. Guess which model will scale better?

There is a tax in thinking about, designing and implementing this – however if done right, you can scale out as far as your compute (and credit card) will let you.

This might have been a rant. Apologies.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS

First, this is what you get as a default:

It looks like there are 4 things that need removing X-Powered-By, X-AspNet-Version, X-AspNetMvc-Version and Server.

The first 2 tags can be removed by editing the web.config of your application:

To remove the “X-AspNet-Version” tag, in <system.web> add:

<httpRuntime enableVersionHeader="false" />

To remove the “X-Powered-By”, in <system.webServer> add:

<httpProtocol>
  <customHeaders>
    <remove name="X-Powered-By" />
  </customHeaders>
</httpProtocol>

To remove the “X-AspNetMvc-Version” you need to edit you Global.asax.cs and add the following to the Application_Start() method:

MvcHandler.DisableMvcResponseHeader = true;

Removing the “Server” tag is a little harder. The only thing I could find was (and I spent around 30 seconds looking) was this great post from Stefan Goßner which I adapted to remove the header. It requires you install a custom Http Module into IIS. Here is the code:

using System;
using System.Web;

namespace ServerHeaders
{
    public class CustomServerHeaderModule : IHttpModule
    {
        public void Init(HttpApplication context)
        {            
            context.PreSendRequestHeaders += OnPreSendRequestHeaders;
        }
        public void Dispose() { }
        static void OnPreSendRequestHeaders(object sender, EventArgs e)
        {
            // remove the "Server" Http Header
            HttpContext.Current.Response.Headers.Remove("Server");
        }
    } 
}

I created a new class project for this class, then added a reference to the project from the web application. To wire up the handler I updated the <modules> tag to the following:

<modules runAllManagedModulesForAllRequests="true">
  <add name="CustomServerHeader" type="ServerHeaders.CustomServerHeaderModule" />
</modules>

Deploying to Windows Azure yields:

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS

General Cloud Security including data center security:

• Microsoft Global Foundation Services security page
• Information Security Management System for Microsoft Cloud Infrastructure
• Microsoft’s  Compliance Framework for Online Services
• Securing Microsoft’s Cloud Infrastructure
• Webcast: Online Services Security and Compliance- Microsoft’s Compliance Framework for Online Services
• Whiteboard Session: Online Services Security and Compliance- Microsoft’s Compliance Framework for Online Services

Office 365 Security

• Office 365 Security Service Description

Windows Azure Platform Security

• Windows Azure™ Security Overview
• TechNet Webcast – Windows Azure Security – A Peek Under the Hood (Level 100)
• MSDN Webcast – Security Talk – Using Windows Azure Storage Securely (Level 200)
• MSDN Webcast – Security Talk – Azure Federated Identity Security Using ADFS 2.0 (Level 300)
• Security Guidelines and Limitations (SQL Azure Database)
• SQL Azure Firewall
• Security Best Practices for Developing Windows Azure Applications
• http://www.microsoft.com/security/sdl/

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS

For those battling with email, I thought I would share my latest setup. Every few months I have a large email purge, usually forced by the “your mailbox is almost at its maximum size” email. Each time I do a purge, I try to think of a way to reduce my email tax, and automate as much as I can – using tools and techniques from GTD etc. The outcome from each purge is that I generally spend less time each day processing email, and more time actually doing the things that matter.

So here is what I have in place right now:

• Inbox has a auto archive (which just deletes) policy of 2 months. I’m working up the courage to switch to just a few days here. Honestly, if I haven’t responded, or moved the email to Action/Wait For or Someday in 72 hours, I probably won’t.
• Sent items has 1 month auto archive policy.
• All emails to DL’s are stored in a DL folder with a 1 week retention policy. There is one DL exception, which contains the team I support and is a mini-action item folder.
• Action items have a 1 month retention policy.
• Wait For & Someday items have a 3 month retention policy.
• I have a reference folder which as no retention policy as well as a business folder with a 1 year retention policy.
• Outlook clears deleted items on exit.

How do you handle your email?

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS

Problem: Windows Azure runs startup tasks as localsystem. Some startup tasks need to be running in the context of a user.

Solution: Use the task scheduler in Windows Server to execute the command.

A few people have already asked how to apply the technique to other things so here goes.

Lets take a look at the original startup task I was trying to execute.

netsh advfirewall firewall add rule name="Windows Remote Management (HTTP-In)" dir=in action=allow service=any enable=yes profile=any localport=5985 protocol=tcp

Powershell -Command Enable-PSRemoting -Force

This doesn’t work because the powershell command “Enable-PSRemoting” doesn’t work unless it runs as an elevated user that belongs to the administrators group. Localsystem doesn’t belong to this group.

In my original blog post, I showed how you could enable this using winrm. But sometimes you want this something to happen when the role starts.

To make a task run as a user account as a startup task simply create a new task for the Windows Scheduler to execute as shown below.

netsh advfirewall firewall add rule name="Windows Remote Management (HTTP-In)" dir=in action=allow service=any enable=yes profile=any localport=5985 protocol=tcp

net user scheduser Secr3tC0de /add

net localgroup Administrators scheduser /add

schtasks /CREATE /TN "EnablePS-Remoting" /SC ONCE /SD 01/01/2020 /ST 00:00:00 /RL HIGHEST /RU scheduser /RP Secr3tC0de /TR "powershell -Command Enable-PSRemoting -Force" /F

schtasks /RUN /TN "EnablePS-Remoting"

Works like a charm. Hopefully you can figure out how to run your own commands instead of PowerShell.

It is safe to say that you probably want to secure the username and password that is created. I think ideally a wrapper that can read a username and encrypted password from config, create all the scheduled tasks and then execute them. Maybe that is a job for next time.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS

I started writing this post a week ago in response to a customer request, so excited I was I even tweeted about writing a post on PowerShell. My Bad – because right after I tweeted I hit a snag – which is worthy of a whole post on its own. Ignoring the snag right now – let me tell you how to do the above.

First – a BIG shout out to the PowerShell team at Microsoft who answered my endless questions. Also a big shout out to Lee Holmes – who once again saved my bacon.

Anyway…

At long last I’ve had a chance to “play” around with the new Windows Azure features we announced at PDC 2010. I thought it would be fun to enable PowerShell Remoting in Windows Azure Roles. (Note I’m talking about Web and Worker roles here – not VM Role).

With new features such as remote desktop, startup tasks and Azure Connect – setting up PowerShell should be easy.

First, I’m going to assume you have worked through the notes/tutorials/stuff to enable Azure Connect & Remote Desktop – that way this post stays within the realms of being relatively small.

Here is our checklist:

• Make sure the OS Family in the ServiceConfiguration.cscfg is set to “2” to enable R2. 
• Create a user account so you can connect to the server.
• Add a startup task to open the firewall port.
• Add the Role to Azure Connect.
• Execute Lee’s magic script to enable PowerShell Remoting

PowerShell v2 is the version you need to do remoting. Server 2008 R2 contains PowerShell v2 in the box. We can tell Windows Azure to use an R2 server by changing the OSFamily in the ServiceCOnfiguration.cscfg to 2 as shown below:

<ServiceConfiguration serviceName="AzureMemcachedTest" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="2" osVersion="*">

Next step is to create a user account so that you can actually connect to the server. The easiest way to do this is to enable remote desktop, which creates a user on the host.

Now to make sure the firewall port is open we will need. I created a .cmd file containing the following 2 commands. The first opens the firewall for WINRM, the second for ping. This was added in the root folder of my role project.

netsh advfirewall firewall add rule name="Windows Remote Management (HTTP-In)" dir=in action=allow service=any enable=yes profile=any localport=5985 protocol=tcp

netsh advfirewall firewall add rule name="ICMPv6 echo" dir=in action=allow enable=yes protocol=icmpv6:128,any

Then I added a startup task to ServiceDefinition.csdef:

<Startup>

  <Task commandLine="EnablePowershellRemoting.cmd" executionContext="elevated" taskType="foreground"/>

</Startup>

Next add the Role to Azure Connect. I’ll assume you know how to do this.

Now deploy your service. Once deployed, do the final step to connect the Azure connect network, and make sure the agent is installed on your computer.

The final step is to “turn on” PowerShell Remoting. For numerous reasons, you cannot just run a startup task with “Enable-PSRemoting” as the command. The biggest reason is that startup tasks run as local system and thus cannot actually complete the Enable-PSRemoting command.

This is where I got stuck for 3 days until Lee shared his script. The script is below and is fairly easy to follow. Basically it will connect to the VM and create a scheduled task to enable-psremoting.

##############################################################################

##

## Enable-RemotePsRemoting

##

## From Windows PowerShell Cookbook (O'Reilly)

## by Lee Holmes (http://www.leeholmes.com/guide)

##

##############################################################################

<#

.SYNOPSIS

Enables PowerShell Remoting on a remote computer. Requires that the machine

responds to WMI requests, and that its operating system is Windows Vista or

later.

.EXAMPLE

Enable-RemotePsRemoting <Computer>

#>

param(

    ## The computer on which to enable remoting

    $Computername,

    ## The credential to use when connecting

    $Credential = (Get-Credential)

)

Set-StrictMode -Version Latest

$VerbosePreference = "Continue"

$credential = Get-Credential $credential

$username = $credential.Username

$password = $credential.GetNetworkCredential().Password

$script = @"

`$log = Join-Path `$env:TEMP Enable-RemotePsRemoting.output.txt

Remove-Item -Force `$log -ErrorAction SilentlyContinue

Start-Transcript -Path `$log

## Create a task that will run with full network privileges.

## In this task, we call Enable-PsRemoting

schtasks /CREATE /TN 'Enable Remoting' /SC WEEKLY /RL HIGHEST ``

    /RU $username /RP $password ``

    /TR "powershell -noprofile -command Enable-PsRemoting -Force" /F |

    Out-String

schtasks /RUN /TN 'Enable Remoting' | Out-String

`$securePass = ConvertTo-SecureString $password -AsPlainText -Force

`$credential =

    New-Object Management.Automation.PsCredential $username,`$securepass

## Wait for the remoting changes to come into effect

for(`$count = 1; `$count -le 10; `$count++)

{

    `$output = Invoke-Command localhost { 1 } -Cred `$credential ``

        -ErrorAction SilentlyContinue

    if(`$output -eq 1) { break; }

    "Attempt `$count : Not ready yet."

    Sleep 5

}

## Delete the temporary task

schtasks /DELETE /TN 'Enable Remoting' /F | Out-String

Stop-Transcript

"@

$commandBytes = [System.Text.Encoding]::Unicode.GetBytes($script)

$encoded = [Convert]::ToBase64String($commandBytes)

Write-Verbose "Configuring $computername"

$command = "powershell -NoProfile -EncodedCommand $encoded"

$null = Invoke-WmiMethod -Computer $computername -Credential $credential `

    Win32_Process Create -Args $command

Write-Verbose "Testing connection"

Invoke-Command $computername {

    Get-WmiObject Win32_ComputerSystem } -Credential $credential

Piece of cake really.

Once you have the script saved, you can execute it like

PS> $computername = "WhateverTheComputerNameOfTheInstanceIs"
PS> .\Enable-RemotePSRemoting.ps1 $computername

Once it finishes executing, you should be able to connect using:

PS> Enter-PSSession –ComputerName $computername –Credential RemoteDesktopUsername

Then you can work interactively – try get-process as an example.

Pretty neat, and great for debugging!

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS

Cutting a really long story short. I wanted an easy way of adding caching to a Windows Azure project. Several people I know are already using Memcached, while waiting for our own caching service to go into production. I thought, this should be a plugin.

A plugin is exactly what it sounds, it is something that gets “plugged” into your role at build time. You can see the existing plugins on you have installed by looking at C:\Program Files\Windows Azure SDK\v1.3\bin\plugins. You should see plugins for diagnostics, RemoteAccess and more.

To make your own plugin is fairly easy – if you look in any of the plugin folders you will see a .CSPLUGIN file. You should be able to work out how this works. If you cannot, you probably don’t want to be building your own.

For the Memcached plugin, all I really need is the memcached.exe file (and any dependencies( I used the version from http://www.urielkatz.com/archive/detail/memcached-64-bit-windows/)), plus a little wrapper to read the config and launch memcached with the correct parameters – and of course the CSPLUGIN file:

<?xml version="1.0" ?>
<RoleModule
  xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition"
  namespace="Memcached64Plugin">
  <Startup>
    <Task commandLine="Memcached64Plugin.exe" taskType="background" executionContext="limited"/>
  </Startup>
  <Endpoints>
    <InternalEndpoint name="Endpoint" protocol="tcp" port="11212" />
  </Endpoints>
  <ConfigurationSettings>
    <Setting name="CacheSizeInMB"/>
  </ConfigurationSettings>
</RoleModule>

What the above essentially does is:

1. Adds a startup task
2. Adds an InternalEndpoint for port 11212
3. Adds a CacheSizeInMb setting

The little wrapper was written as a console app:

static void Main()
{

    var endpoint = RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["Memcached64Plugin.Endpoint"].IPEndpoint;
    var cacheSize = RoleEnvironment.GetConfigurationSettingValue("Memcached64Plugin.CacheSizeInMB");
    var startupInfo = new ProcessStartInfo("memcached.exe", string.Format("-m {0} -p {1}", cacheSize, endpoint.Port));

    Process.Start(startupInfo);
}

Note when we read the values we use the format <name of the plugin>.<key>, rather than just <key>, in this case Memcached64Plugin.Endpoint, rather than just Endpoint.

Once you build the solution and copy the files into the plugin folder, you can then use the Imports tag in your ServiceConfiguration.csdef file:

 <Import moduleName="Memcached64Plugin" />

This will magically add the correct settings to your ServiceConfiguration.cscfg file as shown:

    <ConfigurationSettings>
      <Setting name="Memcached64Plugin.CacheSizeInMB" value="512" />

Note, you won’t see the internal port or the startup task.

When you build and deploy (or even run in the Compute Emulator) the plugin will be packaged up with your code and executed on boot.

For the client side, I used the enyim client, which you can grab from http://memcached.enyim.com/.

You can grab a list of memcached servers using the RoleEnvironment, something like:

RoleEnvironment.Roles["WebRole1"].Instances.Select(instance => instance.InstanceEndpoints["Memcached64Plugin.Endpoint"].IPEndpoint)

Note the role is WebRole1 and the port we are looking for is the one we defined in the CSPLUGIN!

You should also write some code to handle a topology change, including adding more instances (or removing instances). You can do this by adding code to handle the RoleEnvironment.Changed event, and rebuilding your server list.

You can grab my plugin below, minus the memcached.exe and required dll. You can download those from the http://www.urielkatz.com/archive/detail/memcached-64-bit-windows/.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS

PS: No, this was nothing to do with PowerShell, I hit an issue with that which I’m still working on – cross your fingers – it will be splendid.

The Windows Azure Diagnostics Viewer is a prototype web application that can be used to view and analyze Windows Azure Diagnostics information.

The Windows Azure Diagnostics Viewer has 2 components, a website and a scheduler. The website provides a UI to allow you to view diagnostics information collected by Windows Azure, including performance counter graphs, logs as well as some customer data including Windows Azure Queue statistics and service status data.

The website is designed to be “injected” into an existing worker role by adding references to a few dll’s as well as the website “exe”. The goal here is to make adding the viewer to a project painless – although in the prototype there are still several steps to complete.

Once started, the viewer website uses the HWC to host the website.

As well as simply viewing data collected by the existing Windows Azure Diagnostics system, the viewer can also display pre-aggregated data as well as information about Queue lengths and even service status. It does this with the help of the scheduler.

The scheduler is also designed to be “injected” into a role, and simply executes specified tasks. These tasks are contained in dll’s which are loaded from blob storage. The viewer provides shortcuts to common tasks, such as queue length collection and data aggregation.

Whilst this is a prototype, it has inspired some other thinking and ideas. You can expect a rev for the new SDK shortly after the new SDK hits as we’d like to take advantage of some of the new SDK features.

Anyway, check it out and let us know what you think.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS

FabrikamShipping SaaS is a complete subscription based solution running on the Windows Azure platform and publicly available thru www.fabrikamshipping.com. It offers a web-based customer onboarding UI, which anybody can use for creating test subscriptions and obtain their very own application instance, dynamically provisioned by a simple but powerful provisioning engine. Thanks to our partnership with our friends evangelists at PayPal, it even demonstrates how to integrate payments and billing via an external provider!

Find out more at Vittorio’s blog – http://blogs.msdn.com/b/vbertocci/archive/2010/10/07/new-online-demo-introducing-fabrikamshipping-saas.aspx.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS

Here is what is new in the training kit:

• Updated all hands-on labs and demo scripts for Visual Studio 2010, the .NET Framework 4, and the Windows Azure Tools for Visual Studio version 1.2 release
• Added a new hands-on lab titled "Introduction to the AppFabric Access Control Service (September 2010 Labs Release)"
• Added a new hands-on lab "Debugging Applications in Windows Azure"
• Added a new hands-on lab "Asynchronous Workload Handling"
• Added a new exercise to the "Deploying Applications in Windows Azure" hands-on lab to show how to use the new tools to directly deploy from Visual Studio 2010.
• Added a new exercise to the "Introduction to the AppFabric Service Bus" hands-on lab to show how to connect a WCF Service in IIS 7.5 to the Service Bus
• Updated the "Introduction to AppFabric Service Bus" hands-on lab based on feedback and split the lab into 2 parts
• All of the presentations have also been updated and refactored to provide content for a 3 day training workshop.
• Updated the training kit navigation pages to include a 3 day agenda, a change log, and an improved setup process for hands-on labs.

So go download the kit right now and keep your skills sharp.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS

Project Houston is the answer. Here is what the product team have to say about it:

Microsoft® Project Code-Named “Houston” is a lightweight and easy to use database management tool for SQL Azure databases. It is designed specifically for Web developers and other technology professionals seeking a straightforward solution to quickly develop, deploy, and manage their data-driven applications in the cloud. Project “Houston” provides a web-based database management tool for basic database management tasks like authoring and executing queries, designing and editing a database schema, and editing table data. It is now available on SQL Azure Labs.

If you follow the link, then click on the CTP link, you will be presented with the following login screen.

Grab the details from the http://sql.azure.com portal and feed them in and bask in the awesomeness that is Houston.

From here you can create tables, views, stored procedures, create and execute queries. It even has a cube you can spin around.

THIS POSTING IS PROVIDED “AS IS” WITH NO WARRANTIES, AND CONFERS NO RIGHTS