Understanding Windows Azure Storage–Tables

Windows Azure is the “Operating System for the Cloud” that provides building blocks for developers to write scalable applications. Windows Azure Storage allows developers to store the data in the cloud. The data can be accessed from anywhere at any time, and store any amount of data.  It is important to note that SQL Azure is the different from Azure Storage Services . The SQL Azure is essentially SQL Server in the Microsoft Cloud space (with limitations).

I will be only discussing Tables in this post. I have also provided a walkthrough which covers how to use Table Storage API using an Console application. I will cover Blob and Queue in future posts

Windows Azure Storage Services Overview

The following types of storage are available in Windows Azure Storage Services:

1. Blob – storage for large data items

2. Table – structured storage

3. Queue – asynchronous messages to enable service communication

Tables

It is worth pointing out that Azure Table supports highly scalable tables in the cloud. It supports up to 100 TB of data theoretically. The Tables contain set of Entities. Table names are scoped by Storage Account. We can have many tables for a Storage Account.

Entities (Rows) and Properties (Columns)

The Entities (equivalent to a “row”) are basic data items stored in the table. The Entities contain set of Properties. The Properties (equivalent to a “Column”) are <name,value> pairs. A rich set of data types are supported for Property values. Each Entity has two properties, namely “Partition Key and Row Key”. These two keys form the unique key for the entity.

Storage Account

A valid storage account is required to access Windows Azure Storage.  The account name is part of the host name in the URL. The host name for accessing tables is <accountName>.table.core.windows.net.

PartitionKey and RowKey Properties

It is very important to understand that choosing a PartitionKey is critical for the application to scale. This is because Windows Azure allows tables to scale out to thousands of storage nodes by distributing Entities in the table. When distributing the entities, it is desirable to ensure that a set of entities always stay together on a storage node.  An application controls this set by choosing an appropriate value for the PartitionKey property in each entity

The entities within the same partition are stored together.  This allows efficient querying within a partition.  Furthermore, your application can benefit from efficient caching and other performance optimizations that are provided by data locality within a partition.

As mentioned previously, PartitionKey and RowKey for the unique key for the Entity.

Walkthrough

I will now walkthrough a sample console application in which we are going use Table Services API to create a Entity, and will be adding couple of entities. Before we do that, you need to have valid Windows Azure Account. If you don’t have a account, there are couple of options.

1. You can get a free Windows Azure Pass from here. No credit card required!

2. You could use Storage Emulator. It is important to understand the differences between Windows Azure Storage Emulator and Windows Azure Storage Services.

Create a Console Project

I will first create a Console Project and add necessary references.

Once the project is created, go to Project Properties and change the Target Framework from .NET Framework Client Profile to .NET Framework 4 as shown below.

Add the following references to the project.

1. System.Configuration

2. System.Data.Services.Client

3. Microsoft.WindowsAzure.StorageClient (note this should be available in C:\Program Files\Windows Azure SDK\v1.3\bin)

Add ExpenseEntity class

Let us now add a new Entity class called ExpenseEntity by using Add->New Item menu. Once the class is created, make the class “public”.  Also add the following ‘using’ statements.

using Microsoft.WindowsAzure.StorageClient;

Now derive the class from Microsoft.WindowsAzure.StorageClient.TableServiceEntity. Note that this TableServiceEntity class have already defined key properties PartitionKey and RowKey. Add the following code snippet into the ExpenseEntity class.

As discussed before, choosing the PartitionKey and RowKey is application’s responsibility. I have used month for PartitionKey so that data for each month is in one partition. I have used date + Guid for RowKey.

Add ExpenseDataSource class

Let us now add a new class called ExpenseDataSource by using Add->New Item menu. Once the class is created, make the class “public”.  Also add the following using statements.

using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.StorageClient;

In this class, we are going setup the Storage Account, create the table and have couple of methods to query and add Expense Entity.

Add the following variables to the class:

private static CloudStorageAccount storageAccount;                                                         private TableServiceContext context;

Add the following code in the static constructor to initialize the storage account and create the table.

 

It is important to understand the FromConfigurationString method. It actually does not directly read from the configuration table. We have to call SetConfigurationSettingPublisher in CloudStorageAccount class to set up the Action<> delegate which will be ultimately called to get the connection string. So we have to place this call in main() method in the Console Application.

We will now setup the TableServiceContext in the constructor.

this.context = new TableServiceContext(storageAccount.TableEndpoint.AbsoluteUri, storageAccount.Credentials);

this.context.RetryPolicy = RetryPolicies.Retry(3, TimeSpan.FromSeconds(1));

Now add the methods to add Expense Entity and select entities.

public void AddExpenseEntity(ExpenseEntity newItem)
      {

          this.context.AddObject("Expenses", newItem);
          this.context.SaveChanges();
      }

      public IEnumerable<ExpenseEntity> Select()
      {
          IQueryable<ExpenseEntity> expenses =  this.context.CreateQuery<ExpenseEntity>("Expenses");
          var results = from g in expenses
                        where g.PartitionKey ==
              DateTime.UtcNow.ToString("MM")
                        select g;
          return results;

      }

Now that we are done with ExpenseDataSource class, let us add some code in main() to add entities and select entities.

Setting up the app.config

Before we do that, it is important to setup the Azure account information in configuration file (app.config). We can an add an entry to <appSettings> to setup the account information.

<appSettings>
    <add key="DataConnectionString"        value="DefaultEndpointsProtocol=https;AccountName=”<accountName>”;AccountKey=”<key"> />  </appSettings>

You need to replace the <accountName> and <key> with the correct data. This key can be retrieved from windows.azure.com. 

Main()

Add the following code in main().

//configure storage account
CloudStorageAccount.SetConfigurationSettingPublisher((configName, configSettingPublisher) =>
                    {
                        var connectionString = ConfigurationManager.AppSettings[configName];
                        configSettingPublisher(connectionString);
                    }

  

//add couple of expense entries
           List<ExpenseEntity> expenses = new List<ExpenseEntity>
           {
               new ExpenseEntity() { Amount=100, Category="Dining", ExpenseDate=DateTime.Now.Date },
               new ExpenseEntity() { Amount=250, Category="Movie", ExpenseDate=DateTime.Now.Date }
           };

As mentioned previously, it is important to call SetConfigurationSettingPublisher at the start to read configuration information. We have then created two sample entity (rows).

Let us now add these entities to the table “Expenses”.

ExpenseDataSource dataSource = new ExpenseDataSource();
           foreach (ExpenseEntity expenseEntry in expenses)
               dataSource.AddExpenseEntity(expenseEntry);

 

Let us know retrieve the inserted entities and display.

foreach (ExpenseEntity item in dataSource.Select())
           {
               Console.WriteLine(string.Format("PartitionKey={0}, RowKey={1}, Amount={2}, Category={3}, ExpenseDate={4}",
                                                item.PartitionKey, item.RowKey, item.Amount, item.Category, item.ExpenseDate));
           }

Now run the application and you can see that we are able to add and retrieve the entities from Windows Azure Table Storage.

Summary

In this post, we have seen the basics of Windows Azure Table storage, setting up Storage Account and how to create and add entities to the table. In future posts, I will cover other storages like blob and queues. I will also cover some advanced topics in Table Storage.