Unit Testing Azure EventHub, Stream Analytics Job and Storage Table - unit-testing

Question

I am working on this project which uses EventHub -> Stream Analytics Job -> Storage Table / Blob structure and I want to write a couple of unit tests for it.
I can test the EventHub Sender status to see if queries have the expected behavior, but how can I check if the data is being sent to the Table Storage, since the whole process doesn't happen instantly and there is a pretty long delay from the moment I hit the EventHub and the moment the data is being saved in Storage.

Answer 1

First create a new Azure Table storage account and then create a new Azure table within that account. In your Stream Analytics job add a new output for Table storage. When you setup the Output details, you will need to specify the storage account, account key, table name, and which columns names in the event will represent the Azure Table partition and row keys. As an example, I set mine up like this:
After the output is setup, you can create a simple Stream Analytics query that maps input events from Event Hub to the Azure Table output. I also have an Event Hub input named 'eventhub' with Send/Listen permissions. My query looks like this:
SELECT
*
INTO
tableoutput
FROM
eventhub
At this point hit the 'Start' button in Azure portal to run the Stream Analytics job. To generate the events, you can follow the instructions here but change the event message to this:
string guid = Guid.NewGuid().ToString();
var message = "pk,rk,value\n" + guid + ",1,hello";
Console.WriteLine("{0} > Sending message: {1}", DateTime.Now, message);
eventHubClient.Send(new EventData(Encoding.UTF8.GetBytes(message)));
To eyeball the Azure Table results, download a tool like TableXplorer and enter the storage account details. Double click your Azure Table and you should see something like the following. Keep in mind you may need to periodically hit F5 on your TableXplorer query for 10-60 seconds until the data gets pushed through. When it shows up it will look like the following:
For programmatic unit testing you will need to push the partition key / row key values generated in your Event Hub code into a data structure and have a worker poll the Azure Table using point queries. A good overview on Azure Table usage is here.