Create sample data for Cosmos DB with .NET Core

Aren't you bored of stockmarket ticker and ToDo sample apps and data? Do you long for some more realistic data to build great app samples around it? Well, guess what? Problem solved? How do you feel about Volcanos? If you love them (and fear them) like I do, then this is the blog for you. below I'll show you how to create a small console app that generates sample data and then populates a Cosmos DB database with it.

Prerequisites

Since Cosmos DB is an Azure service, you'll need and Azure Subscription. Grab one for FREE here! For this example we will also be using .NET Core 3.1 so make sure to download it here. Finally, you are free to use the IDE/tool of your choice - I'm using Visual Studio 2019 Preview this time around.

Components

• .NET Core 3.1
• Bogus NuGet package( create fake/sample data with Bogus and .NET)
• Cosmos DB NuGet package

Create a Cosmos DB resource

If you already have a Cosmos DB resource, you can skip to the next section. If not, keep going. I will be using the Azure CLI with CloudShell to create the Cosmos DB instance. If you're using the latest Windows Terminal, CloudShell is already a shell option :) If not, you can use the CloudShell instance on the azure portal or navigate to shell.azure.com to fire up a new instance! You can find the necessary CLI command documentation here.

The script necessary to create a new Cosmos DB instance is:

az login
az account set --subscription <SubscriptionName or Guid>
az cosmosdb create --resource-group <ResourceGroupName> --subscription '<SubscriptionName>' --name <CosmosDBName>

We can now grab the connection string so that we can use it later in our console app

az cosmosdb list-connection-strings --name <yourCosmosDBName> --resource-group <ResourceGroupName>

We've now completed our DB setup :)

Create the console app

As I started down this path, I was lucky to find some existing sample data in this Azure Samples repository. However, there were a couple of issues. 1. There were spaces in a couple of JSON properties and 2. I wanted richer data. So I downloaded the json file locally, did a quick search and replace to remove the spaces and then I started writing the necessary code to to enrich and push the data to my Cosmos DB database.

Next, we will use the dotnet CLI to create a new console app and add the necessary NuGet packages.

dotnet new console
dotnet add project <ProjectName> package Microsoft.Azure.Cosmos
dotnet add project <ProjectName> package Microsoft.Extensions.Configuration.UserSecrets
dotnet add project <ProjetName> package Bogus
dotnet add project <ProjetName> package Newtonsoft.Json

I also copied my sampleData.json file into the project folder and added the following into my *.csproj file so that it be copied to the bin folder:

 <ItemGroup>
    <None Update="sampleData.json">
      <CopyToOutputDirectory>Always</CopyToOutputDirectory>
    </None>
  </ItemGroup>

Finally, from the command line, I executed the following command to initialize the secrets file so that I can store my Cosmos DB connection string outside my source code :)

dotnet user-secrets init
dotnet user-secrets set CosmosDBConnection "<one of the connections retrieved earlier"

And now the code in all its glory

There are a couple of helper methods here to set things up:

The BootstrapConfiguration() method returns an IConfiguration object with all the secrets (well, just one for our example). The PopulateMeasurementData() takes the JSON data we loaded from file and adds some extra measurement data using Bogus. The GetJsonDataAsync() loads the data from file and deserializes it into strongly typed objects. Finally, the BulkUploadDataToCosmosDB() takes our enriched data and bulk uploads it to Cosmos DB. The Main() method ties everything together and adds a bit of logging.

If you want to create sample data on the fly, you are welcome to fork the GitHub repo, add your Cosmos DB connection string and run the app to get some sweet, realistic Volcano data for your app :)