Azure Functions custom logging with AppInsights
You can tell I'm working on [Azure Functions](https://azure.microsoft.com/en-us/services/functions/" target="_blank) from the frequency of posts. This one focuses on logging. Functions by default generate plenty of logs which you can view in the portal or download using the [Kudu REST API](https://github.com/projectkudu/kudu/wiki/REST-API" target="_blank). This is great and it comes out of the box without you having to configure a single thing.
However, what if you have some custom, complex logic that you want to capture and log. And I guess that in most instances there will be more than one Function per AppService and potentially a lot more applications and services deployed on Azure. So how to do manage the logs for all these. How do you consolidate and how do you make sense of all the telemetry?
Well Azure's got your back with [Application Insights(AppInsights)](https://azure.microsoft.com/en-us/services/application-insights/" target="_blank). AppInsights started as a standalone tool that developers would opt-in (some times) to get useful telemetry from their applications. However, fast-forward to today and AppInsights has become an invaluable tool for developers and support staff alike. AppInsights helps you triage, prevent and react to issues, performance problems, crashes, availability etc. Yes, there are other products in the market but there's one thing they miss. Full integration with all your Azure services and products! And, of course, managing all these events in one central place, your AppInsights dashboard.
It's only logical then that our Azure Functions should take advantage of this service. A few reasons you should consider AppInsights:
- monitor performance issues
- exception tracking
- custom events
- analytics
Again, you could easily achieve this with other tools, but AppInsights brings all your logging under one central dashboard. DevOps becomes a lot easier with a centralized command post and don't forget the extremely powerful analytics which are part of the service!
Note: This blog post will soon may be redundant because the team has plans to make AppInsights a first-class citizen and full integrated into the service. Until then, this post can fill in the gap.
Integrate AppInsights (.NET) with you Azure Function
1. Create an AppInsights Instance
If you don't have AppInsights running on Azure already, you can spin an new one up from the portal. It should take a few minutes to create. Once up and running, navigate to the Properties blade and copy the Instrumentation Key
The InstrumentationKey is required to instantiate a TelemetryClient and is used to capture your events and send them to your unique instance of AppInsights. This key can be shared by many applications and Functions. Remember that your AppInsights is your central dashboard. This key is your common point of reference.
2. Add AppInsights to the Function
For the purpose of this post I will use the Azure Portal Functions editor. The changes to the code could as easily be implemented using Visual Studio Code or the Visual Studio 2015 with the Azure Functions extension.
Functions can reference external NuGet packages in the exact same way that .NET applications do. To add a reference to a NuGet package, you need to click on Files -> Add New and add a project.json file.
Open the project.json file and add the following code:
{
"frameworks": {
"net46":{
"dependencies": {
"Microsoft.ApplicationInsights": "2.2.0"
}
}
}
}
Make sure you save the file. If you check the logs you should see the NuGet packages downloaded successfully.
The logs should look like this:
2017-01-13T19:06:40.523 Restoring packages.
2017-01-13T19:06:40.523 Starting NuGet restore
2017-01-13T19:06:41.617 Restoring packages for D:\home\site\wwwroot\TimerTriggerCSharp2\project.json...
2017-01-13T19:06:41.836 GET https://api.nuget.org/v3-flatcontainer/microsoft.applicationinsights/index.json
2017-01-13T19:06:42.374 OK https://api.nuget.org/v3-flatcontainer/microsoft.applicationinsights/index.json 529ms
2017-01-13T19:06:42.390 GET https://api.nuget.org/v3-flatcontainer/microsoft.applicationinsights/2.2.0/microsoft.applicationinsights.2.2.0.nupkg
2017-01-13T19:06:42.819 OK https://api.nuget.org/v3-flatcontainer/microsoft.applicationinsights/2.2.0/microsoft.applicationinsights.2.2.0.nupkg 416ms
2017-01-13T19:06:43.006 Installing Microsoft.ApplicationInsights 2.2.0.
2017-01-13T19:06:43.728 Committing restore...
2017-01-13T19:06:43.728 Writing lock file to disk. Path: D:\home\site\wwwroot\TimerTriggerCSharp2\project.lock.json
2017-01-13T19:06:43.759 D:\home\site\wwwroot\TimerTriggerCSharp2\project.json
2017-01-13T19:06:43.759 Restore completed in 2169ms.
2017-01-13T19:06:43.775
2017-01-13T19:06:43.775 NuGet Config files used:
2017-01-13T19:06:43.775 C:\DWASFiles\Sites\cmtestfunction\AppData\NuGet\NuGet.Config
2017-01-13T19:06:43.775
2017-01-13T19:06:43.775 Feeds used:
2017-01-13T19:06:43.775 https://api.nuget.org/v3/index.json
2017-01-13T19:06:43.791
2017-01-13T19:06:43.791 Installed:
2017-01-13T19:06:43.791 1 package(s) to D:\home\site\wwwroot\TimerTriggerCSharp2\project.json
2017-01-13T19:06:43.806
2017-01-13T19:06:43.806
2017-01-13T19:06:43.838 Packages restored.
2017-01-13T19:06:44.146 Script for function 'TimerTriggerCSharp2' changed. Reloading.
2017-01-13T19:06:44.208 Compilation succeeded.
2017-01-13T19:07:12.085 Script for function 'TimerTriggerCSharp2' changed. Reloading.
2017-01-13T19:07:12.147 Compilation succeeded.
2017-01-13T19:10:00.002 Function started (Id=<removed>)
With the NuGet packages installed successfully, we can now reference the external libraries in the code. Next, open the run.csx and replace the contents of the file with the following code:
Save and Run. Depending on the timer you've set up, your Function should fire in the next few minutes. Make sure you keep an eye in the logs to ensure that there are no errors. On a separate tab/window open your AppInsights and wait for the first telemetry to show up.
If all goes well and you followed the instructions, you should soon see the telemetry from your function in the Portal. The telemetry within the AppInsights portal should look like this::
Notice that in this instance, I'm looking into the exceptions raised from my Function - even the fake ones :)
I also like to create custom charts and save them as a favourite so that I can quickly look into all events raised by a specific function. AppInsights is highly customizable and gives you the flexibility to create your own fully tailored views. Image below as a reference
Pro-tip: you can pin these views to your main dashboard to save you having to navigate 3 blades deep in AppInsights.
Another way to drill into your custom logging is through the Analytics. You can access the Analytics page from within your AppInsights from the Menu bar at the top:
With AppInsights Analytics, you can go crazy querying your logs using the SQL-like, lightning fast (Kusto) query engine to analyse billions of rows of data. The editor is great and provides fantastic intellisense to help you write complex queries. You can even output straight to a graph in real time in the results window!
Conclusion
With a few steps and some configuration changes, we were able to add invaluable logging to our Function using the AppInsights .NET SDK. You can also achieve the same using the Node.js SDK if your Functions are written in Node.
There is a downside to this approach and that's the fact that AppInsights lack the full integration with Functions. As such we can't capture valuable telemetry from things happening outside our Functions. For example failed HTTP requests to the Function due to load etc. Ideally, the telemetry should provide information on successful and failed calls to our API due to performance, security etc. However, this is already in the team's backlog and they're working on it. In the meantime, I prefer to have some telemetry over none.
Let me know in the comments if you have any questions.