I love how Microsoft and Azure is making developers' lives so much easier. Consider the title of this blog post. Now try to think how you would solve this problem 2 or 3 years ago. How much code, infrastructure and data would have to be produced in order to generate a text moderation service that scales, performs and works reliably every single time! Fast forward to today and solving the exact same problem takes no less than 100 lines of code by using the tools that MS has made available to our disposal. The heavy lifting is done by the [Azure Cognitive Services](https://azure.microsoft.com/en-us/services/cognitive-services/" target="_blank) which offers a set of tools that perform incredibly advanced tasks using Machine Learning and Artificial Intelligence behind the scenes. This service relies on a wealth of data but, in the end, as developer you're mainly concerned with consuming the services within your own solutions.
As it happens, I was reading about Azure Cognitive Services the other day and I thought that I could put a Proof of Concept together for one of our customers that's looking to make use of the moderation service. You can find more about the Content Moderation Service here which allows both machine and human moderation of text and images. The data is analysed for inappropriate language in submitted text and inappropriate content in the case of images.
To interact with the service you need a Moderation Service account and then you can either use the .NET Client or the REST API.
Creating the Text Moderation Service
The scenario we're going to work with is the following: Create an Azure Function with an HTTPTrigger that accepts text posted in the body of the request and then moderates the text. Depending on the content, it returns a response with the severity of the inappropriate content and the offending words, if any are found.
To get started we first need to create a Content Moderator Account. Head to the following URL https://westeurope.contentmoderator.cognitive.microsoft.com/ and Sign Up for a new account
At the next step we're asked to create a Team. This is where we can allocate human resources to perform manual moderation if we need to.
Once your Team is successfully created, you need to head into Options -> Credentials where you can take a note of your API Key.
There's also a link that explains how the API works. To save you the trouble, I've provided the [API reference docs](https://docs.microsoft.com/en-us/azure/cognitive-services/content-moderator/try-text-api" target="_blank) and [API Playground](https://westus.dev.cognitive.microsoft.com/docs/services/57cf753a3f9b070c105bd2c1/operations/57cf753a3f9b070868a1f66f" target="_blank) here to save you some time. Before starting with the code, this is a great place to get up-to-speed with how the API works and the various capabilities using the online Playground tools.
Prerequisites
There are some pre-requisites if you wish to follow along:
- .NET Core 2.0
- Azure Function Core Tools (v2)
- Azure Functions for Visual Studio Extension (v2)
- Visual Studio 2017 (15.3 or later) if you're on Windows
A wrapper around the Moderation API
I DID create a helper. I can feel your judging eyes, but it's cool, because I have a reason. First of all, I wanted to decouple the code that does the heavy lifting from the Azure Function implementation. This is a recommended practice as it allows us to unit test the code in isolation and keep the code inside the Function nice and lean. In addition, creating a NETSTandard library allows us to port that code easily, as I did with my Console app within the same repo :). So what's in the helper class you may ask? The helper assembly consists of 3 main classes:
ModerationHelper.cs: performs the call to the Moderation service and returns the appropriate responseModerationResponse.cs: the object to deserialise the Moderation service responseModerationResult.cs: the class used to return the results to the user or calling API
Note: The assembly has only one dependency to
Newtonsoft.Jsonand I chose explicitly a lower version (9.x) to avoid any issues with the Azure Functions Runtime which hasn't caught up yet.
You may have noticed that I'm passing the ModerationHelper.cs constructor expects an HttpClient to be passed in. The reason why we do this is because we don't to instantiate an HttpClient with every function call! There are numerous posts written around issues socket depletion etc. and even the Azure Functions docs make a mention in the ["best practices" section](https://docs.microsoft.com/en-us/azure/azure-functions/functions-best-practices#use-async-code-but-avoid-blocking-calls" target="_blank) and the [Improper Instantiation post](https://docs.microsoft.com/en-gb/azure/architecture/antipatterns/improper-instantiation/" target="_blank). In our case, the Azure Function reuses the same HttpClient and passes it to the helper class to perform the appropriate HTTP calls to the Moderation Service API.
Putting it all together in the Azure Function
The Azure Function accepts a POST that contains the text to be moderate, it calls the helper to do the moderation and returns the response to the caller. For this post we will use Visual Studio 2017 to create and run the Azure Function locally using the following steps:
In Visual Studio create a new project: File -> New Project -> Cloud -> Azure Functions
Make sure you give your project a meaningful name and in the next window select the following options
- Runtime: v2
- Trigger: HttpTrigger
- Storage: None
- Access Rights: Function
Next we need to open the Function.cs or whatever name the Function file has to add the following code:
We also need to ensure that we resolve all missing dependencies, i.e the Moderation (helper assembly). We can now run the Function and call it using your tool of choice (PostMan, Fiddler, etc). I use a handy Google Chrome extension. The first picture show the Azure Functions runtime running the Function locally:
Calling the Function using my handy extension produces the following result
Code repository
As always, I've made a full working solution available to you on Github. If you want to grab the code and skip some steps, feel free to do so. That's what Open Source is all about.
GitHub Repo: [Serverless Content Moderation](https://github.com/cmatskas/ServerlessContentModeration" target="_blank)
Conclusion
I find it constantly incredible how empowered we, developers, are today and how using building blocks (existing services) we can create very powerful, scalable and performant solutions. In this instance, with less than 100 lines of code I was able to put together a text moderation service running on top of serverless infrastructure, with Azure Functions. No servers, no frameworks, no complicated decisions to be made. In the end, we had to pick our language of choice and create a Cognitive Service account and write a few lines of code to bring all of these together.
I hope that this blog post will inspire you to go out there and find out what problems you can solve using the powerful building blocks in your disposal? Don't reinvent the wheel, take the one that's already out there and make it better!