Embed facial recognition into SharePoint

Overview

Face API is an AI service that analyses faces in images. Face API is one of the useful offerings of MS Azure Cognitive Services. The Face API perceives faces and attributes in an image. It recognizes the range of facial expressions like happiness, contempt, neutrality, and fear; and recognition and grouping of similar faces in images.

In this article, we will have an overview of the Face API, build, and use it in our applications.

Set up Face API in MS Azure

Let us start by building our first building block in MS Azure Cognitive Services by Face API. Follow the below steps to set up Face API.

Create a resource

1. Login to Microsoft Azure Portal (https://portal.azure.com).
2. Click Create a resource.
3. Select AI + Machine Learning.

4. Select Face.

   

Create a Face API

1. Fill in the details to create the Face API.

   

   The parameters are as follows:

   • Name: Unique name of Face API.
   • Subscription: Select Azure subscription to create Face API under it.
   • Location: The location to host Face API.
   • Pricing tier: Select tier as per features you want to use. More details at https://azure.microsoft.com/en-us/pricing/details/cognitive-services/face-api/
   • Resource group: Select an existing or create a new resource group.
2. Click Create.

3. Once the deployment finishes, the Face API resource will be available to use.

   

4. Note down the API Key and endpoint to authenticate your applications and start sending calls to the service.

Test the Face API

We will use the API Console to quickly try the API without writing any code.

1. From the left menu, select Quick start.
2. Click API Console.

3. Select the testing console in the region where you created your resource.

   

4. The selection of the region will form the request URL.
5. Specify the resource name and query parameters.

6. Specify the API key in Headers.

   

7. In the request body, provide the URL of the image containing Face.

   

8. Click Send.

9. The response will be shown as follows:

   

Set up SharePoint Library

Create a SharePoint library with below schema:

Set up Power Automate Flow

Now we will set up Power Automate flow to integrate Face API for analyzing an image uploaded to the SharePoint library.

Follow the below steps to set up Power Automate flow:

1. On the SharePoint library, click Automate > Power Automate > Create a flow.

   

2. Select a template “When a new file is added in SharePoint, complete a custom”.

   

3. Verify your connection to SharePoint.

   

4. Click Continue.

5. Add an activity “Face API”, select action “Detect faces (preview)”.

   

6. Specify the Connection Name, API Key, and Site URL from MS Azure Face API service.

   

7. Click Create.

8. Specify the Image Url as Link to item.

   

The Challenge

Now, if you run the flow, it fails with below error:

BadRequest: Invalid image URL or error downloading from the target server. Remote server error returned: “Response status code does not indicate success: 403 (FORBIDDEN).”

The reason is Face API in MS Azure subscription is not able to connect to SharePoint which is in another domain.

The Solution

We will create a storage account inside MS Azure to temporarily hold the image uploaded to SharePoint.

1. In the MS Azure portal, click Create a resource.
2. Under Storage, click Storage account - blob, file, table, queue.

3. Enter the name, subscription, Location, and the Access tier.

   

4. Click Create.

5. Once the storage account is ready, create the container inside it.

   

6. Click Create.

Return to Flow

Now let’s use this container as temporary storage for uploaded images to the SharePoint library.

1. Get the file content to upload to the container.

   

2. Add an activity “Azure Blob Storage”, select action “Create blob”.

   

3. Specify the connection name, Storage account name, and Storage account access key from MS Azure.

   

4. In the Create blob, specify container name as folder path, file name as Blob name, and file content as the Blob content.

   

5. Add an action “Detect faces (Preview)”. Compose the Image Url as the URL of the storage account + Path to the uploaded file.

   

6. Add an action “Update file properties”.

   

Test the Solution

In the SharePoint library, upload a human image. This will trigger a flow. A flow will call Face API for analyzing the image uploaded to SharePoint. The flow will then update the returned metadata to the SharePoint column inside the library with gender and age.

But I am just 21! 😊

Summary

Azure Face API analyses faces in images. It recognizes the range of facial expressions like happiness, contempt, neutrality, and fear; and recognition and grouping of similar faces in images.

Use Cases

The Face API can be useful in the below scenarios:

• Find missing persons
• Recognize VIPs

References

• Face recognition (https://azure.microsoft.com/en-in/services/cognitive-services/face/)
• Face API Pricing (https://azure.microsoft.com/en-in/pricing/details/cognitive-services/face-api/)