Azure Custom Vision - Machine Learning Image Recognition

Azure Custom Vision Overview

Azure Custom Vision services is great for many organizations that deal with a high volume photos. Whether you belong to Marketing or Human Resources departments, it may become a very tedious job to organize these photos in a consumable way. In the past, photos were often identifiable and retrievable by a complex folder structure used to organize content.

These organizations now have an opportunity to leverage technology using Artificial Intelligence (AI) to automatically predict tags based on the content that is discovered in the photo. Using Azure Custom Version Services, Microsoft Flow / Logic Apps, and SharePoint, we can implement a solution to store, identify, and tag metadata to images through deep machine learning image recognition. With this metadata, we will have the ability to filter, sort, group, and search for images easily.

Configure Custom Vision Service (Azure)

Creating Azure Resource

Navigate to your Azure Subscription (https://portal.azure.com).
Create a new Resource Group.
Search and add the Custom Version service resource.

Configure resource properties (example):

Name: Predictive Image Tagging
Subscription: Contoso Pay-as-you-go
Location: North Central US
Prediction Pricing Tier: F0 (2 transactions per second, 10k transactions per month)
Training Pricing Tier: F0 (2 tranasctions per second, 2 projects)
Resource Group: DEV Resource Group

Select Create

Create a new project

Navigate to https://customvision.ai/
Create a New Project

Configure the project properties (example)
- Name: Predictive Tags
- Description: Automatically tag images loaded into SharePoint
- Resource Group: DEV Resource Group
- Project Types:
  - Classification (Tags whole images)
  - Object Detection (Find location of content within image)
- Classification Types:
  - Multilabel (Multiple tags per image)
  - Multiclass (Single tag per image)
- Domains:
  - General
  - Food
  - Landmarks
  - Retail
  - Adult
  - General (Compact)
  - Landmarks (Compact)
  - Retail (Compact)

Setup Workflow (Microsoft Flow)

Create a new trigger: When a file is created in a folder
Create a new action: Predict Tags from an Image

Note: You will be prompted for the Predictive Key and Project ID when establishing a new connection. This key can be going to your project settings under the Account information.

Training

The artificial intelligence requires your to provide samples of photos with tags manually to “train” the system. The more images and keywords you are able to train with, the higher the certainty is with the predictive tagging.

Upload a minimum of 5 images for training and specify a tag or keyword (i.e. “Cow”)

Select Train
Make the training the Default Iteration to be used by the workflow

Testing Results

Upload an image to a SharePoint picture library

Trigger workflow and verify results
- Tag: Cow
- Probability: 99.9%

Azure Custom Vision – Machine Learning Image Recognition

Azure Custom Vision – Machine Learning Image Recognition

Azure Custom Vision Overview