Enriching SharePoint Image Libraries with Azure Cognitive Services
- Published on
- Reading time
- Authors
- Name
- Simon Waight
- Mastodon
- @simonwaight
The ability to locate content in SharePoint via search has been continually improving over the years, seeing big jumps in impact with the release of features such as Delve and OCR in OneDrive for Business.
Indexing file content from Document Libraries has improved no end, but one item I see as a gap is extracting inferred meta-data out of images stored in SharePoint.
In this post, and the associated screencast, I'll take a look at how we can utilise several of the capabilities of Azure Cognitive Services and Azure Logic Apps to quickly add value to image data that you upload to SharePoint.
Baseline Services
In order to use this solution you will need:
- A SharePoint Online (or SharePoint on-premises) Document Library to which you can authenticate a Logic App SharePoint Connector.
- Images against which you can train and test a Custom Vision classifier. The Open Images Dataset is a good starting point if you want source images by category.
- An Azure Subscription (Trial or Paid) - sign-up for your free trial at http://azure.com/free. This demo will run happily within a Trial account.
- A trained Azure Cognitive Services Custom Vision classifier. For the purpose of my demonstration I used just the free tier for Cognitive Services which should work just as well for you.
Building the Logic App
The best thing about Logic Apps is not only are they serverless, but they are almost entirely "codeless" as well. Rather than me describe in detail the steps, I've included the following items to help you get going.
1. The designer view of the Logic App
You can see the overall flow below.
2. The Logic App exported to GitHub
Grab the code from here: https://github.com/sjwaight/SharePointCognitiveServices.
Note: this doesn't come with the Connections which you will need to manually recreate using your own SharePoint and Cognitive Services API details.
In the image below you can see how to retrieve the GUID for a List in SharePoint Online - get to this page by clicking on "Library settings" from the settings menu (the cog) that is near your username in the Office 365 header.
3. A Screencast showing how this all hangs together
Note that in the screencast there's a slight difference to my published Logic App above... see if you can find it 😜
Well, there we have it - how you can (fairly) easily add more value to the image data you store in SharePoint. Note that you could apply this approach to documents stored in Google Drive or Dropbox, as well as using many of other categories of Azure's Cognitive Services to enrich data you already have!
One final item before I wrap up - if you want to learn more about Azure Cognitive Services then you can do it for free and without the need for an Azure Subscription. Microsoft Learn has a range of modules you can complete that covers this area. Access them online using just a browser.
Happy Days 😎