Build and deploy a social media analytics solution

To best address customer needs, organizations need to extract insights from social media about their customers. This article presents a solution for analyzing news and social media data. The solution extends the Azure Social Media Analytics Solution Accelerator, which gives developers the resources needed to build and deploy a social media monitoring platform on Azure in a few hours. That platform collects social media and website data and presents the data in a format that supports the business decision–making process.

Apache®, Apache Spark, and the flame logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. No endorsement by The Apache Software Foundation is implied by the use of these marks.

Architecture

Download a Visio file of this architecture.

Dataflow

1. Azure Synapse Analytics pipelines ingest external data and store that data in Azure Data Lake. One pipeline ingests data from news APIs. The other pipeline ingests data from the Twitter API.
2. Apache Spark pools in Azure Synapse Analytics are used to process and enrich the data.
3. The Spark pools use the following services:
   • Azure Cognitive Service for Language, for named entity recognition (NER), key phrase extraction, and sentiment analysis
   • Azure Cognitive Services Translator, to translate text
   • Azure Maps, to link data to geographical coordinates
4. The enriched data is stored in Data Lake.
5. A serverless SQL pool in Azure Synapse Analytics makes the enriched data available to Power BI.
6. Power BI Desktop dashboards provide insights into the data.
7. As an alternative to the previous step, Power BI dashboards that are embedded in Azure App Service web apps provide web and mobile app users with insights into the data.
8. As an alternative to steps 5 through 7, the enriched data is used to train a custom machine learning model in Azure Machine Learning.
9. The model is deployed to a Machine Learning endpoint.
10. A managed online endpoint is used for online, real-time inferencing, for instance, on a mobile app (A). Alternatively, a batch endpoint is used for offline model inferencing (B).

Components

• Azure Synapse Analytics is an integrated analytics service that accelerates time to insight across data warehouses and big data systems.
• Cognitive Service for Language consists of cloud-based services that provide AI functionality. You can use the REST APIs and client library SDKs to build cognitive intelligence into apps even if you don't have AI or data science skills. Features include:
  • Named entity recognition (NER) for identifying and categorizing people, places, organizations, and quantities in unstructured text.
  • Key phrase extraction for identifying key talking points in a post or an article.
  • Sentiment analysis for providing insight into the sentiment of posts by detecting positive, negative, neutral, and mixed-sentiment content.
• Translator helps you to translate text instantly or in batches across more than 100 languages. This service uses the latest innovations in machine translation. Translator supports a wide range of use cases, such as translation for call centers, multilingual conversational agents, and in-app communication. For the languages that Translator supports, see Translation.
• Azure Maps is a suite of geospatial services that help you incorporate location-based data into web and mobile solutions. You can use the location and map data to generate insights, inform data-driven decisions, enhance security, and improve customer experiences. This solution uses Azure Maps to link news and posts to geographical coordinates.
• Data Lake is a massively scalable data lake for high-performance analytics workloads.
• App Service provides a framework for building, deploying, and scaling web apps. The Web Apps feature is a service for hosting web applications, REST APIs, and mobile back ends.
• Machine Learning is a cloud-based environment that you can use to train, deploy, automate, manage, and track machine learning models.
• Power BI is a collection of analytics services and apps. You can use Power BI to connect and display unrelated sources of data.

Alternatives

You can simplify this solution by eliminating Machine Learning and the custom machine learning models, as the following diagram shows. For more information, see Deploy this scenario, later in this article.

Download a Visio file of this architecture.

Scenario details

Marketing campaigns are about more than the message that you deliver. When and how you deliver that message is just as important. Without a data-driven, analytical approach, campaigns can easily miss opportunities or struggle to gain traction. Those campaigns are often based on social media analysis, which has become increasingly important for companies and organizations around the world. Social media analysis is a powerful tool that you can use to receive instant feedback on products and services, improve interactions with customers to increase customer satisfaction, keep up with the competition, and more. Companies often lack efficient, viable ways to monitor social media conversations. As a result, they miss opportunities to use these insights to inform their strategies and plans.

This article's solution benefits a wide spectrum of social media and news analysis applications. By deploying the solution instead of manually deploying its resources, you can reduce your time to market. You can also:

• Extract news and Twitter posts about a specific subject.
• Translate the extracted text to your preferred language.
• Extract key points and entities from the news and posts.
• Identify the sentiment about the subject.

For instance, to see the latest discussions about Satya Nadella, you enter his name in a query. The solution then accesses news APIs and the Twitter API to provide information about him from around the web.

Potential use cases

By extracting information about your customers from social media, you can enhance customer experiences, increase customer satisfaction, gain new leads, and prevent customer churn. These applications of social media analytics fall into three main areas:

Marketing is an integral part of every organization. As a result, you can use this social media analytics solution for these use cases in various industries:

• Retail
• Finance
• Manufacturing
• Healthcare
• Government
• Energy
• Telecommunications
• Automotive
• Nonprofit
• Gaming
• Media and entertainment
• Travel, including hospitality and restaurants
• Facilities, including real estate
• Sports

Considerations

These considerations implement the pillars of the Azure Well-Architected Framework, which is a set of guiding tenets that can be used to improve the quality of a workload. For more information, see Microsoft Azure Well-Architected Framework.

Reliability

Reliability ensures your application can meet the commitments you make to your customers. For more information, see Overview of the reliability pillar.

• Use Azure Monitor and Application Insights to monitor the health of Azure resources.
• Review the following resiliency considerations before you implement this solution:
• For more information about resiliency in Azure, see Design reliable Azure applications.
• For availability guarantees of various Azure components, see the following service level agreements (SLAs):

Security

Security provides assurances against deliberate attacks and the abuse of your valuable data and systems. For more information, see Overview of the security pillar.

Cost optimization

Cost optimization is about looking at ways to reduce unnecessary expenses and improve operational efficiencies. For more information, see Overview of the cost optimization pillar.

To estimate the cost of this solution, use the Azure pricing calculator.

Operational excellence

Operational excellence covers the operations processes that deploy an application and keep it running in production. For more information, see Overview of the operational excellence pillar.

Performance efficiency

Performance efficiency is the ability of your workload to scale to meet the demands placed on it by users in an efficient manner. For more information, see Performance efficiency pillar overview.

Deploy this scenario

To deploy this solution and run a sample social media analytics scenario, see the deployment guide in Getting Started. That guide helps you set up the Social Media Analytics Solution Accelerator resources, which the architecture diagram in Alternatives shows. The deployment doesn't include the following components: Machine Learning, the managed endpoints, and the App Service web app.

Prerequisites

• To use the solution accelerator, you need access to an Azure subscription.
• A basic understanding of Azure Synapse Analytics, Azure Cognitive Services, Azure Maps, and Power BI is helpful but not required.
• A news API account is required.
• A Twitter developer account with Elevated access to Twitter API features is required.

Contributors

This article is maintained by Microsoft. It was originally written by the following contributor.

Principal author:

Next steps