Large enterprises need to consider many factors when modernizing their existing monitoring solution. Enterprises can achieve centralized monitoring management by using Azure Monitor features. This example scenario illustrates enterprise-level monitoring that uses Azure Monitor.
Source: Enterprise monitoring with Azure Monitor
Enterprise teams have different workloads, such as Windows, Linux, SQL, identity-based workloads, virtual desktop infrastructure (VDI), containers, and web apps. These workloads can be running in any cloud providers, on-premises, or a combination. With such a vast array of workloads in different environments, cloud-based monitoring is complex.
Enterprise-level monitoring must also cover governance, operational best practices, effective cost management, and workspace security. Monitoring must provide enough flexibility to set up and manage team environments, and let teams manage themselves with some degree of control.
Other critical monitoring design factors include:
- How to spread Log Analytics workspaces across different geographical regions or teams.
- Monitoring the workspaces themselves, as well as their workloads.
- How to charge back different teams to optimize overall costs.
- How to visualize and possibly archive collected data.
- Creating separate dashboards for operations, apps, and different teams.
- Giving leadership enough visibility into the right set of information.
Potential use cases
This solution can help with the following use cases:
- Consolidated monitoring for different cloud and on-premises workloads.
- Monitoring for container, Azure SQL, and Azure Virtual Desktop workloads.
- Expanded monitoring scope, such as connecting Monitor to Microsoft Sentinel.
- Hybrid and heterogenous cloud monitoring across networks, identity providers, operating systems, and other domains.
Architecture
Download a Visio file of this architecture.
- This architecture follows a resource-context log model. Every log record that an Azure resource emits automatically associates to the resource. This model helps to separate workspaces that collect and ingest from different app owners.
- Different workloads across the enterprise have separate workspaces. Configuring different workspaces gives teams autonomy over their own data, and provides a separate cost overview per workspace.
- Platform-as-a-service (PaaS) services like Azure Web Apps and Azure Functions Apps add configuration for Application Insights within their workspaces.
- For identity, on-premises Active Directory and cloud identity providers each have their own workspaces.
- Azure Virtual Desktop, Azure Pipelines, SQL workloads, apps in Azure Kubernetes Service (AKS) and Azure Web Apps, and other PaaS services all have their own workspaces.
- Each workspace has its own set of configured alerts. Azure Logic Apps and Azure Automation provide advance alerting and remediation. Logic Apps provides integration with IT Service Management (ITSM) tools.
- A set of on-premises virtual machines (VMs) connects through Azure Arc, providing an end-to-end Azure management plane. You can also use Azure Arc to connect infrastructure-as-a-service (IaaS) resources that run in a third-party cloud.
- Custom logging captures information about third-party virtualized environments, and collects custom operating system, software, and application logs.
- Log Analytics Workspace Insights provides comprehensive workspace monitoring. Using a single workspace to store collected data from all resources aligns with the IT organization's operating model. This workspace gives the central team an overview of usage, cost, and performance for all the workspaces. The central workspace respects scoping and role-based access control (RBAC) based on the resources. Log Analytics Workspace Insights has its own separate set of alerts.
- Log Analytics provides further integration by exporting workspace data for archiving or analytics. Archiving data to cool-tier storage saves costs. You can use archived data for further analytics by creating datasets that feed into machine learning models.
- Monitor connects to security information and event management (SIEM) tools like Microsoft Sentinel to create larger enterprise security datastores.
- Power BI and Monitor Workbooks provide data visualization and dashboard capabilities.
Components
This architecture includes the following components:
Monitor components
Azure Monitor collects, analyzes, and acts on telemetry data from cloud and on-premises environments. This solution uses the following Monitor components and features:
- Monitor Metrics collects numeric data from monitored resources into a time series database. Metrics in Monitor are lightweight and support near real-time scenarios, so they're useful for alerting and fast detection of issues.
- Monitor Logs collects and organizes log and performance data from monitored resources. You can consolidate data from multiple sources, including Azure Original Postlatform-logs-overview" target="_blank" rel="noreferrer noopener">platform logs, into a single workspace. You can analyze the data by using a sophisticated query language in Log Analytics.
- Azure Monitor agent can send data to both Monitor Logs and Monitor Metrics. The Azure Monitor agent uses configurable Data Collection Rules (DCRs), and doesn't require workspace keys to connect.
- Application Insights monitors live applications on a wide variety of platforms across cloud, hybrid, and on-premises environments. Application Insights automatically detects performance anomalies. Application Insights includes powerful analytics tools to help you understand usage and diagnose issues.
- Azure Virtual Desktop insights uses Monitor for Azure Virtual Desktop to help IT professionals understand their Azure Virtual Desktop environments.
- Container insights monitors the performance and health of Kubernetes clusters and other container-based workloads.
- Network insights provides a comprehensive view of health and metrics for all deployed network resources.
- SQL insights (preview) monitors health and help you diagnose problems and tune performance for any product in the Azure SQL family.
- VM insights monitors the performance and health of VMs and virtual machine scale sets. VM insights include running processes and dependencies on other resources.
- IT Service Management Connector (ITSMC) provides a bi-directional connection between Azure and supported ITSM tools to help you resolve work items faster.
- Monitor Workbooks provide a flexible canvas to analyze multiple Azure data sources and combine them into interactive visual reports.
- Log Analytics creates and runs queries on Monitor Logs data in Log Analytics workspaces. This solution uses the following Log Analytics features:
- Log Analytics agent collects monitoring data from cloud and on-premises operating systems and VM workloads, and sends it to a Log Analytics workspace.
- Azure Active Directory Monitoring routes Azure Active Directory (Azure AD) activity logs to a Log Analytics workspace.
- Log Analytics gateway sends data to Azure Automation and Log Analytics workspaces for computers that can't directly connect to the internet.
- Service Map uses the Log Analytics agent to automatically discover application components on Windows and Linux systems, and map the communication between services.
- Alert Management helps you analyze all the alerts in your Log Analytics workspaces.
- Log Analytics data export (preview) continuously exports data from selected tables in a Log Analytics workspace. Data can export to an Azure storage account or Azure Event Hubs.
- Log Analytics Workspace Insights provides comprehensive monitoring of all Log Analytics workspaces. Workspace Insights gives a unified view of workspace usage, performance, health, agent, queries, and change logs.
Other components
In this solution, Monitor supports or integrates with the following Azure and Microsoft services:
- Azure Arc simplifies governance and management by delivering a consistent multi-cloud and on-premises management platform.
- Azure Automation delivers cloud-based automation, operating system updates, and configuration to support consistent management across environments. Change Tracking tracks changes in cloud and on-premises VMs to help you identify software issues. Change Tracking forwards the data to Monitor Logs and stores the data in a Log Analytics workspace.
- Azure ExpressRoute extends on-premises networks into the Microsoft cloud. ExpressRoute uses private connections with the help of connectivity providers.
- Azure Data Lake Storage provides secure, scalable, cost-effective cloud storage for big data analytics.
- Azure Functions is a serverless solution that implements readily available code blocks called functions. Functions run on demand and scale up automatically.
- Azure Kubernetes Services (AKS) is a fully managed Kubernetes service to easily deploy and manage containerized applications.
- Azure Load Balancer evenly distributes incoming network traffic across backend resources or servers.
- Azure Logic Apps is a cloud-based platform for creating and running automated workflows. Logic apps can integrate apps, data, services, and systems.
- Azure Resource Manager provides a management layer and templates for creating, updating, and deleting resources in your Azure account.
- Microsoft Defender for Cloud is part of Microsoft Defender for Cloud, a unified infrastructure security management system.
- Microsoft Sentinel is a cloud-native, scalable, security information and event management (SIEM) and security orchestration automated response (SOAR) solution.
- Azure SQL family of SQL database services provides a consistent, unified Azure SQL experience. Azure SQL has a full range of deployment options, from edge to cloud.
- Power BI is a collection of software services, apps, and connectors that turn your data sources into coherent, visually immersive, and interactive insights.
Alternatives
You can use some monitoring alternatives along with or instead of Monitor.
System Center Operations Manager
System Center Operations Manager offers flexible, cost-effective infrastructure monitoring. Operations Manager provides comprehensive monitoring for private and public datacenters and clouds. Operations Manager helps ensure the predictable performance and availability of important applications.
To maintain your existing Operations Manager investment, you can integrate Operations Manager with your Log Analytics workspaces. You can use Monitor logs and extended capabilities while still using Operations Manager for these functions:
- Monitoring the health of your IT services
- Maintaining integration with your ITSM solutions for incident and problem management
- Managing the lifecycle of agents deployed to on-premises and public cloud IaaS VMs.
For more information, see Connect Operations Manager to Azure Monitor.