Companies are investing in large-scale Internet of Things (IoT) projects and deploying global scale IoT platform such as Deutsche Bahn or Carrier. Enterprises are looking for a solution that offers a multi-tenant Single Pane of Glass device lifecycle management (DLM) which caters to both IT and OT operations, says Katja-Maja Kroedel, IoT specialist solution architect, Leo Da Silva, security specialist solutions architect and Hassan Khokhar, senior IoT architect at AWS.
In this blog it focuses on giving perspective guidance on how to architect a multi-tenant Single Pane of Glass IoT Platform for cyber-security posture. Enterprises of all shapes and sizes from different industry can benefit from such platform. From an IT point-of-view this platform would standardise enterprise IoT related cyber-security features such as device on-boarding, visibility and governance. From an OT point of view the platform would accelerate time to production since all the heavy lifting (account management, workload management, security etc.) is baked into the platform from day one.
In this guidance blog we will be referencing several AWS services. These services are integral parts of the reference architectures and practices for the Single Pane of Glass approach.
AWS Organisations is an account management service that enables you to consolidate multiple AWS accounts into an organisation that you create and centrally manage. AWS Organisations includes account management and consolidated billing capabilities that enable you to better meet the budgetary, security, and compliance needs of your business. For more information go to AWS Organisations.
AWS IoT Core lets you connect billions of IoT devices and route trillions of messages to AWS services without managing infrastructure. AWS IoT Core supports a number of communication protocols and connectivity methods. For more information go to AWS IoT Core.
AWS IoT Device Defender is natively integrated with AWS IoT Core, and it is a security service that allows you to audit the configuration of your devices, monitor connected devices to detect abnormal behavior, and mitigate security risks. For more information about go to AWS IoT Device Defender.
AWS Security Hub is a cloud security posture management service that performs security practice checks, aggregates alerts, and enables automated remediation. It provides you with a comprehensive view of your security state in AWS and helps you check your environment against security industry standards and practices. For more information go to AWS Security Hub.
Amazon EventBridge is Amazon EventBridge is a server-less event bus service that you can use to connect your applications with data from a variety of sources. EventBridge delivers a stream of real-time data from your applications, software as a service (SaaS) applications, and AWS services to targets such as AWS Lambda functions, HTTP invocation endpoints using API destinations, or event buses in other AWS accounts. For more information go to Amazon EventBridge.
AWS Lambda is a server-less, event-driven compute service that lets you run code for virtually any type of application or back-end service without provisioning or managing servers.
Use case introduction
The above figure shows a high level view on the issue we want to solve. We are displaying three example workloads: Refinery, Fuel Cells and Lubricants. Each of these use cases has their own IoT deployment in a distinct AWS region. Two different personas are displayed within the architecture: The Business User on a per use case level as well as the IoT platform administrators. Each business user persona needs access to their own IoT workload deployment. In this case the Refinery Business User needs the authentication as well as authorisation to access the Refinery Deployments. The Lubricants Business User needs access to the Lubricants IoT workload, but not others like the Refinery. On the other hand we have the IoT platform admins that need access to all the workloads, no matter the region, account or use case. Additionally, the IoT security admin also will need to access and gain visibility into the security posture of all workloads deployed and be aware of e.g. expiring certificates.
For the above-mentioned use-case we are will lay a foundation of our design by employing a style of tenancy which provides complete isolation of the IoT workloads. The highly isolated tenancy design provides cost, data and workload isolation. This allows easier management of resources deployed in an AWS account and setups the foundation for isolated IoT workloads for our OT business users while providing global insight for the IT Platform admins and security personas. This tenancy style also reduces the blast radius from a security point of view since the business users and their devices are accessible through their own tenant workload. This type of tenancy comes with its own challenges related to meshing of the tenants, cost visibility and implementing Single Pane of Glass IoT platform for global device management.
Control, data and edge plane
From the above illustrations in figure 1 & 2 we can compartmentalise the components in such a way where all control related use-cases are achieved through a single common interface called the IoT platform. This component serves as the Single Pane of Glass IoT device management portal for all personas. Since this component is control related, we can house this component in a conceptual boundary known as “Control Plane”.
The distinct tenant specific workloads component is specified as “IoT workload”. Since these are isolated workloads where devices connect to and send their telemetry these isolated tenant specific components can be housed in a conceptual boundary known as the “Data/Telemetry plane“. All devices managed by individual business deployed across their businesses can be housed in a conceptual boundary known as the “Edge Plane”.
The individual IoT workload can comprise of (n) number of accelerators. These accelerators can perform a function such as provisioning a device, control & commanding a device, patching device, provisioning AWS IoT Greengrass Core etc. To learn more about function or use-case specific accelerator refer to the AWS connected device framework for more information. This framework can serve as the foundational building block for this architecture.
Isolating accounts using AWS organisations
We now extend the guidance through the use of AWS specific services. AWS Organisations in this case allows customers to use organisational units (OUs) that provide capabilities to the accounts within those OUs. All OUs apply their own guardrails for the accounts as well as governance for the tenant accounts. We utilise three different OUs in Figure 4.
1/ Shared services organisational unit
The Single Pane of Glass resides within the Shared Services OU. It has its own account which hosts the aggregated dashboard. The OU in this case provides the capabilities to the own platform and grants access to the different user types to access the data they are allowed to see.
2/ Workloads organisational unit
The workloads OU hosts has multiple accounts, one account per tenant. It permits the users coming from the Single Pane of Glass access to their workloads and the outbound and inbound data from IoT Devices.
3/ Suspended organisational Unit
Workloads in the suspended OU are no longer active but still part of the setup within AWS which allows for later investigation as well as deletion once no longer needed. That suspension can occur automatically base on criteria defined by the system administrators.
Event driven solution
In section we add the use of Amazon EventBridge integrated with AWS IoT Core. That approach allows for an event driven solution which will interact with the control pane or Single Pane of Glass. The control plane account will have Amazon EventBridge set up to send and receive messages to and from the individual workload OU accounts. This integration allows to invoke the different IoT workloads in the accounts and also the collection of data from the individual devices up to a aggregated view in the control plane account. Cross account interactions require specific permission which can be understood in more detail in the Service control policies (SCPs) documentation.
The workload OU accounts subscribe to the messages coming from the control plane side, and vice versa. Each workload is isolated by an individual tenant account, which also allows for cost isolation and thus achieves the tenancy model we needed.
Single Pane of Glass architecture
Finally we can focus on above diagram with all the elements of the Single Pane of Glass architecture.
Starting from the right side we have multiple devices connected to AWS IoT Core. First, we focus our attention to the connection from AWS IoT Core into the workload. As the workload interacts with IoT devices through AWS IoT Core, Amazon EventBridge can be configured to react to specific events. Those events will then be passed onto the Single Pane of Glass accounts, where the user has access only to the relevant data and alarms.
Now we turn our attention to the connection from AWS IoT Core on the right side to AWS IoT Device Defender. Natively integrated with AWS IoT Core, AWS IoT Device Defender will execute auditing and monitoring tasks, reporting any anomalies or non-compliance to the reporting pipeline. The reporting pipeline is composed by an Amazon SNS topic and AWS Lambda function which then deliver the alerts to AWS Security Hub. Respectively, AWS Security Hub is integrated cross account, delivering the alarms to IT administrators and delegating actions to operations if necessary.
This architecture allows the security operations team as well as IoT platform admins access to security insights and findings across the different accounts and regions.
Few examples of deviations that should be shared with security operation teams using AWS Security Hub are:
- MQTT-based data exfiltration: Data exfiltration occurs when a malicious actor carries out an unauthorised data transfer from an IoT deployment or from a device. The attacker launches this type of attacks through MQTT against cloud-side data sources.
- Impersonation: An impersonation attack is where attackers pose as known or trusted entities in an effort to access AWS IoT cloud-side services, applications, data, or engage in command and control of IoT devices.
- Command and control, malware and ransomware: Malware or ransomware restricts your control over your devices, and limits your device functionality. In the case of a ransomware attack, data access would be lost due to encryption the ransomware uses.
If you want to find out more about the different security use cases covered by AWS IoT Device Defender you can access here. Also, feel free to check out the Blog Post that describes in detail how to set up the flow from AWS IoT Core through AWS IoT Devices Defender with the final destination of AWS Security Hub.
In this blog post we walked you through the considerations for building a Single Pane of Glass for your multi-tenant IoT workloads when considering enterprise-wide security operations. With this approach now your IT teams and OT teams can rely on a single place for cyber-security posture, as well facilitate the standardisation of already existing practices and organisation requirements.
For further learning and reading about AWS IoT solutions and ways to improve the overall security of your environment, please visit the following blog posts:
If you want to know more about designing and building a multi-tenant architecture for your AWS IoT environment, you can follow this workshop.
Leo Da Silva
The authors are Katja-Maja Kroedel, IoT specialist solution architect, Leo Da Silva, security specialist solutions architect and Hassan Khokhar, senior IoT architect at AWS.