Category: IT Professional

Build a serverless pipeline to analyze streaming data using AWS Glue, Apache Hudi, and Amazon S3

To deliver on these requirements, organizations have to build custom frameworks to handle in-place updates (also referred as upserts), handle small files created due to the continuous ingestion of changes from upstream systems (such as databases), handle schema evolution, and compromise on providing ACID guarantees on its data lake

To consume this streaming data, we set up an AWS Glue streaming ETL job that uses the Apache Hudi Connector for AWS Glue to write ingested and transformed data to Amazon S3, and also creates a table in the AWS Glue Data Catalog.

Continue reading

green and yellow printed textile

Automate Amazon Redshift load testing with the AWS Analytics Automation Toolkit

In this post, we demonstrate the use of the AWS Analytics Automation Toolkit for JMeter load tests on cloud benchmark data, using Amazon Redshift as a target environment

To use the AWS Analytics Automation Toolkit to run a JMeter load test, deploy the toolkit with the JMeter option, load data into your Amazon Redshift cluster, and customize the default test plan as you see fit.

Continue reading

Leveraging OpenTelemetry to democratize Cloud Spanner Observability

However, some of our customers would like to have the flexibility of consuming Cloud Spanner metrics in their own observability tooling, which could be either an open source combination of a time-series database like Prometheus coupled with a Grafana dashboard, or it could be a commercial Application Monitoring (APM) tool like Splunk, Datadog, Dynatrace, NewRelic or AppDynamics

Continue reading

Deploying service-mesh-based architectures using AWS App Mesh and Amazon ECS

Service-mesh-based architectures provide visibility and control for microservices (a group of loosely coupled services that function together to make an application operate) by providing a consistent way to route and monitor traffic between them

In this post, we’ll explain how to use AWS App Mesh to provide visibility and control for microservices by providing a consistent way to route and monitor traffic between them.

Continue reading

Salt Security Survey Surfaces API Security Weaknesses

A survey of more than 250 security, application and DevOps executives and professionals published today by Salt Security found 95% of respondents experienced a security incident involving application programming interfaces (APIs) in the last 12 months, with 62% reporting they slowed down the rollout of an application because of API security concerns.

Continue reading

How reliable is your IoT platform?

Question #1: Can you prove that your IoT platform offers at least 99.9% reliability for data queried out of your digital twin?

Question #3: Can you provide performance metrics proving your platform’s reliability during a worst-case IoT scenario, like a massive disconnect of IoT devices?

Continue reading

Designing IoT Solutions With Microsoft Azure

Getting the fundamental knowledge on how Azure creates, manages, and monitors multiple IoT and Edge devices at a single place using IoT Hub along with providing additional features like Time Series Insights, storage containers, event-based triggers using LogicApp actions and operations, simulating devices using Azure bash commands in the shell, gaining knowledge on the hot, warm, and cold data path analytics, sending, and receiving telemetry data to various IoT devices and so on

Continue reading

1 … 174 175 176 177 178 … 186