As shown in the following diagram, the FSx for Lustre CSI driver plugin is deployed to an Amazon EKS cluster to dynamically provision the FSx for Lustre file system with a given PVC. The Spark application driver and
Tag: IT AI ML
Build a pseudonymization service on AWS to protect sensitive data, part 1
The pseudonymization service is built using AWS Lambda and Amazon API Gateway . The request response model of the API utilizes Java string arrays to store multiple values in a single variable, as depicted in the
Machine learning operations (MLOps) v2
The data and model monitoring and event and action phases of MLOps for NLP are the key differences from classical machine learning. Classical machine learning: Time-Series forecasting, regression, and classification
Powered by Snowflake: Why the Hunters Team Embraces a Connected App Model
My co-founder and I started Hunters in 2018 with a mission to revolutionize security operations. We designed the Hunters security operations center (SOC) platform to automatically identify threats, enable
Real-time scoring of R ML models
This reference architecture shows how to implement a real-time web service in R using Azure Machine Learning running in Azure Kubernetes Service (AKS). Azure Machine Learning lets you define the number of R
Accelerate machine learning with AWS Data Exchange and Amazon Redshift ML
In this post, we show you the process of subscribing to datasets through AWS Data Exchange without ETL, running ML algorithms on an Amazon Redshift cluster, and performing local inference and production. Create
Real-time machine learning architecture on Azure
This article expands on Citizen AI with the Power Platform, which provides a high-level example of a low-code, end-to-end lambda architecture for real-time and batch data streaming. Consume: A real-time
Introduction to Amazon QuickSight ML Insights
In 2018, ML Insights for QuickSight (Enterprise Edition) was announced to add machine learning (ML)-powered forecasting and anomaly detection with a few clicks. Amazon QuickSight was launched in
Closer to AGI?
Gato is a model that can solve multiple unrelated problems: it can play a large number of different games, label images, chat, operate a robot, and more . Building bigger and bigger models in hope of somehow achieving
Snowpark for Python: Bringing Efficiency and Governance to Polyglot ML Pipelines
Machine learning (ML), more than any other workflow, has imposed the most stress on modern data architectures. Its success is often contingent on the collaboration
Optimize AI/ML workloads for sustainability: Part 2, model development
If you missed the first part of this series, we showed you how to examine your workload to help you 1) evaluate the impact of your workload, 2) identify alternatives to training your own model, and 3) optimize data processing. These rules monitor your workload and will automatically stop a training job
Understanding Deep Learning Algorithms that Leverage Unlabeled Data, Part 1: Self-training
We introduce the augmentation graph on the population data, a key concept which allows us to formalize the input consistency regularizer and motivates natural assumptions on the data distribution. In the rest of this blogpost, we’ll
How Cynamics built a high-scale, near-real-time, streaming AI inference system using AWS
This post shows how we used AWS managed services and in particular Amazon Kinesis Data Streams and Amazon EMR to build a near-real-time streaming AI inference system serving hundreds of production customers in both AWS commercial and government environments, while seamlessly auto scaling
Reuse recommender systems and algorithms from R with Azure
The functions included with these packages were used extensively throughout the code to import the data, create the classification models, and deploy them into production.