BrandPost: The case for using purpose-built databases for modern applications

For almost three decades, people typically built applications against a single database. In the 1980s, client-server was introduced and apps started to become more distributed in nature, but the underlying data model was predominately structured, and the database was often a monolith.

Source: BrandPost: The case for using purpose-built databases for modern applications

The introduction of Internet-enabled applications changed the demands companies place on their databases. The latency requirements are much lower, and they're expected to be able to handle millions of transactions per second, with many millions of people using the app simultaneously across the globe. These expectations have resulted in developers rethinking how they architect their application. They are now building and using databases that can run faster and are more scalable than ever, thanks to the cloud. Traditional relational databases, once the default choice, are now just one option for building highly resilient, scalable applications more economically.

Two key technology shifts have led to the rise of the “purpose-built” database. First, data volume requirements are increasing from gigabytes and terabytes to petabytes, and sometimes exabytes. The same old tools that worked in the past will not work in this new world of data that we’re in.

Second, the rise of microservices enabled developers to begin separating large, monolithic applications into smaller, modular, independent components. This approach allowed them to deliver specialized functionality and increase the speed of release, because changes to any component are easier to make. This allows developers to do what they do best: break large apps into smaller services and pick the right purpose-built database for the job.  

Developers looking for the right purpose-built database should consider three factors:

• Application workload: Understand the type of data being stored and their access patterns. These fall into one of three categories: transactional (for a high number of concurrent applications), analytical (aggregating and summarizing large sums of data, which operate on many more rows per query), and caching (for read-heavy workloads that require faster load times to improve response times for end users).
• Type of data: Understand the types of entities and their relationships.

• Relational databases normalize data into separate data tables.
• A key-value or wide column is designed for scale, with data split across multiple storage nodes.
• A document data model is for large records that assemble heterogeneously for frequently accessed data.
• A graph data model emphasizes relationships between data to find relationships. Examples here include social network relationships or fraud-detection services.
• Performance and scale requirements: Consider not just the speed of the database, but also how it will serve end users. Developers should ask whether the database will be customer-facing versus just used by internal users. Geographically, if a database is closer to users, lower response times are achievable.

Amazon Web Services (AWS) provides developers with purpose-built databases across a range of services. For example:

• Amazon Aurora is a MySQL and PostgreSQL-compatible relational database built for the cloud, offering performance and availability of commercial-grade databases at one-tenth the cost. When Samsung moved over a billion users across three continents from Oracle to Amazon Aurora, it improved latency and scalability while lowering monthly costs.
• Amazon DynamoDB is a key-value and document database that offers high performance and scale for NoSQL databases. Disney+ is using DynamoDB to ingest content, metadata, and billions of customer actions each day to enable Disney+ viewers to add content to “watch lists” or start viewing a video on one device and pick it up on another.
• Amazon ElastiCache provides fully managed in-memory data store to power real-time applications with sub-microsecond latency. It offers caching for high-volume workloads and is compatible with Redis or Memcached. In-home fitness company Peloton is utilizing ElastiCache to deliver customizable rider data for its community of users riding together on its leaderboard, which requires low latency, real-time processing, and high elasticity.
• Amazon Neptune is a cloud-based, purpose-built high-performance graph database engine that can store billions of relationships and query the graph in milliseconds. Siemens Energy used Neptune to build a Turbine Knowledge Graph to visualize connections between similar parts across its entire fleet of gas turbines. This helped save up to 1,500 hours of manual effort in managing spare parts and configurations.
• The Amazon DocumentDB database service is purpose-built for JSON data management at scale, designed from the ground up for scalability and durability for mission-critical MongoDB workloads. Woot! replaced their aging, self-managed product catalog database running on MongoDB 2.2 with Amazon DocumentDB and cut their infrastructure costs for the database by 82%.
• Amazon Keyspaces (for Apache Cassandra) is a wide column database built for scale and performance that is used to run Cassandra workloads on AWS using the same application code without provisioning, patching, or managing servers. HERE Technologies look to use Keyspaces to deploy Cassandra in a few clicks and free up their developers to focus on innovation instead of managing infrastructure.

Purpose-built databases are a critical part of data-driven transformation because they enable development teams to pick the right technology for specific business needs, without any tradeoffs on functionality, performance, or scale. This approach gives organizations the ability to innovate quickly to address changing business or customer demands.

Learn more about ways to reinvent your business with data.