A big data architecture is designed to handle the ingestion, processing, and analysis of data that is too large or complex for traditional database systems. Often, data from multiple sources in the organization may be consolidated into a data warehouse, using an ETL process to move and transform the source data.īig data solutions. Data in OLTP systems is typically relational data with a predefined schema and a set of constraints to maintain referential integrity. These workloads include online transaction processing (OLTP) and online analytical processing (OLAP). This guide is structured around two general categories of data solution, traditional RDBMS workloads and big data solutions. Instead, solutions are designed around a data pipeline that describes how data flows through a solution, where it is processed, where it is stored, and how it is consumed by the next component in the pipeline.

There are no longer multiple layers of business logic that read and write to a single data layer. The perspective on data in the solution changes as a result. Instead of a single general-purpose database that handles all of a solution's data, polyglot persistence solutions use multiple, specialized data stores, each optimized to provide specific capabilities.

The cloud is changing the way applications are designed, including how data is processed and stored. Learn more about adopting your systems for data governance, analytics, and data management, in Cloud adoption for data management.

Cloud data architect