Modernizing Enterprise Data Architecture: From Traditional Data Warehouses to Scalable Information Platforms
Article Sidebar
Main Article Content
Abstract
Enterprise data architecture is undergoing a significant transformation as organizations increasingly require scalable, flexible, and intelligent systems to manage growing volumes of structured, semi-structured, and unstructured data. Traditional data warehouses have played a central role in enterprise reporting, business intelligence, and historical data analysis; however, their limitations in scalability, real-time processing, data integration, and support for advanced analytics have created the need for more adaptable information platforms. This paper examines the evolution of enterprise data architecture from conventional warehouse-centric models toward modern, cloud-native, and distributed data platforms. It explores key architectural approaches, including data lakes, data lakehouses, cloud data warehouses, data mesh, and data fabric, highlighting their capabilities in data storage, integration, governance, and analytical processing. The study also discusses enabling technologies such as distributed computing, automated data pipelines, metadata management, application programming interfaces, and artificial intelligence-driven analytics. Particular attention is given to interoperability, data quality, security, privacy, governance, and cost optimization as essential considerations in designing sustainable enterprise information ecosystems. Furthermore, the paper presents a conceptual framework for modernizing legacy data infrastructure through phased migration, architectural standardization, workload optimization, and continuous performance monitoring
Article Details
References
Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., & Zaharia, M. (2010). A view of cloud computing. Communications of the ACM, 53(4), 50–58. https://doi.org/10.1145/1721654.1721672
Armbrust, M., Ghodsi, A., Xin, R., & Zaharia, M. (2021). Lakehouse: A new generation of open platforms that unify data warehousing and advanced analytics. Proceedings of the Conference on Innovative Data Systems Research (CIDR). https://www.cidrdb.org/cidr2021/papers/cidr2021_paper17.pdf
Batini, C., & Scannapieco, M. (2016). Data and information quality: Dimensions, principles and techniques. Springer. https://doi.org/10.1007/978-3-319-24106-7
Bernstein, P. A., & Haas, L. M. (2008). Information integration in the enterprise. Communications of the ACM, 51(9), 72–79. https://doi.org/10.1145/1378727.1378745
Codd, E. F. (1970). A relational model of data for large shared data banks. Communications of the ACM, 13(6), 377–387. https://doi.org/10.1145/362384.362685
Dean, J., & Ghemawat, S. (2008). MapReduce: Simplified data processing on large clusters. Communications of the ACM, 51(1), 107–113. https://doi.org/10.1145/1327452.1327492
Dehghani, Z. (2022). Data mesh: Delivering data-driven value at scale. O'Reilly Media.
Dixon, J. (2010, October 14). Pentaho, Hadoop, and data lakes. James Dixon's Blog. https://jamesdixon.wordpress.com/2010/10/14/pentaho-hadoop-and-data-lakes/
Elmasri, R., & Navathe, S. B. (2016). Fundamentals of database systems (7th ed.). Pearson.
Fowler, M., & Lewis, J. (2014). Microservices: A definition of this new architectural term. MartinFowler.com. https://martinfowler.com/articles/microservices.html
Garcia-Molina, H., Ullman, J. D., & Widom, J. (2008). Database systems: The complete book (2nd ed.). Pearson.
Gray, J., & Reuter, A. (1993). Transaction processing: Concepts and techniques. Morgan Kaufmann.
Hey, T., Tansley, S., & Tolle, K. (Eds.). (2009). The fourth paradigm: Data-intensive scientific discovery. Microsoft Research.
Inmon, W. H. (2005). Building the data warehouse (4th ed.). Wiley.
Khatri, V., & Brown, C. V. (2010). Designing data governance. Communications of the ACM, 53(1), 148–152. https://doi.org/10.1145/1629175.1629210
Kimball, R., & Ross, M. (2013). The data warehouse toolkit: The definitive guide to dimensional modeling (3rd ed.). Wiley.
Kleppmann, M. (2017). Designing data-intensive applications: The big ideas behind reliable, scalable, and maintainable systems. O'Reilly Media.
Kreps, J., Narkhede, N., & Rao, J. (2011). Kafka: A distributed messaging system for log processing. Proceedings of the NetDB Workshop.
Marz, N., & Warren, J. (2015). Big data: Principles and best practices of scalable real-time data systems. Manning Publications.
Melnik, S., Gubarev, A., Long, J. J., Romer, G., Shivakumar, S., Tolton, M., & Vassilakis, T. (2010). Dremel: Interactive analysis of web-scale datasets. Proceedings of the VLDB Endowment, 3(1–2), 330–339. https://doi.org/10.14778/1920841.1920886
Stonebraker, M., & Çetintemel, U. (2005). One size fits all: An idea whose time has come and gone. Proceedings of the 21st International Conference on Data Engineering, 2–11. https://doi.org/10.1109/ICDE.2005.1
Strengholt, P. (2023). Data management at scale: Best practices for building large-scale data solutions. O'Reilly Media.
Zaharia, M., Chowdhury, M., Franklin, M. J., Shenker, S., & Stoica, I. (2010). Spark: Cluster computing with working sets. Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing.
Zaharia, M., Xin, R. S., Wendell, P., Das, T., Armbrust, M., Dave, A., Meng, X., Rosen, J., Venkataraman, S., Franklin, M. J., Ghodsi, A., Gonzalez, J., Shenker, S., & Stoica, I. (2016). Apache Spark: A unified engine for big data processing. Communications of the ACM, 59(11), 56–65. https://doi.org/10.1145/2934664
Zikopoulos, P., Eaton, C., deRoos, D., Deutsch, T., & Lapis, G. (2012). Understanding big data: Analytics for enterprise class Hadoop and streaming data. McGraw-Hill.