Abstract

Modern Hybrid Transactional and Analytical Processing (HTAP) systems continuously struggle with data layout conflicts, as row-oriented storage favors transactional throughput while column-oriented layouts excel in analytical execution. Traditional database engines rely on static partitioning or periodic manual re-indexing, which fail to adapt to volatile, real-time workload shifts. This paper introduces an autonomous structural optimization framework that dynamically realigns physical storage layouts based on predictive workload forecasting. By integrating a lightweight, continuous learning agent directly with the database engine's metadata layer, the proposed model evaluates incoming query patterns and predicts workload skew for upcoming execution windows. Instead of maintaining rigid hybrid structures, the framework executes non-blocking, sub-table partition shifts between row and column layouts during low-contention micro-intervals. Experimental evaluations conducted on mixed-workload simulation benchmarks demonstrate a significant reduction in overall query response latency and minimal transactional overhead during structural transitions. The findings indicate that self-optimizing physical layouts offer a viable alternative to resource-heavy hardware scaling in high-volume, dynamic data environments.

Citation details of the article

Journal: International Journal of Applied Mathematics
Journal ISSN (Print): ISSN 1311-1728
Journal ISSN (Electronic): ISSN 1314-8060
Volume: 36
Issue: 3
Year: 2023

Download Section

Download the full text of article from here.

You will need Adobe Acrobat reader. For more information and free download of the reader, please follow this link.

References

1. [1] SAP HANA Architecture Technical Report, "In-memory unified transactional and nalytical processing engines," In Proc. VLDB Endow., vol. 7, no. 13, pp. 1541-1552, 2014.
2. [2] A. Kemper and T. Neumann, "HyPer: A hybrid OLTP&OLAP main memory database system based on virtual memory snapshots," in IEEE 27th ICDE, 2015,pp. 195-206.
3. [3] T. Lahiri et al., "Oracle Database In-Memory: A dual-format in-memory database," in IEEE 31st ICDE, 2015, pp. 1253-1258.
4. [4] C. Diaconu et al., "Hekaton: SQL Server's memory-optimized OLTP engine," in Proceedings of the ACM SIGMOD, 2013, pp. 1243-1254.
5. [5] J. Arulraj, A. Pavlo, and S. R. Dulloor, "Let's talk about storage & recovery overrides in non-volatile memory database systems," in Proceedings of the ACM SIGMOD, 2015, pp. 707-722.
6. [6] M. Stonebraker and U. Çetintemel, "One size fits all: An idea whose time has come and gone," in Proceedings of the 21st ICDE, 2005, pp. 2-11.
7. [7] J. Feser, S. Chaudhuri, and I. Dillig, "Synthesizing specialized storage layouts for relational databases," ACM SIGPLAN Notices, vol. 55, no. 2, pp. 45-58,2019.
8. [8] QuickSel Framework, "Deep learning-driven automated column-selectivity estimation and database layout adaptation," SIGMOD, pp. 112-126, 2020.
9. [9] DBBERT Developers, "Transformer-based language models for autonomous database knob tuning and performance projection," IEEE TKDE, vol. 34, no. 8, pp. 3901-3914, 2022.
10. [10] D. Van Aken et al., "Automatic database management system tuning through large-scale machine learning," in Proceedings of the ACM SIGMOD, 2017, pp. 1009-1024.
11. [11] RL_QOptimizer Group, "Reinforcement learning for dynamic structural query path determination over fixed relations," Journal of Database Research, vol. 18, pp. 89-104, 2022.
12. [12] GRQO Architecture, "Graph reinforcement learning implementations for multi-tenant hybrid data systems," In Proc. VLDB, vol. 17, no. 4, pp. 512-524, 2024.
13. [13] R. Marcus et al., "Neo: A learned query optimizer," Proceedings of the VLDB Endowment, vol. 12, no. 11, pp. 1705-1718, 2019.
14. [14] I. Alagiannis et al., "NoDB: Efficient query execution on raw data files," Communications of the ACM, vol. 61, no. 5, pp. 91-99, 2018.
15. [15] C. Curino et al., "Schism: a workload-driven approach to database replication and partitioning," Proceedings of the VLDB Endowment, vol. 3, no. 1,pp. 48-57, 2010.
16. [16] M. Serafini et al., "Clay: Online adaptive partitioning for transactional workloads," Proceedings of the VLDB Endowment, vol. 10, no. 3, pp. 145-156, 2016.
17. [17] S. Idreos, M. L. Kersten, and S. Manegold, "Database cracking," in Proceedings of the CIDR, 2007, pp. 68-78.
18. [18] P. Holupirek et al., "Stochastic tuning strategies for transactional storage boundaries," Journal of Systems Software, vol. 142, pp. 119-131, 2018.
19. [19] F. Halim, S. Idreos, and P. Karras, "Stochastic database cracking: Towards robust fine-grained adaptive indexing," in Proceedings of the IEEE 28th ICDE, 2012, pp. 8-19.
20. [20] L. S. Schmidt et al., "Efficient and lock-free incremental adjustments in dynamic structural indices," IEEE Transactions on Reliability, vol. 71, pp. 204-216, 2021.
21. [21] P. Boncz, M. Zukowski, and N. Nes, "Ankommen der Vectorwise execution model on modern architectures," IEEE Data Eng. Bull., vol. 35, no. 1,pp. 11-23, 2012.
22. [22] Y. Yuan et al., "Hardware-conscious hybrid processing frameworks for wide-column relational segments," ACM Transactions on Database Systems, vol. 45, no. 3, pp. 1-32, 2020.
23. [23] G. Graefe, "The five-minute rule 20 years later, and how flash memory changes the rules," Communications of the ACM, vol. 52, no. 7, pp. 48-59, 2009.
24. [24] R. Sudhir, "Self-Organizing Data Containers: Cloud-scale metadata-aware expressive partitioning blocks," International Journal on Digital Data Engineering, vol. 22, no. 2, pp. 104-118, 2025.
25. [25] A. Jindal et al., "Selecting tile layouts for big data analytics vectors," in Proceedings of the IEEE 34th ICDE, 2018, pp. 189-200.
26. [26] X. Yu et al., "Tictoc: Time-travel concurrency control for distributed in-memory databases," in Proceedings of the ACM SIGMOD, 2016, pp. 1629-1642.
27. [27] AlloyDB Omni Core Labs, "Adaptive execution mechanisms using automated monte-carlo tree pathselections," Google Cloud Systems Whitepaper, Tech. Rep. v3.4, 2025.
28. [28] B. Bhattacharjee et al., "IBM DB2 analytics accelerator v4.1 architectural deep-dive," IBM Journal of Research and Development, vol. 58, no. 5, pp. 1-12, 2014.
29. [29] T. Neumann and M. Freitag, "Umbra: A disk-based system with in-memory performance," in Proceedings of the CIDR, 2020.
30. [30] Z. Medin et al., "Resource isolation vs physical structure conversion limits in modern multi-tenant environments," Advanced Cloud Infrastructure Analytics Journal, vol. 12, pp. 441-455, 2025.