One common challenge for IT departments is how to effectively manage Oracle enterprise data warehouses (EDW), given the data volume growth rates and its’ effect on management, capex, and performance.
As data volume increases, inefficiencies in your enterprise data warehouse (EDW) can prevent you from realizing the full value of your data. Extract-transform-load (ETL) processes consume more CPU capacity, leading to higher licensing and management costs. Scheduled downtime to manually manage databases interrupts availability. Backup and archive cycles take increasingly longer, significantly slowing access times for users. Query operations must sort through massive amounts of data—much of it cold, infrequently accessed, or irrelevant—resulting in degraded performance. Optimizing your EDW by offloading cold and unused data to an open-source, horizontally scalable, NoSQL database can help you overcome these challenges to reduce costs, deliver faster access to data, and provide better information for decision making.
Is there an alternative way to reduce CPU utilization by BA/BI processes, or save old or cold data in data warehouses? Is it best to offload certain workloads and data from Oracle to NOSQL? Is there a reliable and convenient way to achieve that?
In a typical EDW environment, data volumes grow faster and faster while Extract-transformation-load (ETL) processes consumes more and more CPU cycles of Oracle database. Backup and archiving processes could take longer and longer time to complete. That would affect performance of user processes. Critical user queries may not be able to complete within acceptable length of time. Upgrading database infrastructure is the costly way to deal with this kind of challenges as the total cost of the infrastructure including database licenses would increase dramatically.
With Hitachi Vantara, we enable customers to offload cold data from EDW to a less cost prohibitive solution. Pentaho Data Integration (PDI) provides unique functionalities. It can work with many data sources including an RDBMS product like Oracle, and a NOSQL product like MongoDB. With Pentaho the process of offloading data from Oracle EDW to MongoDB is transparent and seamless.
Professional services are available from Hitachi Vantara to simplify the implementation for complex environments which require multiple data sets being offload. Potential user errors may be avoided as well.
Below is the overall architecture design for this solution. It is preferred but not required that the Oracle data warehouses are running on Hitachi UCP CI for Oracle infrastructures.
For more details, please visit here and a full copy of reference architecture document can be downloaded.