How should I store my data?

Today companies have to capture, organise, access and store an overwhelming amount of data.

Even organisations whose products and services are not based in the digital arena need to properly store information.

There are many systems that enable users to effectively manage crucial data, but selecting the right one is not always easy.

Alicornio Africa, specialists in data warehousing and business intelligence, offers some insight into two popular integration systems and the challenges that may arise in either.

Data Warehouse (DW)

This is known as the traditional data warehouse.

“Most of the data stored in the traditional data warehouse is sourced from the operational systems in organisations and consists of mostly structured information.”

According to Alicornio it contains accurate, consistent, historical and legally binding data.

This allows users to access information that can ultimately improve business decisions and performance.

DW Challenges

Some of the problems that arise within data warehouses is inadequate documentation and poor planning.

A lack of skills and understanding at management level can also hamper the functioning of the system and lead to unrealistic expectation with regards to what can be delivered within specific timeframes.

“It is important that these issues need to be resolved before considering enhancing the traditional data warehouse with a data lake.  Extended Data Warehousing might be called for but basic reasons for having access to information remains unchanged:  supporting the business goals and objectives, easy to find good quality, reliable information that backup the business goals and objectives. “

Extended Data Warehouse (EDW)

“This is the new modern data warehouse environment which caters for the best of both worlds.”

The benefit of the EDW is that it can cater for structured operational data, semi-structured and unstructured data in the Extended portion of the Data Warehouse called the Data Lake.

“In addition it also makes provision for new requirements to enable querying and interrogation of this data.”

EDW's can combine data from the traditional data warehouse with data residing in the data lake.

EDW Challenges

Due to its capacity to house structured, semi-structured and unstructured data the EDW system is extremely complex.

This creates challenges as very few people have the skills to operate EDW's. In addition “staff turnaround causes loss of organisational knowledge”.

“The advent of big data necessitates new skills and new methods.”

Do you need to be upskilled in the area of Extended Data Warehousing?

Join the Concepts, Design and Modelling for Extended Data Warehousing course offered by Alicornio Africa.

By Cindy Payle - Portal Publishing