With the exponential speed in creation of data over the internet, organisations find it difficult to store, transfer,segregate, analyse and squeeze the information for their best interests. The advent of Big Data technology has enabled organisations to retrieve relevant data (even of complex nature and in different formats) and has successfully encountered the data analysis issue through advanced analytics, thus helping storage of huge data and understanding them to devise future business strategies.
COSO IT strongly believes that 3V's framework explaining Big Data by Doug Laney is intended to help companies in recognizing the potential of Big Data for augmenting profit volumes and performance levels while effectively dealing with the challenges of managing petabytes of unstructured data.
As per Doug Laney, factors such asserting up virtual business, global level business expansions and the need for maximum trend information is driving businesses to understand the need for BigData technology. When key business data such as current and historical records of external and internal working, market trends, product feedback, change in organisational processes, periodical changes in the market reputation, etc., is readily accessible, then enterprises can bring greater level of consistency in their operations. In a bid to consolidate systems and develop accurate databases that enable in attaining greater operational, collaborative and analytical consistencies along with effective risk handling, Doug Laney propounded the 3-D Data Management: Controlling Data Volume, Velocity and Variety.
Since most businesses are reaping the benefit of e-commerce channels due to lower investment aspects, they have widened their outreach, thus making way for multi-fold rise in data collection. Since the data keeps on piling upat exponential rate, it demands purchasing of additional storage. This data,however, loses its value relatively with increase in data, therefore, putting burden on the business' finances. To counter this issue, enterprises can implement tiered storage systems, limiting storage to relevant data only,limiting analytic structured data to percentage form, profiling data sources,frequent monitoring of data usage or outsource the whole process to a professional Big Data management company.
While majority of the data is generated through e-commerce, it has caused a surge in the Point of Interaction (POI)speed and even the pace data, which supports interactions and is produced through interactions. Generally, it is the POI performance, which sets the parameter for a business' competitive performance and its ability to manage the data velocity. The data velocity management is therefore, very crucial and thesis why, companies implement variety of Big Data solutions such as setting up Operational Data Stores (ODSs), caches to provide instant access to transaction related data, point to point data routing and designing architectures to attains balance between data latency with application data needs and decision periods.
One of the major challenges thatenterprises suffer while managing this humongous data is its variety in termsof inconsistencies in data semantics, non-compatible data formats, unstructureddata alignment, etc. Moreover, increased partnerships, M&A activities andimplementation of software that are incompetent in handling the serious datastorage and analysis tasks, add to more variety of challenges. Big Datatherefore, poses as a smart approach in dealing with this variety issue andincorporates unique techniques such as data profiling, data access middleware,metadata management solutions, advanced indexing techniques, distributed querymanagement, XML-based data format and Enterprise Application Integration (EAI).