Big Data enigma has been doing rounds for a while and various industries have been anticipating quantum leaps in their businesses riding on this wave and have even factored in its Future Financial Impacts. On a broad survey of the early adopters of the technology, one finds that a strong impact on Financial Statements is already being felt by most, albeit only on the Overhead Costs end. The key reason for this has been a co-existence of the preceding BI-CRM technologies built around the Data Warehouse systems and an incomplete switch to the Big Data Systems. Not that they are not complementary but the dual cost centre handling needs a more mature architecture. Besides, on close observation one finds that most of the organizations do not even possess enough data quantum, types and velocity to feed into the voracious Big Data Systems and savor the promises it has to offer. Yet, some other organizations are still caught between the In-house versus Off-shore stack decisions and have not still relied on the sea of advancements that is taking place on the cloud front. Such organizations often also have found it difficult to ingest the non-proprietary content and open-source tools or complementary stacks. However, the variety of choices and its innovations are far from being ignorable with each progressing day.
Nevertheless there are certain emerging Big Data trends that may provide advanced choices, mitigate apprehensions and complement technologies to hasten further adoption and productivity of the technology
- Hybrid Cloud adoption: Big data systems have so far co-inhibited the workspace with multiple other technologies and may be so until the organizations adapt to the complex big data systems. In several cases a part of the business data is retained at premises and non-core data is worked upon at offshore locations. The hybrid data clouds are implemented both at the offshore and in-house work-zones to divide and manage the data. Cloud adoption helps organizations in strategizing cost factors, risk mitigation, and scaling database to a great extent, with its further depths yet to be explored. The C-suite and IT heads of all industry verticals are diligently designing the Big Data Ingestion Pathway following the cloud adoption and currently, over 50% of global enterprises are relying on cloud platforms. The cloud-based PAAS (Platform-as-a-service) is befitting the market requirements of Big Data Science and delivers versatile solutions with Opex driven models. The high capability solid state drives coupled with in-memory technology pairs with the cloud-based Opex model to deliver the most optimal results. The global public cloud market is expected to cross the US $170 billion (As per Forbes) marks by 2018 and the cloud adoption rate will near 22% (According to Gartner) in the coming year.
- Deployment of unprecedented content and advanced Logical data model (LDM): The spurt of public content has been substantial in this decade and along with the owned data of the organizations, it becomes the most important input into the big data systems. It includes content that spans social, video, audio data and thousands of widgets of financial, economic, entertainment, GPS, health area significance. Such content along with advanced LDMs which logically structure the business information of the organization into decision making variables through the future big data systems. The quantum variety and velocity of this kind of data are unprecedented as it captures pre, post and during transaction level information. Advance LDM are adaptive and they evolve themselves as per the changing customer or stakeholder microclimate.
- Integration of other major technologies including AI and IOT (Internet of Things): It provides an ecosystem to embed multiple technologies, platforms, and systems under the same umbrella. As a result, it helps in the adoption of new technologies like AI and IOT. While AI cuts across several industry verticals, the best utilization of the integration of the same is in a customer interface oriented environment as well as an operations-centric industry. The AI systems simulate best impact response to any systematic or unsystematic stimuli and in the process consume terabytes of data across millions of transactions. It creates outputs that are simple at the front end but complex and data-hungry at the backend. The retail industry, as well as e-commerce, provides one of the best case scenarios for the big data success. IOT has the maximum adoption in industry operations and security systems, and the gain from it is still being quantified because of the monumental benefits that organizations are already deriving out of it. Over 40% (Statista, UK) of Industries will adopt IOT in some form by 2020. IOT systems are capable of running at microsecond level and as a result, the data is also generated at the same frequency and it often produces significant unstructured data also (as in the case of video data for image analytics). That is where it integrates well with machine learning algorithms which could all be housed comfortably in a big data environment.
- End of proprietary platform era and open source distribution to be the complementary backbone to Big Data Systems: Last couple of decades were dominated by various proprietary platforms and tools from companies such as IBM, Oracle, Microsoft, SAP, SAS, Teradata that made development and implementation restricted to an environment and somewhat scalable within the same. However, last few years have already seen a decline in adoption as well as scaling-up of proprietary platforms because of the plethora of choices among the high-quality, big variety open source distribution systems, which have been challenged on the grounds of data security . A lot of open source material is getting organized as systematic notebooks for e.g. A Jupyter notebook, which encompasses not only Julia, python, and R but also houses 25+ more languages. Such kind of assimilation of important programming and implementation systems under one umbrella add to the big data flexibility and complimentarily.
By: Dr. Kamaljit Anand, Managing Partner and Regional Head – Europe & Mainland Asia (EMEA), KiE Square
