{"id":22807,"date":"2019-06-14T08:30:31","date_gmt":"2019-06-14T15:30:31","guid":{"rendered":"https:\/\/insidebigdata.com\/?p=22807"},"modified":"2019-06-15T10:01:47","modified_gmt":"2019-06-15T17:01:47","slug":"the-future-of-open-source-big-data-platforms","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2019\/06\/14\/the-future-of-open-source-big-data-platforms\/","title":{"rendered":"The Future of Open Source Big Data Platforms"},"content":{"rendered":"\n

Three well-funded startups \u2013 Cloudera Inc., Hortonworks Inc., and MapR Technologies Inc. \u2014 emerged a decade ago to commercialize products and services in the open-source ecosystem around Hadoop, a popular software framework for processing huge amounts of data. The hype peaked in early 2014 when Cloudera raised a massive $900 million funding round, valuing it at $4.1 billion.<\/p>\n\n\n\n

The recent struggles of Cloudera and MapR have made many headlines and left some wondering what this means for the future of big data,” observed Unravel Data<\/a> CEO Kunal Agarwal. “Is enterprise interest in data waning? Not at all. These companies have faltered as a result of big data\u2019s rapid transition to the public cloud, leaving little growth potential for platforms like these that were designed for on-premises deployments. Big data is a better fit in the cloud due to its highly elastic compute requirements. In addition, modern data systems are becoming more complex, and they\u2019re more difficult to manage on-premises than in the cloud. There\u2019s a new data stack emerging and Hadoop is no longer the definitive big data technology: technologies like Spark and Kafka are rising to support modern data applications that use artificial intelligence and machine learning. Hadoop won\u2019t disappear and not every data workload will go to the cloud, but the public cloud and technologies like Spark will increasingly define big data and any vendors who don\u2019t aggressively support them will continue to suffer.<\/p><\/blockquote>\n\n\n\n

Hortonworks went public in 2014 and Cloudera followed in 2017, but both saw shares tumble as market competition intensified and customers began moving rapidly to the cloud. Cloudera and Hortonworks merged last fall, but the stock of the combined entity has continued to fall, slicing market value by half over the last seven months. MapR announced its intentions to go public more than four years ago, but never followed through, opting instead to raise two more rounds of venture funding in 2016 and 2017. It was recently revealed that MapR may cut up to 122 jobs and shut down its Santa Clara, California headquarters if it can\u2019t secure additional funding.<\/p>\n\n\n\n

“The recent news around Cloudera and MapR is stirring up a lot of debate around the future viability of Hadoop, and really all open-sourced frameworks for managing big data workloads,” observed Chandra Ambadipudi, CEO of Clairvoyant<\/a>. “A big factor is that Hadoop was greatly underestimated by the market regarding the resources needed to manage and leverage it. Hadoop did deliver on its promise as a low cost, scalable and robust open-source solution, but the talent and number of data engineers required to manage its complexity, and shortage thereof, has come to a head.”<\/p><\/blockquote>\n\n\n\n

With Cloudera now being the remaining significant Hadoop company, past the MapR news dust up, the following are some insights and thoughts about the future of open source big data platforms being tied to the cloud (and cloud giants like Microsoft, AWS, Google):<\/p>\n\n\n\n