Obsidian’s journey with Hadoop

Big Data started with Hadoop

Hortonworks was a big data platform provider which spun out of Yahoo! in 2011. The firm was one of the main contributors to the Apache Hadoop codebase. The elephant in the room had many parts to it and Obsidian’s vision started its data journey with a partnership with Hortonworks.

We soon realised that a big data platform was not only about the collection of information about customers, clients and businesses.  The value of digital identity and security against cybercrime and other business attacks can save our customers billions.

Do you see your data in a smarter way and does your big data play to allow for advancement in machine learning and predictive modelling?

“This means companies can use their existing toolsets for what they have been designed to do and not as their data source. It is not necessarily about Big Data but about utilising the data you have at your disposal. With Hortonworks, you can derive value out of your data in an effective manner –smarter data.”

Muggie van Staden | MD 2016

Our customers started realising the value of big data analysis. Hadoop enabled them to process data faster than before and analyse it to make more agile business decisions. While it was an ideal solution for larger companies, the technology was, and is, very accessible for small to medium enterprises.

Hadoop is so powerful that users can extract value out of the analytics using standard spreadsheet software. This means smaller companies can rely on very basic software and leverage Hadoop to assist with analysing its data.

The fintech sectors were facing more frequent fraud challenges, whether through breaches that steal customer data, easy-to-hide criminal transactions, or real-time payments that demand much faster risk analysis.

Industry has turned to machine learning for help. The sheer volume of data that must be analysed exacerbates the problem. Storage limits can require companies to archive historical data, limiting its availability.

By using a fully equipped data platform, businesses can accelerate their speed to analytics and extend their data retention. They can analyse data from the current workday and keep it highly available for years to come.

Preparing for the Data Journey

Collaborate, collaborate and then collaborate some more. The benefits of increasing the ability to harness IoT perspective for understanding the value of data requires teams to be able to work together with a common goal of better information to guide business decisions.

While many companies are often intimidated by the potential complexity of an effective big data environment, the benefits are clear. By becoming more efficient and adaptive to change, real-time analysis of the wealth of data available inside a company can bring a vital competitive advantage in the digital world.

Muggie van Staden | MD Embracing the big data world

Having a unified vision backed by teamwork. With 2020 being the turning point in history, all businesses must ensure that teams are able to perform at their peak and be empowered to act regardless of physical location.

Lessons of a Data Journey

At the time with relatively new technology came the implications of big data rollouts across South Africa and the rest of the world. Storage for clusters in Hadoop and the fear of the unknown risks of cloud made CIOs hesitant to take the leap of faith into Big Data. And yet the biggest hurdle was yet to come.

Another challenge is the lack of big data skills in SA. Granted, many companies believe they understand what is required to make big data work, but this cannot happen without having the right skills in place. As data becomes the currency of the digital world, employees across all business units need to understand how the latest technology works and how to extract meaningful value out of the information they have at their disposal.

Muggie van Staden | MD Taking the big leap

Despite the challenges and many barriers that big data adoption, the banking industry in South Africa has led the race, and it’s paid off. Data Management Plans (DMP) helped mitigate many of the risks of compliance. Data-driven enterprises increased their priority relating to data protection compliance and regulations, and so Obsidian has also responded to that with a single pane of glass solution.

Hybrid IT in a Data World

As Obsidian System’s has matured while on this data journey on finding sensible data that informs your next business move, we reflect on how the world has adapted to technology in SmarterData for you. The new digital currency of the world is data.

The technology should always underpin the goal which is to marry up the real-time with historic data in order to predict the future.

When Smarter Hybrid IT hit the fan, our partnership with Cloudera solidified. Together we can make sense of the present by learning from the past.

by Angela Ho