How to Prepare your Infrastructure for the World of Big Data

Add Your Comments

More and more I’ve been a part of conversations revolving around the power of data, automation, and even building AI into business processes. At the core of this conversation is big data. And big data is certainly getting bigger.

A recent Gartner study showed that organizations already have multiple goals when it comes to big data initiatives. This includes initiatives like enhancing the customer experience, streamlining existing processes, achieving more targeted marketing, and reducing costs.

Gartner’s study found that organizations are overwhelmingly (64 percent) targeting enhanced customer experience as the primary goal of big data projects. From there, process efficiency and more-targeted marketing are tied at 47 percent. Finally, as data breaches continue to make headlines, big data initiates around enhanced security capabilities saw the largest increase, from 15 percent to 23 percent.

“As big data becomes the new normal, information and analytics leaders are shifting focus from hype to finding value,” said Lisa Kart, research director at Gartner. “While the perennial challenge of understanding value remains, the practical challenges of skills, governance, funding and return on investment (ROI) come to the fore.”

More from Bill Kleyman: 5 Steps to Avoid a Cloud Outage and Save Money

Another Gartner report on big data indicates even more growth around business intelligence and digitizing the entire business process. Consider the following:

  • By 2020, information will be used to reinvent, digitalize or eliminate 80 percent of business processes and products from a decade earlier.
  • Through 2016, less than 10 percent of self-service BI initiatives will be governed sufficiently to prevent inconsistencies that adversely affect the business.
  • By 2017, 50 percent of information governance initiatives will have incorporated the concept of information advocacy, to ensure they are value-driven.

Throughout this growth we’re seeing directed investments into big data systems. A recent GE Capital study showed that capital spend on server, storage and cloud infrastructure for purposes of supporting big data efforts on a global basis are anticipated to increase at a 37.6 percent CAGR between 2012 and 2016 and with it, comes increased demand for housing the incremental equipment at a data center and within the cloud.

Throughout all of this growth there are still some real challenges in adopting and actually leveraging big data strategies. The earlier Gartner report states that through 2017, fewer than half of lagging organizations will have made cultural or business model adjustments sufficient to benefit from big data.

So, with all of this growth and emerging value around information – how can you benefit from big data? Most of all, how do you create a cloud or data center platform capable of supporting big data requirements?

To make sense of it all – consider these three simple, yet key, points getting to a world of better big data utilization:

  1. Big data – do I actually need it? Before you get nervous after reading all of these stats and figures you need to understand where big data actually fits in. You might be an organization that doesn’t produce a lot of data or you simply don’t retain information about your clients. However, whichever way you look at it, you’re selling a product or service. That means you’re trying to address a market. So, even if you’re not creating a big data platform, you might want to consider leveraging from a partner.

However, if you’re an organization that’s rich with customer data, sales figures, and other critical data points – you might be wasting away a gold mine. Big data engines help analyze a vast amount of different data types. This means looking at both structured and unstructured data points. Remember, structured data refers to information with a high degree of organization, like text files in easy-to-read formats. Basically, something a data mining engine, with fairly straightforward search engine capabilities, can go through with ease. Unstructured data is, more or less, the opposite. This can be bitmap images/objects, text and other data types that are not part of a database. For example, an email can be considered unstructured data because the data is ‘raw’ and doesn’t really follow a structured (easy for a database to understand) format. From there, correlating and quantifying this information can really impact business decisions. So, if you have enterprise data which you know has value behind it – you should start looking at big data options.

  1. No, you most likely can’t use your legacy infrastructure for big data. OK, at this point you’ve realized that maybe big data can actually help you out. So, you install a couple of apps on a legacy server, hook it up to your existing SAN, and you are big data ready… right? No, not quite. If you’re serious about creating a big data platform you need to understand the underlying technology components required to make it all work. First of all, you may be potentially running batch queries against your big data sets which are extremely resource consuming. Can your existing infrastructure actually handle that? Now, let’s touch on storage. There’s a very good reason vendors like Pure Storage just released the FlashBlade platform, aiming to deliver real-time all-flash analytics to help bring transactional capabilities to big data repositories. The idea is to better support big data repositories (NFS, Object/S3) and delivery highly scalable storage solutions (via blades) for a variety of data research requirements.

If you want to create solid strategies around big data, take the time to build an ecosystem that can support it. Importantly, don’t forget to leverage the cloud. If you want to utilize cloud components, there are partners out there who can help you take big data into the cloud. For example, there are great reference architectures for powerful big data engines like Cloudera running on Azure. Or, you can leverage the AWS cloud ecosystem to run Hadoop and other big data processes.

  1. Classifying data and leveraging the value of information. This is really where your big data journey evolves. This is where you truly understand whether you’re building this on your own, leveraging a cloud partner, or maybe leveraging a hybrid solution. Designing a big data solution must start with a deep exploration of the business. What type of data do you have? How is it potentially valuable? Where will does it help you make decisions? Learning your business from a data perspective must be done from an entirely different lens. You’re taking into consideration variables in the market, users, customers, and your own business. From there, you identify where there are data points within your enterprise which can impact these variables. And then, you leverage big data to help you make business decisions to stay agile in a very volatile market. The point here is that you must first learn the value of your data before you can truly make a big data platform which can impact your organization.

[Tweet Designing a big data solution must start with a deep exploration of the business via @theWHIR @Quadstack]

The simple truth is that there is real value behind big data. However, it’s up to you to learn how to leverage it and how to apply it to business. As the consumer and business grow more interconnected and digitized, there will be more information created to help identify what we do in the digital world. This information can help form business decisions, enable improved processes, and create competitive market advantages for organizations. As digital transformations continue across more verticals big data will help enable learning engines and business intelligence at unprecedented levels. Those organizations which can capture the power of data will be a lot more agile and competitive in the market.

Add Your Comments

  • (will not be published)