The entire concept of big data isn’t an easy one to understand, but that is not entirely necessary. In the same way that we don’t know how to calculate the trajectory or the velocity of a thrown ball, but we are still able to catch it, we can use big data to improve our business without knowing exactly how it all works.
As a starting point, we know that it somehow influences the advertising we see online, and how and when we receive marketing communications. But, the applications of big data go way beyond the business of trying to sell us more things more often (though it would be a nice skill to learn), they will impact even our work processes and are at the root of some of the key decisions made about our future at a governmental level.
What is ‘Big Data’
The term ‘Big Data’ refers to an amount of gathered data that is just so large that it cannot be analyzed using normal processes. This is also true for any data that arrives at a faster rate than can be processed by traditional means or data that is more complex than processing methods can handle.
To break this quite massive concept down into much smaller chunks, we can divide it up more neatly as follows:
- Volume – A measure of how much data is collected for analysis. Just consider the amount of information collected every day by, for instance, Facebook or Google, and it’s easy to see where the ‘big’ in ‘Big Data’ comes from.
- Velocity – Next, it’s about how fast that data arrives, then how it is managed and marked on the way in. Almost like presorting mail before it goes into various pigeonholes in the mailroom.
- Variety – It matters which format that data is in. Is it a stream of numbers, a text field, or maybe even something else? This has implications for both storage and the tools required to analyze the data.
- Variability – The data will not be generated at the source or received at a constant rate. Any ‘Big Data’process will have to take this into account and make necessary allowances.
- Veracity – Not all data is of the same quality, so it has to be assessed according to certain criteria, or algorithms to establish its level of significance.
Why is Big Data so important to business?
While Facebook and Google were used there as rather obvious examples, they are far from being the only processors of data on such a massive scale. The concept of taking data from a source and analyzing it to get answers is used throughout the entire business world.
The key point here, though, is that it is not necessarily the amount of data, but what you do with it that will give you the most productive results. You can discover the root causes of failures, or detect patterns that may harm your business.
Of course, there are many considerations before analyzing the data for such sweeping decisions. If the data is flawed, incomplete, or not gathered widely enough to provide a representative sample, then the conclusion will be equally flawed. Or, to put it more traditionally – ‘garbage in, garbage out.’
‘Big Data’ in practice
Regardless of the size of your business, the same principles apply when putting Big Data to work for you. It doesn’t matter how many sources of data, how many locations it may be coming from, and who or what is generating the data; the same five key steps apply.
#1 Set your data strategy
This is your masterplan. You need to decide whether to analyze your business as a whole or areas you have identified as needing improvement. You need to consider what resources you have available and how you intend to use the result. In short, you need to manage this like any other project or asset in your business.
#2 Identify your data sources
Now you have your plan, you need to work on where your data is coming from. Is it feedback from customers? Sales data? Production data from your manufacturing operation? Or is it all of them?
This key part of the process will reflect heavily on the usefulness of the end data. Try and analyze too much data and you might find it hard to draw conclusions. Look at too little, and all you may find is that the root cause might be in a different part of the business.
#3 Collect and store the data
Once you know what you want to collect, you now need to work out how to collect it and how you will store it. In line with the’ Variety’ principle above, you need to be aware of which kind of format the data will arrive in. It then needs to be tagged for sortation and stored for the next part of the process.
#4 Analyze the data
While this might sound like the only operation that needed doing, if you’ve got steps 1-3 correct, this, if you have the right people involved, will be the easiest part. You’ll only be analyzing the relevant data, in a compatible format, collected from all of the correct sources. The results you have now arrived at are one of your business’s greatest assets.
#5 Make changes based on the results.
And this is why. Making data-driven decisions is a key part of every modern business. Your ‘gut’ decisions have probably got you this far, as a small business owner or entrepreneur. But, when your business has grown to the stage where you can no longer see every part of it, you have to start relying on the data.
The effect ‘Big Data’ has on all of us
You’ll see the effects of Big Data most obviously in retail and finance. As an example, we can look at the coupon that was printed with your receipt when you were at the supermarket checkout.
Through (predominantly) the use of loyalty cards, data about our weekly shopping habits are stored; not only what we buy but how often we buy it. So if we miss buying a product for a week or two, we can be presented with a money off coupon as a reminder the next time we swipe our card at the checkout.
Learning from Big Data (and what it learns from us)
Just for the basic example presented above, the techniques used included predictive modeling, descriptive modeling, pattern mining, and anomaly detection.
Or, for those of us who aren’t data scientists (more on them in a moment) – knowing what you were going to do, how you were going to do it, how often you usually do it, and identifying that this time you didn’t.
However, the art of extracting and interpreting, and learning from Big Data takes high levels of technical expertise and strategic thinking. While some of these traits are true of the business owner or entrepreneur, those exact disciplines are best left to a data scientist.
What is a Data Scientist and why would you hire one?
A data scientist is a qualified specialist (usually to master’s level) in applied statistics, but will also have an excellent knowledge of computer science and business. So, not only will they be experts in data, they will have the skills to communicate their findings, and the business knowledge to know the relevance of what they have discovered.
They also have an intellectual curiosity, a desire to learn, that makes them particularly good people to have as part of your business – even if only for specific projects. To see exactly what a data scientist would learn as part of their master’s program click here.
The true value of Big Data to you
Ok, so we’ve explored what big data is, we’ve seen how it works, who to hire to get it done for us, and even how it’s used to give you a coupon for a can of beans. But, as a small business owner or entrepreneur, where is the value of all of this? Basically, what’s in it for you?
Well, from what you’ve seen already – you’d have to agree that, when used correctly, it could reduce your costs, increase sales, help you find the right price for your goods, and help with your decision making.
Getting Big Data to work for you
By performing the same analysis that got you your coupon at the checkout, you could discover your customers buying habits so that you offer them the right thing at the right time – to put things in front of them when they are ready to buy, rather than when you want to sell.
The mantra for most businesses is test, test, and test again when it comes to pricing, but analyzing data of customer’s habits may narrow the band that testing takes place within, and arrive at the optimum price much faster than the traditional trial and error method.
By analyzing customer habits (or more importantly, those of the ones that didn’t buy), you can tailor your product range to include something they certainly would have bought if given the opportunity.
All of this adds up to you being able to make better decisions more quickly. The fewer hours or even days that are taken up with guesswork and trying things out, the more time you have to get on with expanding your business.
