What scale of data increases the potential for finding insight from Big Data?
I think it really depends on the data set... While units like MBs and GBs are small by today's standard, they can contain extremely useful types of information My vote is at least GBs, since we still need enough data to draw strong correlations.
Most customers seek to find value in smaller data sets because they are more easily understood. Further when we get to the exabyte scale for architectures statistical sampling and analysis of these data close to the source is very likely. Therefore I'd like to suggest that the trend will be to manage smaller data sets.
Data analytics may make results from large data sets more easily trusted even when the results are misleading. It is sometimes too easy to rely only on the analytic results fromhuge data sets without fully understanding the underlying assumptions and complex algorithms. Analytic results become the new 'facts.' Journalism relies on fact checkers. Maybe we need a new breed of independent fact checkers focused on big data analytics.
Retrieving data ...