IEEE Technology and Society Magazine - September 2019 - 88

to fit the learned hypothesis on unseen data. In Big Data
research, generalization is the procedure of spanning
the characteristics of a group or class to the entire
group or population. This enables the inference of attributes of an entire population without getting to know
every single element in that population individually. The
problem comes along when we wrongly generalize; or
more precisely when we overdo it. The generalization
fallacy occurs when statistical inferences about a particular population are asserted to a group of people for
which the original population is not a representative
sample. In other words, models overfit when they learn
not only the signal from the training data but also the
noise that impedes the model's capability to predict on
unseen data. In order to avoid excessive generalization
error, researchers should also check the scope of the
results instead of extending scientific findings to the

"I understand that my work may
have enormous effects on society and
the economy, many of them beyond
my comprehension.''

whole population. Regularization is helpful to avoid
overfitting by reducing the number of parameters to fit
model in high-dimensional data [35]. It also prevents
model parameters to change easily, which helps in
keeping the focus of the model on the persistent structure. Apart from regularization, other techniques such
as cross-validation, early stopping, weight sharing,
weight restriction, sparsity constraints, etc., can also be
used for reducing the generalization error based on the
algorithm being used.

Avoiding Bias
Big Data tends to have high dimensionality and may be
conflicting, subjective, redundant, and biased. The
awareness of potential biases can improve the quality
of decisions at the level of individuals, organizations,
and communities [25]. In a study of 1000 major business investments conducted by McKinsey, it was found
that when organizations worked to minimize the biases
in their decision-making, they achieved up to 7% higher
returns [30]. The biases associated with multiple comparisons can be deliberately avoided using techniques
such as the Bonferroni correction [48], the Sidak correction, and the HolmBonferroni correction [1]. Another
source of bias is called data snooping or data dredging,

which occurs when a portion of data is used more than
once for model selection or inference. In technical evaluations of results, it is conceivable to repeat experiments using the same dataset to get satisfactory results
[49]. Data dredging can be avoided by conducting randomized out-of-sample experiments during hypotheses
building. For example, an analyst gathers a dataset and
arbitrarily segments it into two subsets, A and B. Initially, only one subset - say, subset A - is analyzed for
constructing hypotheses. Once a hypothesis is formulated, it should then be tested on subset B. If subset B
also supports such a hypothesis, then it might be trusted as valid. Similarly, we should use such models that
can consider the degree of data snooping for obtaining
genuinely good results.

Finding Causality Rather than Correlations
In most data analysis performed in the Big Data era, the
focus is on determining correlations rather than on
understanding causality [37]. For BD4D problems, we're
more interested in determvining causes rather than correlates and therefore we must place a premium on performing causal BD4D analysis since causally driven
analysis can improve BD4D decisions. Discovering causal relations is difficult and involves substantial effort,
and requires going beyond mere statistical analysis as
pointed out by Freedman [14] who has highlighted that
for data analytics to be practically useful, it should be
problem-driven or theory driven, not simply data-driven.
As Freedman says, using Big Data for development
requires "the expenditure of shoe leather" to situate the
work in the proper context. The focus on correlation has
arisen because of the lack of a suitable mathematical
framework for studying the slippery problem of causality
until the recent fundamental progress made by Pearl
[37], whose work has now provided a suitable notation
and algebra for performing a causal analysis.

Stress High-Quality Data Analytics Rather
Than Big Data Analytics
A better and thoughtful understanding of risks or pitfalls
of Big Data is crucial to decrease its associated potential harms to individuals and society. There needs to be
a stress on utilizing Big Data along with data collected
through traditional sources to provide a deeper, clearer
understanding of problems, instead of being fixated on
only generating and analyzing large volumes of data.
Although it is generally preferred to have more data, it is
not always desirable, especially in the cases where data
is biased. Another disadvantage of large datasets is cost
in terms of processing, storage, and maintenance. However, some simple methods like sampling and/or resampling enable us to extract the most relevant data from a
larger chunk of data. Another very important aspect is

IEEE TECHNOLOGY AND SOCIETY MAGAZINE

∕

SEPTEMBER 2019

IEEE Technology and Society Magazine - September 2019

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - September 2019