Sieve Strategy of Nonparametric Statistical Data Modelling

Modern attempts to define the foundations of statistical science emphasize that it is a way to think about data modeling. The influential 1970 book of Box and Jenkins taught us that statistical science is NOT just about estimation, testing, or even data analysis, but is about data modeling done in a systematic way by an iterated series of steps which I formulate as SIEVE: sample statistics, identify parametric model, estimate parameters and orders, validate goodness of fit, estimation non-parametric. Another summary of SIEVE: observation, expectation, estimation, comparison, decision. 

The SIEVE data modeling strategy can be illustrated by data (X,Y₁,Y₂) where X=1 or 2 according as the patient observed is healthy or diseased. One observes also medical variable Y₁ and Y₂ (in one data set, Y₁ is total heart weight and Y₂ is body weight). From the small set of n₁ observations of (1,Y₁,Y₂) and n₂ measurements of (2,Y₁,Y₂) one obtains an extensive output to describe the relations between the variables and probabilities such as P[X=1|Y₁,Y₂]. 

Univariate analysis provides data models of the probability distributions of Y₁|X=1, Y₁|X=2, Y₁, Y₂|X=1, Y₂|X=2, Y₂. The initial no-model step estimates mean, standard deviations, median, quartiles, tails, skewness, histogram, boxplot, quantile/quartile plots. 

Two sample univariate analysis tests homogeneity (equality of the distributions of Y₁ in the two sample X=1 and X=2). It compares Y₁|X=1 and Y₁|X=2 by comparing Y₁|X=1 and Y₁. It compares Y₂|X=1 and Y₂. 

Bivariate analysis tests independence of Y₁ and Y₂, (conditional on X=1) by testing the independence of Y₁ and Y₂ by comparing Y₁|Y₂ and Y₁. One computes scattergram-boxplot, correlation coefficients, conditional quantile Q_Y2|Y1=y (u), non-parametric regression E[Y₂|Y₁=y]. 

Two sample bivariate analysis test homogeneity of distribution in the two groups (X=1,X=2) of (Y₁,Y₂) by comparing (Y₁,Y₂)|X=1 with (Y₁,Y₂). This is a generic problem of machine learning.