CHAPTER ONE

BASIC PARAMETRIC CORRELATION

A single correlation coefficient represents a correlation matrix with a dimensionality of one squared. Before we consider more complex correlation matrices, it is fitting to begin with the understanding of a correlation matrix with the size of one. Such a correlation matrix has 2 dimensions and a 0 (X - 2) degrees of freedom.

Any two sets of data, as long as they are evenly aligned within a data-table, may be correlated with one another regardless of the relative difference of the magnitude or scale of the data--in short, practically anything can be correlated with anything else, as long as both can be numerically represented. The advantage of the correlation coefficient is that it allows us to readily compare differently scaled sets of data, the disadvantage is that correlations may be measuring relationships which do not in fact resist. Thus, strong correlations may exist between completely unrelated sets of data, though this is probably unusual, and weak correlations may exist between strongly determined relationships between two sets of data.

Basic correlation may be said to measure the degree of similarity between two sets of data--or rather the relative degree to which the two parallel and aligned sets of data points "move together" when plotted on a scattergram. A correlation ranges between the values of -1 and +1, with 0 indicating a complete lack of correlation, negative values indicating a inverse relationship between the data sets, such that when the values of one set tend to be high, the values of the other set tend to be low, and vice versa, and a positive correlation indicating that when one set of values is high, the corresponding values of the other set also tend to be relatively high. The higher the correlation, the more "similar" the two sets of data--the more the data points move together in what might be described as a linear relationship.

The Pearson product moment correlation coefficient is a general estimator of the strength of a linear relationship between two sets of data. It is a "dimensionless" index which relfects the extent of a linear relationship between two points.

The graph above demonstrates the linear relationships that are involved when there is a perfect positive correlation, a perfect negative correlation, a weak negative correlation, and no correlation at all. The perfect positive correlation can be seen to be a diagonal line of points passing through the origin. The correlation is perfect despite the fact that the magnitude of the y points are different from the x by a factor of .1. The perfect negative correlation is the same, but running in an orthogonal direction. The weak negative correlation shows a dispersion about the general line of the perfect negative correlation, with a few points drifting off in the opposite direction in the center. The set of points showing a lack of any linear relations shows no line at all.

The graph above shows the lines drawn through the respective points. The correlation matrix below shows the correlations derived from this distribution and graph.

It is closely related to least squares linear regression analysis (model 1 regression), in fact it is equal to the slope of the functional equation of the regression line of the sample multiplied by the ratio f the sample standard deviations of X and Y. The linear reqression equations for the data for these graphs was computed using the following formula:

where

and where

Least squares regression has different presumptions than the Pearson product moment correlation coefficient--the predictor variable X is measured without error, the two samples are homoscedastic, having the same variance, there must be some linear relationship between X and Y. While regression coefficients measure the form of the linear relationship between two sets of points, correlation coefficients measure the degree of dispersion about the regression axis. The population correlation coefficient always presumes a bivariate normal distribution of both X and Y, which is never the case with regression analysis in which values of X are predetermined. The difference between the correlation coefficient and linear regression is that with linear regression, the x-coordinates are known and fixed, while with correlation coefficient both the x and y coordinate values are not known and variable. Whenever the predictor or independent variable (X) is known or fixed by the experimenter, linear regression analysis can be used by the experimenter to describe the relationship between the abcissa and ordinate. In this case the predictor variable has no real "distribution" and no assumptions about the variance of X need be made--it is selected rather than sampled. While we may predictively plot points of x and y according to our equation, we cannot do the same on the basis of correlation coefficients alone--in fact we cannot guess either x nor y. It is possible that if we know one coordinate, we can determine the other, but we do not have this information to begin with in the use of correlation.

The population coefficient () is defined as the square root of the Coefficient of Determination:

)

The Coefficient of Determination () is the same as the coefficient of Dispersion ( ) in linear regression, defined as:

The coefficient of Disperson is interpreted as the proportion of the variance in y that is attributable to the variance in x. The coefficient of Determination is therefore simply the inverse complement of the Coefficient of Nondetermination which represents the proportion of variability in the Y variable which remains unaccounted for by the relationship between X and Y:

In other words, the population correlation coefficient (or "rho") is expressed as:

The relations of the population correlation coefficient (p) to the sample correlationa coefficient (r) is that r is used to estimate p.The sum of standard deviations of the points is an indicator of the strength of a linear relationship. Dividing the sum of deviations by n expresses the average deviation of the points about X and Y, while dividing each individual deviation by the sample stndard deviation eliminates the relative units of analysis--hence the magnitude--of the scale of measurement.

or, replacing the sample standard deviation (S) for its formula:

This sample correlation coefficient (r) is an alternative algebraic form of estimating (p).

The sample correlation coefficient is related to the sample regression constants--it is the slope constant b multiplied by the ratio of the sample standard deviations of X and Y, which converts the slope into a dimensionless statement of correlation:

Because the values of X are distributed and undetermined in the sample, one cannot infer p directly from r, though is the square of the Pearson product moment correlation coefficient.

The first issues to be addressed in understanding and analyzing correlational matrices are the questions of significance--both theoretically and statistically. It is in attempting to answer this question systematically that we find the real strength and value of this form of analysis in hypothetically uniting qualitative and quantitive dimensions in the representation of complex realities.

The first question of significance of the correlation coefficent has to do with the significance of the individual scores, as determined by a t-test which fits the correlation, given the degrees of freedom, to a normally distributed curve:

I

n which () is assumed to be equal to 0 and degress of freedom (df) is equal to (n-2) or the number of dimensions of the matrix (X) minus 2. Either a one or two tailed test can be used--one-tailed tests do not ignore the sign of the correlation coefficient and so are used. A table of values assuming a two tailed test is given below:

If one wishes to test whether p is equal to some value other than zero, then the correlation coefficient r must be converted to Z using Fisher's conversion formula:

Determining the confidence limits for Fishers conversion with p values other than 0 is more useful with natural data sets than the standard t-test assuming p = 0. Hypothesis which lie outside the confidence interval will be rejected, but rejection of the correlation coefficient on the basis of the hypothesis that H is equal to 0 may be misleading about the actual significance of the correlation coefficient. The conversion of standard correlation coefficients is given by the following table:

The sampling distribution of Z is approximately normal, with a standard error estimated by the formula:

The confidence limits are used when estimating the probability of p, and are found by the z-test at an arbitrary level of confidence (p) times the standard error:

Confidence interval =

In which the interval is finding the z value of the interval on the z conversion table which is determined by subtracting the arbitrary confidence value by 1 and then dividing the difference in half, and then multiplying this z value by the sample mean.

The confidence interval about Z is therefore given as:

Fishers transformation (denoted as Z) of the correlation coefficient should not be confused with the z test (denoted as z) which measures the normal distribution of a point. The two values are entirely unrelated.

Fisher's transformation of the correlation coefficient must be used when comparing two independent correlation coefficients drawn from different samples. Both correlation coefficients must be converted using the formula above, and the standard error of the difference between these converted scores is given by the formula:

The standardized normal deviate for the difference between two Z scores is given as:

The hypothesis to be tested is whether there is a critical difference between the correlation coefficients, determined by comparing the resulting z value with the z score of the arbitrary confidence value p.

A separate but related question has to do with the question of relative variation of scores within the table--are individual scores significantly different from one another? The case may be imagined where two or more correlations are not individually significant in the external sense, but are significantly diffferent from one another. A correlational table may lack internally significant differences of pattern, and yet be externally coherent.

The possibility of cross-correlational analysis rests upon the following premises:

1. In general, all else being equal, a higher correlation is probably more significant than a lower correlation.

2. A negative and positive correlation of the same absolute value are of equal strength but possibly of very different theoretical significance.

3. A high correlation that is significant, especially of a large data set, suggests an underlying functional patterning.

4. a strong correlation may represent a functional relationship, but a weak, insignificant correlation represents a probable lack of a direct functional relationship.

5. a stronger correlation is more likely to represent such a relationship than a weaker one.

6. correlational sets of a matrix may be grouped and described statistically in a meaningful way--they form a distributed curve the characteristics of which (mode, median, mean, etc.) may be evaluated.

7. a correlational matrix can be reorganized in different ways, resulting in different descriptions of the resulting tables--these values can be compared (i.e. rows versus columns).

8. The rows and columns of a correlational matrix represent hypothetical dimensions of analysis which can be topically characterized.

9. In a complex and large correlational matrix, even low correlational values may be significant indicators of a functional relationship.