How to do Procrustean Factor Rotation

Procrustean Factor Rotation

 Today, it is a little bit less light-hearted, but hopefully a bit more practical. 

Aim: To make factor structures maximally comparable & provide a statistical estimate of factor similarity

Why are we concerned with Procrustean Rotation? Factor rotation is arbitrary, therefore apparently dissimilar factor structures might be more similar than we think; procrustean rotation is necessary to judge structural and metric equivalence

Statistical Procedure:

 A SPSS routine to carry out target rotation needs to be run (adapted from van de Vijver & Leung, 1997)

The following routine can be used to carry out a target rotation and evaluate the similarity between the original and the target-rotated factor loadings. One cultural group is being assigned as the source and the second group is the target group. The varimax rotated (or unrotated) factor loadings for at least two factors obtained in two groups need to be inserted. The loadings need to be inserted, separated by commas and each line is ended with a semicolon. The last line is not to end with a semicolon, but with a ‘}’. Failure to pay attention to this will result in an error message and no rotation will be carried out. To use an example, Fischer and Smith (2006) measured self-reported extra-role behaviour in British and East German samples. Extra-role behaviour is related to citizenship behaviour, voluntary and discretationary behaviour that goes beyond what is expected of employees, but helps the larger organization to survive and prosper. These items were supposed to measure a more passive component (factor 1) and a more proactive component (factor 2). The selection of the target solution is arbitrary, in this case we rotated the East German data towards the UK matrix. 

Table 1. Items and varimax-rotated loadings in each sample separately

           

	UK		Germany
	Factor 1	Factor 2	Factor 1	Factor 2
I am always punctual.	.783	-.163	.778	-.066
I do not take extra breaks.	.811	.202	.875	.081
I follow work rules and instructions with extreme care.	.724	.209	.751	.079
I never take long lunches or breaks.	.850	.064	.739	.092
I search for causes for something that did not function properly.	-.031	.592	.195	.574
I often motivate others to express their ideas and opinions.	-.028	.723	-.030	.807
During the last year I changed something. in my work....	.388	.434	-.135	.717
I encourage others to speak up at meetings.	.141	.808	.125	.738
I continuously try to submit suggestions to improve my work.	.215	.709	.060	.691

Syntax:

This can not be done using the windows interface within SPSS. You should run a factor analysis in each sample separately first. Use Varimax (orthogonal) rotation.  Then insert the loadings in the loadings and norm matrices in the SPSS syntax described in Fischer and Fontaine (2011, in Matsumoto and Van de Vijver’s Cross-Cultural Research Methods in Psychology). I can also email this syntax to you (contact me at Ronald.Fischer@vuw.ac.nz).

The start of the syntax is printed below. Be careful to separate the loadings by a ‘,’ and the last loading for each item needs to be followed by ‘;’. The last loading should be indicated by }.

matrix.

compute LOADINGS={

.778,    -.066;  

.875,    .081;   

.751,    .079;   

.739,    .092;   

.195,    .574;   

-.030,   .807;   

-.135,   .717;   

.125,    .738;   

.060,    .691     }.

compute       NORMs = {

.783,    -.163;  

.811,    .202;   

.724,    .209;   

.850,    .064;   

-.031,   .592;   

-.028,   .723;   

.388,    .434;   

.141,    .808;   

.215,    .709}.

…

Output and Interpretation:

The edited output for this example is shown below. It shows the rotated matrix of the group (East Germany in our case) that was rotated to maximal similarity:

*********************************************************************

Run MATRIX procedure:

FACTOR LOADINGS AFTER TARGET ROTATION

   .77  -.10

   .88   .04

   .75   .05

   .74   .06

   .22   .57

   .00   .81

  -.10   .72

   .16   .73

   .09   .69

DIFFERENCE IN LOADINGS AFTER TARGET ROTATION

  -.01   .06

   .07  -.16

   .03  -.16

  -.11   .00

   .25  -.03

   .03   .08

  -.49   .29

   .02  -.08

  -.13  -.02

Square Root of the Mean Squared Difference per Variable (Item)

   .05

   .12

   .12

   .08

   .18

   .06

   .40

   .05

   .09

Square Root of the Mean Squared Difference per Factor

   .19   .13

IDENTITY COEFFICIENT per Factor

   .94   .97

ADDITIVITY COEFFICIENT per Factor

   .86   .92

PROPORTIONALITY COEFFICIENT per Factor

   .94   .97

CORRELATION COEFFICIENT per Factor

   .86   .93

------ END MATRIX -----

The output shows the factor loadings following rotation, the difference in loadings between the original structure and the rotated structure as well as the differences of each loading squared and then averaged across all factors (square root of the mean squared difference per variable column).

The first matrix could be pasted in a new table, showing the rotated loadings (instead of using the loadings from the original analysis as reported above in the table). The second matrix shows the differences after rotation. You should look for large values, because they indicate that some items are problematic. A low value would indicate good correspondence.

The column of values entitled: Square Root of the Mean Squared Difference per Variable (Item) gives you information about each item. The larger the value, the more problematic is an individual item. The next row (Square Root of the Mean Squared Difference per Factor) shows the same information per factor. Again, smaller values are better, larger values indicate trouble for a particular factor. There are no hard and fast criteria for any of these indices above, you should look at the relative values and particular discrepant values.

The most important information is reported in the last four lines, namely the various agreement coefficients. As can be seen there, the values are all above .85 and generally are beyond the commonly accepted value of .90. The most common indicator is Tucker’s Phi which is called Proportionality coefficient here.

It is also worth noting the first factor shows lower congruence and that the estimate vary across indicators. An examination of the differences between the loadings shows that one item (During the last year I changed something. in my work....) in particular shows somewhat different loadings. In the British sample, it loads moderately on both factors, whereas it loads highly on the proactivity factor in the German sample. Therefore, among the British participants making some changes in their workplace is a relatively routine and passive task, whereas for German participants this is a behaviour that is associated more with proactivity and initiative (e.g., Frese et al., 1996). We might want to exclude this item and re-run the analyses. Overall, we could cautiously conclude that our scales meet structural equivalence and most items might even meet metric equivalence (although this syntax routine does not provide a statistical test for this higher level of equivalence). 

Good on ya... if you made it to this point ; ) Hope your eyes are looking slightly better than that of a Tarsier...

Are cultural differences (in psychological processes) reducible to individual differences?

A colleague alerted me to an interesting paper published in the prestigious journal Proceedings of National Academy of Sciences. In this article (www.pnas.org/cgi/doi/10.1073/pnas.1001911107), Jinkyng Na and colleagues around senior author Richard Nisbett argue that differences in social orientation and cognition that exist between social classes and national cultures can not be reduced to individual differences. They present data from a moderately sized sample (N=235) of US citizens. The researchers administered a total of 20 tests, which were subdivided into 10 tests supposedly measuring social orientation and 10 tests measuring some form of cognitive style. They found significant differences between two groups of participants described as low and middle class for 5 of the cognitive tests (4 in the predicted direction) and 4 significant differences in the social orientation (3 in the predicted direction). Combining all measures, they also found significant effects. At the individual level, correlations were close to zero and mainly not significant. I have no problems so far.

The findings capitalize on a well-known statistical phenomena that within-group correlations and between-group correlations are independent and that the total correlation is the combination of both types of correlations. Put differently, correlations between variables (within groups) are statistically independent from the mean differences between groups. The correlation between all variables across all groups can be predicted if the variances and covariances are known. To their credit they discuss this phenomenon and present some simulations in this direction (without acknowledging that these issues have been discussed for at least 30 years in methodological circles, but also fail to realize the importance of separating the within-group from the between-group effects).

What really concerns me is their interpretation of this pattern. They argue that cultural differences are valid and can not be reduced to individual level differences. Where is the problem here?

In their starting example, they refer to both IQ and personality. For both variables, reliable and stable instruments had been constructed first and subsequently, differences were also found between ethnic and cultural groups. Therefore, we have a clear idea of what is happening at the individual level and then need to figure how between-group differences can be understood (which is an on-going debate). Following on from this, they now proceed to two vaguely defined and heterogeneously measured domains, namely social orientation (defined as independence and interdependence) and cognitive style (broadly defined as analytic versus holistic). The cross-cultural literature is full of studies that show small to moderate differences between different societies (most typically US student samples versus East Asian student samples). Yet, there is no consensus on definitions, measurement or a clear understanding of what these variables really are. Indeed, variables are lumped together that are studied as different phenomena in different areas of psychology. For example, happiness is certainly not the same as self-construals or intensity of emotions. Attributions are not the same construct as thematic versus taxonomic categorization tasks. The tests also involve various different methods (which can be seen as strength or weakness depending on the viewpoint). A stroop task is often capturing different psychological processes compared to self-reports, as the heated debate on implicit versus explicit attitudes demonstrates. Looking at this array of tests, I would simply not expect a strong correlation. We are not dealing with a coherent psychological phenomenon.

The mean differences between the two groups could be due to a large number of variables. Some of the causal variables may be strongly related: education, opportunities in life, and various other variables related to wealth immediately spring to mind. In a famous quote, one of the pioneers on cognitive styles Herman Witkin explicitly argued that field independence (as measured by the FLT) is associated with formal education. Wealth is probably also the most important variable influencing social orientation, a fact that is widely known since the famous study by Hofstede published in 1980. Hence, nothing new here. But the problem is  how wealth is influencing these variables and this most likely happening through different psychological processes. Hence, there is no reason to assume that these group differences are actually related rather than being probably influenced by a host of similar but distal variables.

In the latter parts of the paper, the authors then sink in a mess of tautological arguments. Because previous studies have shown these differences, then these differences need to be real and valid and hence, the individual and group level are distinct. Remember, this was what they wanted to test!

Yet, the authors also fail to notice the major shortcomings in many of the previous studies. For example, many of them did not test whether the instruments were equivalent or whether there were instrument, method, sampling or administration biases that could have influenced these results. Furthermore, these previous studies did not link potential explanatory variables to the observed differences, which is a standard practice in comparative psychological research (see for example http://pps.sagepub.com/content/1/3/234.abstract).

The current paper also fails to provide evidence on the equivalence of the measures. More importantly, it is not discussed how the two different groups are actually defined. What is social class and how were people grouped and on what criteria? The current study essentially is non-replicable.

What is a better approach? This has been discussed by a number of eminent cross-cultural psychologists, including Michael Bond, Kwok Leung, Ype Poortinga, Fons van de Vijver, David Matsumoto and others; and I have also occasionally added my two cents to this debate in a number of publications. The way forward is to come up with meaningful, reliable and valid instruments in each group that are equivalent in all cultures of interest. Next, we need to identify the situational, biological or psychological variables that are likely to explain the (predicted) differences between groups. The important element for understanding differences comes in empirically measuring these variables and demonstrating the link between our expected explanatory variable and the observed difference. As an added bonus, we could examine the influence at individual and group (cultural) level. Essentially, we are talking about a multi-level model here. There are now good examples and an increasing number of people use this important approach.

This is a scientifically more productive endeavor than setting up straw arguments based on statistical artefacts. One only wonders how something like this could get into a prestigious journal like PNAS....