Are cultural differences (in psychological processes) reducible to individual differences?

A colleague alerted me to an interesting paper published in the prestigious journal Proceedings of National Academy of Sciences. In this article (www.pnas.org/cgi/doi/10.1073/pnas.1001911107), Jinkyng Na and colleagues around senior author Richard Nisbett argue that differences in social orientation and cognition that exist between social classes and national cultures can not be reduced to individual differences. They present data from a moderately sized sample (N=235) of US citizens. The researchers administered a total of 20 tests, which were subdivided into 10 tests supposedly measuring social orientation and 10 tests measuring some form of cognitive style. They found significant differences between two groups of participants described as low and middle class for 5 of the cognitive tests (4 in the predicted direction) and 4 significant differences in the social orientation (3 in the predicted direction). Combining all measures, they also found significant effects. At the individual level, correlations were close to zero and mainly not significant. I have no problems so far.

The findings capitalize on a well-known statistical phenomena that within-group correlations and between-group correlations are independent and that the total correlation is the combination of both types of correlations. Put differently, correlations between variables (within groups) are statistically independent from the mean differences between groups. The correlation between all variables across all groups can be predicted if the variances and covariances are known. To their credit they discuss this phenomenon and present some simulations in this direction (without acknowledging that these issues have been discussed for at least 30 years in methodological circles, but also fail to realize the importance of separating the within-group from the between-group effects).

What really concerns me is their interpretation of this pattern. They argue that cultural differences are valid and can not be reduced to individual level differences. Where is the problem here?

In their starting example, they refer to both IQ and personality. For both variables, reliable and stable instruments had been constructed first and subsequently, differences were also found between ethnic and cultural groups. Therefore, we have a clear idea of what is happening at the individual level and then need to figure how between-group differences can be understood (which is an on-going debate). Following on from this, they now proceed to two vaguely defined and heterogeneously measured domains, namely social orientation (defined as independence and interdependence) and cognitive style (broadly defined as analytic versus holistic). The cross-cultural literature is full of studies that show small to moderate differences between different societies (most typically US student samples versus East Asian student samples). Yet, there is no consensus on definitions, measurement or a clear understanding of what these variables really are. Indeed, variables are lumped together that are studied as different phenomena in different areas of psychology. For example, happiness is certainly not the same as self-construals or intensity of emotions. Attributions are not the same construct as thematic versus taxonomic categorization tasks. The tests also involve various different methods (which can be seen as strength or weakness depending on the viewpoint). A stroop task is often capturing different psychological processes compared to self-reports, as the heated debate on implicit versus explicit attitudes demonstrates. Looking at this array of tests, I would simply not expect a strong correlation. We are not dealing with a coherent psychological phenomenon.

The mean differences between the two groups could be due to a large number of variables. Some of the causal variables may be strongly related: education, opportunities in life, and various other variables related to wealth immediately spring to mind. In a famous quote, one of the pioneers on cognitive styles Herman Witkin explicitly argued that field independence (as measured by the FLT) is associated with formal education. Wealth is probably also the most important variable influencing social orientation, a fact that is widely known since the famous study by Hofstede published in 1980. Hence, nothing new here. But the problem is  how wealth is influencing these variables and this most likely happening through different psychological processes. Hence, there is no reason to assume that these group differences are actually related rather than being probably influenced by a host of similar but distal variables.

In the latter parts of the paper, the authors then sink in a mess of tautological arguments. Because previous studies have shown these differences, then these differences need to be real and valid and hence, the individual and group level are distinct. Remember, this was what they wanted to test!

Yet, the authors also fail to notice the major shortcomings in many of the previous studies. For example, many of them did not test whether the instruments were equivalent or whether there were instrument, method, sampling or administration biases that could have influenced these results. Furthermore, these previous studies did not link potential explanatory variables to the observed differences, which is a standard practice in comparative psychological research (see for example http://pps.sagepub.com/content/1/3/234.abstract).

The current paper also fails to provide evidence on the equivalence of the measures. More importantly, it is not discussed how the two different groups are actually defined. What is social class and how were people grouped and on what criteria? The current study essentially is non-replicable.

What is a better approach? This has been discussed by a number of eminent cross-cultural psychologists, including Michael Bond, Kwok Leung, Ype Poortinga, Fons van de Vijver, David Matsumoto and others; and I have also occasionally added my two cents to this debate in a number of publications. The way forward is to come up with meaningful, reliable and valid instruments in each group that are equivalent in all cultures of interest. Next, we need to identify the situational, biological or psychological variables that are likely to explain the (predicted) differences between groups. The important element for understanding differences comes in empirically measuring these variables and demonstrating the link between our expected explanatory variable and the observed difference. As an added bonus, we could examine the influence at individual and group (cultural) level. Essentially, we are talking about a multi-level model here. There are now good examples and an increasing number of people use this important approach.

This is a scientifically more productive endeavor than setting up straw arguments based on statistical artefacts. One only wonders how something like this could get into a prestigious journal like PNAS....