The motivational basis of personality & why threat is important

How do we describe what other people are like? What are the major characteristics that we can use to describe the personality of friends and strangers? As far as we know, these questions has been discussed since the emergence of ancient civilizations in Greece, India and China. In modern psychology, starting with Gordon Allport, a more or less sharp distinction has been made between values - that is motivational goals important for people in their lifes - and personality traits - described as behavioural consistency across situations. The study of values has flourished in social psychology and the study of personality traits has been a core area of research of personality psychology, with little discussion of the overlap or convergence between the two approaches.

Diana Boer and myself were intrigued to test the similarities of these two approaches. We were inspired by newly developed neural network models of personality that conceptualized personality as expressions of basic motivational goal systems. If personality traits are expressions of motivational goal systems, we should see some systematic relationships with values. Furthermore, we were aware of a number of studies that have found quite variable associations between values and personality traits in the Big Five tradition. If there is some systematic association, as we presumed, why should there be such variability in empirical studies?

We set out to address these two broad sets of questions in a paper that is appearing in the Journal of Personality.

We collected all the studies that have reported correlations between any set of Big Five instruments with the circular value theory described by Shalom Schwartz and conducted a meta-analysis to identify the overall patterns that might be obscured in individual studies. Let us first quickly review personality traits and values. 

Overview of Personality & Values

Here is a summary of the Big Five traits from Wikipedia:

Openness to experience: (inventive/curious vs. consistent/cautious). Appreciation for art, emotion, adventure, unusual ideas, curiosity, and variety of experience. Openness reflects the degree of intellectual curiosity, creativity and a preference for novelty and variety a person has. It is also described as the extent to which a person is imaginative or independent, and depicts a personal preference for a variety of activities over a strict routine. Some disagreement remains about how to interpret the openness factor, which is sometimes called "intellect" rather than openness to experience.
Conscientiousness: (efficient/organized vs. easy-going/careless). A tendency to be organized and dependable, show self-discipline, act dutifully, aim for achievement, and prefer planned rather than spontaneous behavior.
Extraversion: (outgoing/energetic vs. solitary/reserved). Energy, positive emotions, surgency, assertiveness, sociability and the tendency to seek stimulation in the company of others, and talkativeness.
Agreeableness: (friendly/compassionate vs. analytical/detached). A tendency to be compassionate and cooperative rather than suspicious and antagonistic towards others. It is also a measure of one's trusting and helpful nature, and whether a person is generally well tempered or not.
Neuroticism: (sensitive/nervous vs. secure/confident). The tendency to experience unpleasant emotions easily, such as anger, anxiety, depression, and vulnerability. Neuroticism also refers to the degree of emotional stability and impulse control and is sometimes referred to by its low pole, "emotional stability".

Shalom Schwartz' theory of values differentiates at least 10 value types that can be organized into two major higher order dimensions. These two major dimensions are openness to change (individualistic) versus conservative (collectivistic) values on one hand and self-enhancing (dominance) versus self-transcendence (altruistic) values. Individual values can now be ordered in a circular structure along the two dimensions. Moving around the circle, Power (PO) captures the goals of striving towards social status and prestige, controlling or dominating over people and resources. Achievement (AC) emphasises personal success through socially approved standards of competence. Hedonism (HE) values focus on pleasure and sensuous gratification of the sense. Stimulation (ST) captures excitement, novelty and pursuing challenging goals in one’s life. Self-direction (SD) entails valuing independent thought and action. Universalism (UN) values refer to the motivation to understand, appreciate, tolerate and protect the welfare of all people and nature. Benevolence (BE) in contrast has a more narrow focus on preserving and enhancing the welfare of people close to oneself (family and close friends). Tradition (TR) values are focused on respecting, accepting and committing to the customs and ideas of the traditional culture and religion. Conformity (CO) refers to restraining actions or impulses that may upset or harm others and violate social expectations and norms. Finally, security (SE) emphasises values around safety, harmony and stability of society, social relationships and the self.

Values and Personality Traits are systematically linked

Our basic argument was that values as motivational goals and personality traits as behavioural consistencies should be systematically linked. Using a new method that we developed in a previous article that allows us to track the systematic relation of personality traits to the underlying structure of values (see our previous article), we were able to examine the overall relationship between the two constructs. 

Schematically, the overall association can be described like in this figure:

Agreeableness and Self-Transcendence (positively) versus Self-Enhancement (negatively) values are strongly and consistently related. Agreeable individuals are also very benevolent and they tend to care about others, close and distant alike. Similarly, Openness personality traits were strongly associated with Openness to Change (positively) versus Conservatism (negatively) values. Open individuals also strongly value stimulation and self-direction values. The correlations for these two traits with values were substantive and not significantly different from correlations between different personality instruments measuring the same trait. Hence, for these two personality traits, the relationship with values is strong and values and personality are highly convergent. 

Extraversion is somewhat more weakly related to Openness values, but also weakly to Self-Enhancement values (achievement and power values). Conscientiousness is related to Conservatism values, but also shows some correlations with Self-Enhancement values (in particular achievement values). These two personality traits correlated significantly weaker with values and there were sometimes notable secondary associations of traits with the other main axis of values. This suggests that these two traits are more complex in their basic motivational structure. 

Finally, Neuroticism is a stability oriented personality trait and no surprisingly shows some weak associations with values that ensure stability (Conservatism values). 

This was a really re-assuring and strong pattern overall. By using a holistic approach to values and personality across a large number of studies, we were able to show the overall systematic relationships between these two constructs. 

Threat undermines the value-personality trait relationship

But moving to the second main question, how can we make sense of the variations in these correlations across different studies? 

Here we need to first briefly consider how values and personality traits might be causally related. First, values may provide the motivational structure for humans, that is then expressed in behaviour (personality traits). Assuming this logic, people who value conformity will follow rules and orders with great care. Values come first and actions follow. Second and alternatively, following classic self-perception theory, people might engage in behaviours in relatively consistent ways, which they then interpret in terms of overarching goals and re-interpret as their stable values. E.g., I am always conscientious and follow rules and orders in a consistent way, therefore, I am probably a person who values conformity and tradition.  Here, actions come first and values are inferred from these actions in a secondary step. 

We thought about what may weaken this link between values and traits. Environmental threats should play a major role in how values and personality traits are linked. In highly threatening environments, that is environments where there is poverty, lots of environmental threats such as cold winters and superhot summers, lots of diseases (you can easily get infected and die), little available food, lots of violence and no personal freedom, people are probably quite restricted in their choices and behaviours. Hence, their personal values may not be expressed in behaviours, because the environment determines their actions more than their personal orientations and motivations. Equally, since the environment strongly influences people's behaviour, individuals may not interpret their own behaviour as reflecting some underlying values because their behaviour is more strongly influenced by environmental pressures. Hence the overall link between values and personality traits should be lower, regardless of how values and personality are causally related. 

We tested this hypothesis using a good number of different indicators of threat. Amazingly, threat turned out to be a strong and consistent moderator for most of the value-personality associations. Hence, our analysis of environmental threat in a broad sense can explain why there was so much variability in the literature previously. 

Exploring Personality Systems

It also could explain why researchers have not integrated personality research with value research in a more systematic way. Given the substantive variability, researchers might have thought that the links are too weak and too inconsistent to be worth pursuing. However, we believe we have shown that by taking a broader picture, examining the value-personality link in a more systematic way and examining the conditions in which the links are stronger or weaker, we can move both value and personality research forward. 

In my view, values and personality traits are expressions of underlying motivational systems that are encoded in similar and overlapping language. It is more of an accident of history that these two systems have emerged in different research fields. I hope this study will help to bring the fields back together and allow a more sophisticated examination of how values and traits are both expression of human personality. 

Indigenous or not indigenous.... that is the question

Today I listened to a really interesting talk by Peter Smith from Sussex Uni. He was presenting his work on social influence, including some of the new stuff on different indigenous social influence strategies such as the Chinese guanxi, Brazilian jeitinho, Middle Eastern wasta and Russian svasy. These are all local behaviours that individuals may adopt to solve problems in their environment, typically by relying on social relationships or their power (e.g., for a great example from the news this morning - the son of the Iraqi transportation minister forcing a plane to return to Beirut). Peter and his colleagues asked students and managers to come up with good examples of each local cultural strategy in their local culture. They then took the most representative scenarios from each culture, removed any identifying content and gave it to managers in other cultures. What they found was that the supposedly indigenous influence strategies were generally seen as typical even in other cultures. In other words, British 'pulling strings' was often as likely to be seen as applicable and typical in China and Saudi Arabia as in the original British context.
This clearly challenges notions that indigenous influence strategies are unique and distinct to a specific local context. Of course, he immediately got challenged by some people in the audience defending the indigenous approach, claiming that these wimpy scenarios miss the rich context and the social relationships that go with each style.
I think that there are subtle differences in how these influence strategies work and are employed (see for example our qualitative ethnographic work on Brazilian jeitinho here and a set of empirical studies where we also make some theoretical claims about jeitinho vs guanxi here). Yet, there are three major issues that I think the indigenous people are missing.

First, there are limited behavioural options for humans. We are live in social settings with a core family, extended family and a relatively stable set of limited contacts in an extended social network. All these networks are more or less hierarchically structured. We all need to negotiate these networks and there is only a limited set of behavioural strategies for any of us (e.g., ingratiation, calling in favors, returning favors, making compliments, breaking some rules, paying a bribe, giving some gifts... you name it). See work by David Ralston. We can not just come up with something completely different. It is all there. We are humans. Therefore, people in most contexts will be able to recognize and distinguish particular types of behaviours. Hence, people can call a spade a spade... even if it looks a bit funny shaped.

Second, the functions of all these behaviours are to solve problems. It is the functionality of these behaviors, even if not socially approved and even considered illegal (think of corruption or nepotism), it still gets things done. This is why they are so widespread and so similar in form. We made this argument and showed some data supporting this claim here.  Peter Smith and his colleagues also found similar results in their cross-national study. Think behaviours - think functions. And think power corrupts... probably as universal a function of human behaviour as there can be.

Third, many of these behaviours are locally embellished, discussed, criticized, analyzed, debated. By doing this, these behavioural strategies take a life on their own in the minds of concerned members of a community. Go to Brazil and talk to them about jeitinho - you will be listening to complaints for hours - hopefully while having some good cool caipirinhas. Go to Lebanon and ask somebody about wasta - and better have a good shisha or coffee next to you, because you are not going to move for a while. These behaviours are often recognized as problematic, but they are so damn useful and this is why they continue. At the same time, discussing and gossiping about them becomes a reinforcer of the social norm and therefore serves as an identity marker. The behaviour is not just a behaviour anymore, but has taken a cultural life of its own. Therefore, it has to be unique - you can't say that another place has also something that really seems to be jeitinho... or wasta... or guanxi. It is what makes us who we are as people... So dare you say that somebody else may have come up with something similar.

So how does my claim that there are subtle differences fit in with that? I think the first and second point are the answer to that. There are a number of limited behaviours and strategies that people can use to solve problems. The nature and type of problems will differ slightly by context. Therefore, some behaviours will be more common or be expressed with greater force or variety than others. Hence, there is a matrix of behaviours which is latently present in all contexts, but then is expressed to slightly different degrees. Some patterns of the behavioural matrix may be missing or be expressed very weakly in some places. Others may take a particular form due to the different social relations- compare the loose social relations in Brazil which allows more flexibility in social norm bending with the still relatively strong family networks in China that may be less flexible. So what differentiates the various styles is how the matrix is filled with specific behaviours in a specific context. Jeitinho may be a bit more norm breaking, wasta a bit more relying on social hierarchy, guanxi a bit more social relationship harmony focused. But the matrix is there. It is recognizable. It has blends of the same ingredients. It is this matrix that makes us human and helps us to interact with anyone in the world. It is what makes us humans.

A Brazilian will recognize Chinese guanxi and know what it is all about. A Russian will painfully remember some personal experiences when hearing an example of wasta in Lebanon. We all can understand what happened in Beirut this morning - even though we may not want to do or can not do it ourselves (even though I have to admit it would be bloody awesome sometimes to force that damn train or bus to come back when I just missed it... Just saying... :).

Unpackaging culture & cultural differences

One of the most fascinating questions arises when we observe that individuals in a different cultural system behave or act in a different way. Why do they do that? What is the explanation or reason for showing these particular behaviours or responses? For example, we may have stepped on an exotic island and observe that the inhabitants eat way more chocolate ice cream that we do. Or they may tell you that there are lots of little ghosts out there taking care of them, many more than you ever thought would be possible to inhabit a small island like this. Or they may simple say in some interviews or surveys that they do not like to work as hard as you normally would expect in adult samples. How could we explain any of these differences?

Given the perpetual problem of potential bias in comparative research, we can never really rule out that our observations were simply erroneous - we might have had the wrong instruments, there may have been language problems in interactions (remember Lost in Translation?), we may have mis-interpreted the data or it may have simply been a chance difference. 

One persuasive idea that has been around for quite a while in the social sciences is the idea of unpackaging. The terms goes back to a classic study conducted by Whiting and Whiting (1975). They orchestrated a large ethnographic study of child development among six communities: a New England Baptist community; a Philippine barrio; an Okinawan village; an Indian village in Mexico; a northern Indian caste group; and a rural tribal group in Kenya. They reported differences in a number of psychological processes, socialization and child-rearing patterns. Going beyond just noting these differences, they reasoned that there must be specific contextual variables that could explain the differences found, linking ecological constraints faced by these communities to psychological processes via adaptive socialization practices. For instance, they compared the activities of children from the same families, some of whom were living in cities and others in villages. They also compared families in which young boys helped with baby-tending with those in which girls did the helping. Therefore, these social conditions were linked to observed behavioural differences, leading to one plausible explanation of why they may have occurred in the first place. This is essentially what psychologists study as mediation:  processes and variables that explain the relationship between an independent or predictor variable and the dependent or criterion variable. It is about the causal theoretical processes, the how and why of the observed effects. We often think of mediators as internalized psychological processes of external conditions that lead to other outcomes down the causal chain. In cross-cultural work, it does not necessarily always be an internal psychological variable - it could also be living conditions or social constraints and norms that can act as mediators. 

Put differently, unpackaging studies are extensions of basic cross-cultural comparisons in which the active ingredient presumed to cause the observed differences in psychological processes is directly measured and explicitly tested for its role in explaining the outcome. Have a look at the graphical representation of mediation. Unpackaging culture is one term often found in the literature, others include ‘linkage studies’ (Matsumoto & Yoo, 2006), ‘mediation studies’ (Kirkman, Lowe & Gibson, 2006) or ‘covariate studies/strategies’ (Leung & van de Vijver, 2008).

 For example, Tinsley (2001) found that differences in the conflict management strategies of German, Japanese and US managers were completely mediated by the values held by members of these cultural groups, and Felfe, Yan and Six (2008) reported that individuals’ scores on a ‘collectivism’ scale mediated differences in organizational commitment across samples of Romanian, German and Chinese employees.  

In an ideal test of mediation, the researcher tests whether other relevant variables that are not related to the hypothesis also yield mediation effects. This provides greater certainty in establishing exactly what the causes the results that are obtained. For instance, Y. Chen, Brockner, and Katz (1998) showed that a measure of individual-collective primacy mediated the intergroup effects that they had predicted and found. They then tested whether six other measures derived from the concept of individualism-collectivism also mediated these effects, and found that they did not. Studies of this kind help to clarify the loose and varied ways in which the psychological aspects of individualism and collectivism have been employed by different authors. 

What are some general concerns?

In the above examples, the mediator and dependent or criterion variable were measured using the same or similar methods. If there is some third unmeasured variable that is related or unrelated to the independent variable, we may end up with a situation where it appears that there is mediation, whereas in reality, there is none. Having multiple mediators measured with the same method as the DV may lead to some reassurance about the findings, but ultimately, the best test would be a test using independent methods

Experiments have been much in vogue recently to study cultural differences. One of the major concerns is whether the manipulation was effective or not. This is again the problem of potential bias in comparative studies. If we have a psychological mediator in our experiment that highlights how the manipulation is affecting the DV, then we are much safer grounds in terms of explaining the psychological processes. 

In summary, unpackaging has two important and inter-related features: identification of the theoretical factors or processes that may cause cultural differences in psychological outcomes of interest, and an explicit empirical test of the proposed processes leading to these outcomes. Therefore, it is as much about theory as it is about methods and stats. Having unpackaged an observed difference in behaviour, attitudes or beliefs and having ruled out alternative theoretical explanations (other potential mediators), we can also place much more confidence in our results. I leave it up to you to come up with potentially meaningful variables that we could use in those three semi-silly examples (ice cream, ghosts and motivation). Once you have some mechanism, the next phase would be to test whether the mediator(s) actually do the job. Ideally, this is one of the best ways to rule out measurement bias - explain where the difference came from and that the difference is not driven by artefacts. 

Some more technical explanations are available in Fischer (2009); Leung & Van de Vijver (2008) and Poortinga & Van de Vijver (1987, this is an excellent discussion early discussion with some great multi-method examples). Excellent resources and tutorials for running state-of-the-art mediation analyses are available from on Kristopher Preacher's and Andrew Hayes' websites. Use them!!!!!

Overall, I think this is the most exciting part of cross-cultural research - put on your detective hat and find out where any difference that you perceive in the world ultimately stems from. 

How to run a Conditional ANOVA

Today is a wee bit heavier on the stats side again. If you are interested in Differential Item Functioning and how to do it with an easy to use tool, this is for you...

Aim: Identify differential item functioning in numerical scores across groups in order to decide whether the items are unbiased and can be used for cross-cultural comparisons.

General approach: Van de Vijver and Leung (1997) describe a conditional technique which can be used if you use Likert-type scales. It uses traditional ANOVA techniques. The independent variables are (1) the groups to be compared and (2) score levels on the total score (across all items) as an indicator of the true observed or ‘latent’ trait (please note that technically it is not a latent variable). The dependent variable is the score for each individual item. Since we are using the total score (divided into score levels) as an IV, the analysis is called ‘conditional’.

Advantages of Conditional ANOVA: It can be easily run in standard programmes such as SPSS. It is simple. It highlights some key issues and principles of differential item functioning. One particular advantage is that working through these procedures, you can easily find out whether score distributions are similar or different (e.g., is an item bias analysis warranted and even possible?).

Disadvantages of Conditional ANOVA: There are many arbitrary choices in splitting variables and score groups (see below) that can make big differences. It is not very elegant. Better approaches that circumvent some of these problems and that can be implemented in SPSS and other standard programmes include Logistic Regression. Check out Bruno Zumbo’s website and manual. I will also try and put up some notes on this soon.

What do we look for? There are three effects that we look for.

First, a significant main effect of score level would indicate that individuals with low score overall also show a lower score on the respective item. This would be expected and therefore is generally not of theoretical interest (think of it as equivalent to a significant factor loading of the item on the ‘latent’ factor).

Second, a significant main effect of country or sample would indicate that scores on this item for at least one group are significantly higher or lower, independent of the true variable score. This indicates ‘uniform DIF’. (Note: this type of item bias can NOT be detected in Exploratory Factor Analysis with Procrustean Rotation).

Third, a significant interaction between country and score level on the item mean indicates that the item discriminates differently across groups. This indicates ‘non-uniform DIF’. The item is differently related to the true ‘latent’ variable across groups. For example, think of an item of extroversion. In one group (let’s say New Yorkers), ‘being the centre of attention at a cocktail party’ is a good indicator of extroversion, whereas for a group of Muslim youth from Mogadishu in Somalia it is not a relevant item of extroversion (since they are not allowed to drink alcohol and probably have never been at a cocktail party, for obvious reasons).

Note: Such biases MAY be detected through Procrustean Rotation, if examining differentially loading items.

Important: What is our criterion for deciding whether an item shows DIF or not?

Statistical Procedure:

The procedure requires in most cases at least four steps.

Step 1: Calculate the sum score of your variable. For example, if you have an extraversion scale with ten items measured on a scale from 1 to 5, you should create the total sum. This can obviously vary between 10 and 50 for any individual. Use the syntax used in class.

For example:

Compute extroversion=sum(extraversion1, extraversion2,…,extraversion10).

Step 2: You need to create score levels. You would like to group equal numbers of individuals into groups according to their overall extroversion score.

Van de Vijver and Leung (1997) recommend having at least 50 individuals per score group and sample. For example, if you have 100 individuals in each group, you can maximally form 2 groups. If you have 5,000 individuals in each of your cultural samples, you could theoretically form up to 100 score levels (well actually not, because you would have only 40 meaningful groups in this example since the difference between maximum and minimum possible score is 40). Therefore, it is up to you how many score levels you create. Having more levels will obviously allow more fine-grained analyses (you can make finer distinctions between extroversion levels in both groups) and probably more powerful (you are more likely to detect DIF). However, because you have fewer people in your analysis, it might also be less stable. Hence, there is a clear trade-off, but don’t despair. If an item is strongly biased, it should show up in your analysis independent of you have fewer or more score levels. If the bias is less severe, analyses might change across different options.

One issue is that if you have less than 50 people in each score group and cultural sample, the results might become quite unstable and you may find interactions that are hard to interpret. In any case, it important to consider both statistical significance as well as effect sizes when interpreting item bias.

A simple way of getting the desired number of equal groups is to use the rank cases option. You find this under ‘Transform’ -> ‘Rank cases’. Transfer your sum score into the variables box. Click on ‘Rank types’. First, unclick ‘Rank’ (it will rank your sample, but this is something that you do not need). Second, click on ‘Ntiles’ and specify the number of groups you want to create. For example, if you have 200 individuals, you could create 4 groups. If you have larger samples, the discussion from above applies (you have to decide about the number of levels, facing the before-mentioned trade-off in terms of power versus stability).

As discussed above, it is strongly advisable to interpret effect sizes (how big is the effect) in addition to statistical significance levels. This is particularly important if you have large sample sizes in which often minute differences can become significant. SPSS gives you partial eta-squared values routinely (if you click on ‘effect sizes’ under the ‘options’). Cohen (1988) differentiated between small  (0.01), medium (0.06), and large effect size (0.14) for eta-squared. Please note that SPSS gives you partial eta-squared values (which is the variance due to the effect, independent of the effect of other effects), whereas eta-squared does not take the other effects take into account. Partial eta-squared values are often larger than the traditional eta-squared values (overestimating the effect), but at the same time there is much to be recommended for using partial instead of traditional eta-squared values (see Pierce, Block & Aguinis, 2004, in Educational and Psychological Measurement).

Step 3:  Run your ANOVA for each item separately. The IV’s are country/sample and score level (the variable created using ranking procedures). Transfer your IV’s into the ‘Fixed Factor’ boxes. As described above, the important stuff to look out for is the significant main effect of country/sample (indicating uniform DIF) and/or the significant interaction between country/sample x score level (indicating non-uniform DIF). You can use plots produced by SPSS to identify that nature and direction of the bias (under plots, transfer your score level to the ‘horizontal axis’ and the country/sample to ‘separate lines’, click ‘add’ and then ‘continue’). Van de Vijver and Leung (box 4.3) describe a different way of plotting the results. However, the results are the same, only different way of visualising the main effect and/or interaction.

This little figure for example shows evidence of both uniform and nonuniform bias. The item is overall easier for the East German sample and it does not discriminate equally well across all score levels. Among higher score levels, it does not differentiate well for the UK sample. 

Step 4: Ideally, you would not like to have DIF. However, it is likely that you will encounter some biased items. I would run all analyses first and identify the most biased items. If all items are biased, you are in trouble (well, unless you are a cultural psychologist, in which case you rejoice and party). In this case, there is probably little you can do at this point except trying to understand the mechanisms underlying the processes (how do people understand these questions, what does this say about the culture of both groups, etc.).

If you have only a few biased items, remove them (you can either remove the item with the strongest partial eta-square or all of the DIF items in a single swoop – I would recommend the former procedure though) and recompute the sum score (step 1). Go through step 2 and 3 again to see whether your scale is working better now. You may need to repeat this analysis various times, since different items may show up as biased at each iteration of your analysis.

Problems:

My factor analysis showed that one factor is not working in at least one sample: In this case, there is no point in running the conditional ANOVA with that sample included. You are interested in identifying those items that are problematic in measuring the latent score. You therefore assume that the factor is working in all groups included in the analysis.

My overall latent scores do not overlap: This will lead to situations where the latent scores are so dramatically different that you can not find score levels with at least 50 participants in each sample. In this case, your attempt to identify Differential ITEM functioning is problematic, since something else is happening. One option is to increase score levels (make the groups larger – obviously this involves a loss of sensitivity and power to detect effects). Sometimes, even this might not be possible.

At a theoretical level, it could be that you have a situation where you have generalized uniform item bias in at least one sample (for example because one group gives acquiescent answers that are consistently higher or lower). It also might indicate method bias (for example, translation problems that make all items significantly easier in one group compared to the others) or construct bias (for example, you might have tapped into some religious or cultural practices that are more common in one group than in another – in this case your items might load on the intended factor but conceptually the factor is measuring something different across cultural groups). Of course, it can also indicate a true differences. Any number of explanations (construct or method bias or substantive effects that lead to different cultural scores) could be possible.

What happens if most items are biased and only a few unbiased items remain? In this situation you run into the paradox that you can not actually determine whether your biased items are actually unbiased or unbiased items are biased. This type of analysis only functions properly if you have a small number of biased items, up to probably half the number of items in your latent variable. Once you move beyond this, it means that there is a problem with your construct. If you mainly find uniform bias, but no interactions, you can still compare correlations or patterns of scores (since your instrument most likely satisfies metric equivalence). If you have interactions, you do not satisfy metric equivalence and you may need to investigate the structure and function of your theoretical and/or operationalized construct (functional and structural equivalence). 

Any questions? Email me ;) 

How to do Procrustean Factor Rotation with more than 2 groups

Today, I am continuing the torture with a bit more detail on options for comparing factor loadings across three or more groups within SPSS. This is a crucial issue for cross-cultural research and is becoming increasingly important, because researchers start studying more than two groups. More complex designs are more powerful in uncovering processes that can explain emerging behavioural differences, so this research should be strongly encouraged!

Aim: Compare the factor structure when you have more than two cultural groups, get an estimate of factor similarity

Why are we concerned with Procrustean Rotation? Factor rotation is arbitrary, therefore apparently dissimilar factor structures might be more similar than we think; procrustean rotation is necessary to judge structural and metric equivalence

Statistical Procedure:

The same syntax as for the two group case (see previous post: http://culturemindspace.blogspot.co.nz/2012/03/how-to-do-procrustean-factor-rotation.html) can be run with SPSS, but the greater number of countries adds additional problems. You have various options:

1. Run all pairwise comparisons. However, this will lead to a substantive number of comparisons (especially if you have many samples). This also leads to a number of statistical problems (remember family-wise error rate and increased Type I errors)
2. Select one country as your target group. For example, if an instrument was developed in the US, you may want to compare each group to the US.
3. Compute the average correlation matrix and use it for your factor analysis. The average is sometimes called pooled-within matrix. Therefore, you would compare each sample with the average structure across all samples (this can be done via discriminant function analysis in SPSS, you can then read the resulting correlation matrix into spss and use as an input for your factor analysis - see my discussion of how to do this here). This is highly appealing if you have many samples. This procedure of computing the average correlation matrix as input to the factor analysis can be simplified if (a) you have samples with similar sample size (no sample is dominating others; eg., if you have one sample of 10,000 and three samples of 50 participants each, the large sample is driving the factor structure) and (b) you mean centre each item within each sample prior to the overall factor analysis. This is necessary to account for any group mean differences that might obscure relationships if the samples are pooled. See below for a graphical explanation of why this might be a problem. As you can see, the relationship within each sample is negative, more sleep problems within each sample are associated with less laughter by participants. However, one group is consistently higher, for both the reported sleep problems as well as laughing. There may be reasons of why this is the case (I will come back to this example when talking about multilevel analysis), but for our analysis, combining the two samples would mean that we have a positive relationship across both samples combined (compared to negative relationships within both samples separately). This effect is due to the mean differences across both groups (I will post something soon on the beautiful complexity of these multi-level problems in psychology - very fascinating stuff). As a consequence of this confounding of group differences with individual differences, we need to take any such mean differences into account before we can combine the samples. This can easily be done using the z-transformation option in SPSS (‘Save standardized values as variables’ under the ‘Analysis’ -> ‘Descriptives’ option). 

I believe the last option is the most appealing with large data sets.

 However, cross-cultural psych never stops to be complicated. What happens if you find that some samples show good factor congruence with the average factor structure and others not? Ideally, you would exclude those samples from the average factor structure and re-run the analysis. Proceed iteratively till no sample shows any problems with factor similarity anymore.

If you have lots of cultural samples, you are really curious (and stats savvy) and want to find out what is happening in the strange worlds of culture, you may want to run cluster analysis on the congruence coefficients to identify clusters of samples that show greater similarity with each other. This might provide some interesting insights from a cross-cultural perspective. However, it is computationally demanding and relies on purely statistical criteria. There is a neat paper discussing various options and strategies, written by Welkenhuysen-Gybels and van de Vijver (2001, published in the Proceedings of the Annual Meeting of the American Statistical Association – I think this gives you an idea about what level of analysis we are talking about[1]). You can also download a SAS macro (the link is in the paper) that does much of the computational work for you. I have never worked with SAS, it seems a parallel universe to me and I am fascinated, but scared of it. But there are people who think it is easy. Conceptually, it is a nice tool.  

---

[1] You can download the paper at: http://www.amstat.org/sections/srms/Proceedings/y2001/Proceed/00106.pdf

How to do Procrustean Factor Rotation

Procrustean Factor Rotation

 Today, it is a little bit less light-hearted, but hopefully a bit more practical. 

Aim: To make factor structures maximally comparable & provide a statistical estimate of factor similarity

Why are we concerned with Procrustean Rotation? Factor rotation is arbitrary, therefore apparently dissimilar factor structures might be more similar than we think; procrustean rotation is necessary to judge structural and metric equivalence

Statistical Procedure:

 A SPSS routine to carry out target rotation needs to be run (adapted from van de Vijver & Leung, 1997)

The following routine can be used to carry out a target rotation and evaluate the similarity between the original and the target-rotated factor loadings. One cultural group is being assigned as the source and the second group is the target group. The varimax rotated (or unrotated) factor loadings for at least two factors obtained in two groups need to be inserted. The loadings need to be inserted, separated by commas and each line is ended with a semicolon. The last line is not to end with a semicolon, but with a ‘}’. Failure to pay attention to this will result in an error message and no rotation will be carried out. To use an example, Fischer and Smith (2006) measured self-reported extra-role behaviour in British and East German samples. Extra-role behaviour is related to citizenship behaviour, voluntary and discretationary behaviour that goes beyond what is expected of employees, but helps the larger organization to survive and prosper. These items were supposed to measure a more passive component (factor 1) and a more proactive component (factor 2). The selection of the target solution is arbitrary, in this case we rotated the East German data towards the UK matrix. 

Table 1. Items and varimax-rotated loadings in each sample separately

           

	UK		Germany
	Factor 1	Factor 2	Factor 1	Factor 2
I am always punctual.	.783	-.163	.778	-.066
I do not take extra breaks.	.811	.202	.875	.081
I follow work rules and instructions with extreme care.	.724	.209	.751	.079
I never take long lunches or breaks.	.850	.064	.739	.092
I search for causes for something that did not function properly.	-.031	.592	.195	.574
I often motivate others to express their ideas and opinions.	-.028	.723	-.030	.807
During the last year I changed something. in my work....	.388	.434	-.135	.717
I encourage others to speak up at meetings.	.141	.808	.125	.738
I continuously try to submit suggestions to improve my work.	.215	.709	.060	.691

Syntax:

This can not be done using the windows interface within SPSS. You should run a factor analysis in each sample separately first. Use Varimax (orthogonal) rotation.  Then insert the loadings in the loadings and norm matrices in the SPSS syntax described in Fischer and Fontaine (2011, in Matsumoto and Van de Vijver’s Cross-Cultural Research Methods in Psychology). I can also email this syntax to you (contact me at Ronald.Fischer@vuw.ac.nz).

The start of the syntax is printed below. Be careful to separate the loadings by a ‘,’ and the last loading for each item needs to be followed by ‘;’. The last loading should be indicated by }.

matrix.

compute LOADINGS={

.778,    -.066;  

.875,    .081;   

.751,    .079;   

.739,    .092;   

.195,    .574;   

-.030,   .807;   

-.135,   .717;   

.125,    .738;   

.060,    .691     }.

compute       NORMs = {

.783,    -.163;  

.811,    .202;   

.724,    .209;   

.850,    .064;   

-.031,   .592;   

-.028,   .723;   

.388,    .434;   

.141,    .808;   

.215,    .709}.

…

Output and Interpretation:

The edited output for this example is shown below. It shows the rotated matrix of the group (East Germany in our case) that was rotated to maximal similarity:

*********************************************************************

Run MATRIX procedure:

FACTOR LOADINGS AFTER TARGET ROTATION

   .77  -.10

   .88   .04

   .75   .05

   .74   .06

   .22   .57

   .00   .81

  -.10   .72

   .16   .73

   .09   .69

DIFFERENCE IN LOADINGS AFTER TARGET ROTATION

  -.01   .06

   .07  -.16

   .03  -.16

  -.11   .00

   .25  -.03

   .03   .08

  -.49   .29

   .02  -.08

  -.13  -.02

Square Root of the Mean Squared Difference per Variable (Item)

   .05

   .12

   .12

   .08

   .18

   .06

   .40

   .05

   .09

Square Root of the Mean Squared Difference per Factor

   .19   .13

IDENTITY COEFFICIENT per Factor

   .94   .97

ADDITIVITY COEFFICIENT per Factor

   .86   .92

PROPORTIONALITY COEFFICIENT per Factor

   .94   .97

CORRELATION COEFFICIENT per Factor

   .86   .93

------ END MATRIX -----

The output shows the factor loadings following rotation, the difference in loadings between the original structure and the rotated structure as well as the differences of each loading squared and then averaged across all factors (square root of the mean squared difference per variable column).

The first matrix could be pasted in a new table, showing the rotated loadings (instead of using the loadings from the original analysis as reported above in the table). The second matrix shows the differences after rotation. You should look for large values, because they indicate that some items are problematic. A low value would indicate good correspondence.

The column of values entitled: Square Root of the Mean Squared Difference per Variable (Item) gives you information about each item. The larger the value, the more problematic is an individual item. The next row (Square Root of the Mean Squared Difference per Factor) shows the same information per factor. Again, smaller values are better, larger values indicate trouble for a particular factor. There are no hard and fast criteria for any of these indices above, you should look at the relative values and particular discrepant values.

The most important information is reported in the last four lines, namely the various agreement coefficients. As can be seen there, the values are all above .85 and generally are beyond the commonly accepted value of .90. The most common indicator is Tucker’s Phi which is called Proportionality coefficient here.

It is also worth noting the first factor shows lower congruence and that the estimate vary across indicators. An examination of the differences between the loadings shows that one item (During the last year I changed something. in my work....) in particular shows somewhat different loadings. In the British sample, it loads moderately on both factors, whereas it loads highly on the proactivity factor in the German sample. Therefore, among the British participants making some changes in their workplace is a relatively routine and passive task, whereas for German participants this is a behaviour that is associated more with proactivity and initiative (e.g., Frese et al., 1996). We might want to exclude this item and re-run the analyses. Overall, we could cautiously conclude that our scales meet structural equivalence and most items might even meet metric equivalence (although this syntax routine does not provide a statistical test for this higher level of equivalence). 

Good on ya... if you made it to this point ; ) Hope your eyes are looking slightly better than that of a Tarsier...