Extreme rituals in transition: Some reflections

I just returned from a turbulent five days in Phuket, Thailand. Although many people associate Phuket with beach, sand and holidays, it is also one of the biggest centres for a Buddhist religious festival, which in all likelihood is one of the most extreme rituals still existing in modern societies. The rituals have a long 5,000 year history, going back to pre-Chinese rituals (see work by Margaret Chan). The ceremonies arrived in Thailand with Chinese groups that worked in mines during the 19th century. I will share some brief impressions of my short visit. I could talk about what I saw for ages, so this is just a quick rambling of thoughts and observations. The purpose was to collect some data on the values and personality of participants in this fascinating ritual and to examine health and well-being effects of a ritual like this. Watch this space for the results coming soon...

The Processions

The festival lasts about 10 days and involves thousands of worshipers from Thailand and the region. It starts with the raising of a lantern pole which invites some Emperor Gods to descend from heaven.

In the following days, believers who act as mediums go into trance and impersonate various gods and warriors. These mediums will walk over heaps of glowing coal and climb bladed ladders (both events have specific religious significance and are performed on days before important events of the larger ritual). The most impressive show of these mediums are the processions though. Starting in the early hours of day, believers will go into trance at the temple which will sponsor and organize the procession of the day. Supporters will dress the mediums (once they are in trance) with the characteristic Chinese style garments and hand them artefacts that show their heavenly power (typically some flag with Chinese inscriptions and a whip with a snake-shaped grip).

Depending on the spirits that possess them, a good number of the believers will engage in piercings and acts of self-mortification. The piercings in some cases can be hard to stomach for outsiders. It ranged from needles and skewers to guns, beach umbrellas, metal saws, swords, tree branches, basket ball hops attached to a car tire, beads and any unimaginable object. I asked how people decide what objects to drive through their cheeks. The answer was that they dream it (the spirit telling them what to do). Apparently, the greater the pain, the more blessing for the individual and the community because this wards off evil spirits (many of the mediums are supposedly fierce warriors that battle evil).

There was blood, sometimes lots of it. Some were cutting their tongue on swords or hit their back with sharp objects. There were moments when I thought it went a bit too far (I remember one moment when I was wondering whether the weight of the object inserted into a person's cheek would rip off the skin completely from his face - well, it did not and I was glad about that. I spare you the photos...). The mediums parade through town in a procession that lasts several hours (covering up to 20 km in distance). They walk barefoot over hot asphalt in the tropical heat.

Onlookers invite them to bless them, their family and their house. In order to increase the torment, firecrackers are thrown on these mediums, with the belief that the more noise being made, the more fortune for the family. Despite all the goriness of the piercings, the throwing of the firecrackers leading to war-like scenes left probably the most impressions on me. I felt transported into a war zone between aliens from a different planet.

The smoke from the firecrackers made breathing difficult. Visibility was reduced to a few meters, only the light of the firecrackers able to break through the fog. The sound of the firecrackers echoing back and forth between the buildings in the narrow streets. Transported into an apocalyptic war zone, the best protection of my face was to hold my camera and shoot back (photos against firecrackers). It was hard to imagine how people were able to endure this for hours. Trying to shoot photos, I got a few hits from firecrackers. The physical sensation of a firecracker exploding on your back, in front of your face or your bare feet (call me well-prepared for venturing out in sandals) is not to be underestimated.

Once back at the temple, the piercings were removed by priests and other participants in the ritual and the spirits were asked to leave (ie. exorcised) in a small ceremony in front of the shrines inside the temple. Soon after the procession, there would be more rituals... a non-stop chain of religious rituals and a kaleidoscope of passion, devotion, shared food, blood and ritualistic suffering.

The other extreme: Silence

In addition to these spectacular extreme rituals, there are other ceremonies that oscillate to the other extreme of tedium and stillness. One of the rituals called Propitiation of Seven Stars involves the warrior mediums in trance guarding a platform structure where more mediums assembled. The temple community is crowded around in silent prayer. One individual beats a drum monotonously for about half an hour. There is not movement and no sound except the tiny dingdingdingding of this drum for the whole period. Unfortunately, I could not really got more information on what is happening during this ritual beyond some general vague stories. But the contrast between extremes could not have been stronger.

Transitions: Between Ancient Traditions and Modernity

Transitions and change were notable. For one, tablets, cell phones and cameras were everywhere to document the extremities. Often it appeared that friends and family of the medium were documenting every single step of the medium, as if to document the suffering to document for posterity. What are these people doing with these documents? Many of the mediums would stop and pose for photos - what is the purpose of this exhibitionism? 

There is a line of research that argues that costly rituals bring communities together and increase the status of individuals who engage in the most extreme forms (see for example work by my colleagues Joseph Bulbulia and Markus Frean). Are these documents used later to increase the status and prestige of the individuals who engage in these activities? Is the posing for pictures a mechanism to increase one's self-esteem? Is the greater shock value of extreme acts translated into social capital within the community?

The extreme nature of some of these piercings was striking. I had seen some postings of such piercings before on the internet, but seeing it in person is different from seeing it from the comfortable safety of your computer. There seems to have been a shift towards more extremes. We did some work on Thaipusam last year in Mauritius (a major field site for studying rituals in the field, directed and organized by Dimitris Xygalatas). In conversations with Temple leaders, they commented that piercings and size of the kavadees that believers carry has increased over the years. An interview with a medium from Phuket posted on the internet came to similar observations. The choice of piercings is supposedly dictated by spirits during dreams.... To me it nearly seemed like a ratcheting of extreme actions (think of the 'keeping up with Joneses' effect). To what extent is the posting of extreme piercings, sensational violence, and all sorts of other evidence of self-inflicted harm on internet and global media leading to a 'spiritual arms race'? 

Yet, at the same time, other changes led to changes in the rituals. Blade-walking and bladed ladder climbing used to be a significant part of the ritual. Yet, these practices in the times of HIV are dangerous and can infect a large number of participants. For this reason, a number of temples have abandoned this practice. 

Some changes were also quite cute. It was fun to see how kids and youngsters went through some of the more boring parts of the ritual by secretly using their cellphones and tablets for entertainment. Some of these folks had devised ingenious techniques for entertaining themselves while pretending to be good believers. 

In many ways, these simple observations of human reactions during a period that felt so extreme punctuated the humanness of it all. The pained expressions of bystanders, the open eyes in awe and disbelief of some of the suffering that was displayed, the innocent attempts to ward off both shock and boredom revealed the human aspect. It is these small observations that brought it home to me that through the extremity it actually shows the universality of what it means to be human. There will be people who go to extremes for all sorts of reasons in all cultures. There will be pain and boredom in all places. And it is the reaction to these universal elements that reveals our shared humanity.  

I am glad I went. Many thanks to Janpaphat Kruekaew for introducing and opening this fascinating world for me. Quite often when I was too tired to continue, she tirelessly continued asking people and getting interviews and responses from participants. It was a humbling experience working with her. I am looking forward to going back and getting a better understanding of the minds of some of these people, those who go to extremes and those who just stand by and watch the procession unfold in front of them.

Sexy, hot and easy! Should we trust student evaluations of university courses and teachers?

How should we rate the effectiveness of university teachers and university education in general? This is a million dollar question that is hardly ever been questioned, yet determines the course of countless lives on both sides of the divide (student and teacher). 

Teaching evaluations are treated with suspicion by profs and teachers, but are loved by university bureaucrats and administrators. Students may often not realize, but these evaluations, specifically the mean numbers that come out after some basic number crunching often have a huge impact on the careers and success of academics. A rather small difference between a 1.9 and 2.5 can determine whether somebody gets promoted, is hired or gets a bonus. In extreme cases, a teacher may lose a contract. So how much evidence is there that these numbers provide good evidence of teacher effectiveness? And are teachers who do get higher ratings actually those teachers that help students succeed in other courses?

A number of recent studies cast some big doubt on the usefulness of these criteria. Let's start with a fun example. 

Quality of teaching or just easy and 'hot'? 

James Felton, a Professor of Finance at Central Michigan University and colleagues examined the evaluations submitted to http://www.ratemyprofessors.com/. There are a couple of different criteria that students can rate their professors on. The two core areas are 'helpfulness' (how helpful and approachable a teacher is) and 'clarity' (how organized, clear and effective is a teacher). These two are averaged to get a rating of overall teaching quality. There are two more evaluations though. The first one is 'easiness', meaning how easy or difficult the classes are and how much work is needed to get an A. The second is 'hotness', a simple rating of whether a student thinks that a teacher is hot or not. Obviously, we would want to have teachers that are effective and helpful, but these perceptions should not be driven by how easy a course is or how attractive a teacher is. When looking at the data from ratings for 6,852 profs from 369 institutions... the answer is that the hotter you are and the easier your course is, the better are your evaluations. The correlation between easiness and quality is a whopping .62, whereas hotness and quality correlate .64. 

They offer this explanation: 

We see Quality as a function of Easiness, but it could be argued that Easiness is a function of Quality, where professors who are skilled in the classroom take difficult material and make it seem easy. We wish that were the case, but we see Quality as a function of Easiness the majority of the time for two reasons. First, as stated previously, Ratemyprofessors.com (2004) defines Easiness as the ability to get a high grade without having to work very hard. Second, professors with high Easiness scores usually have student comments regarding a light work load and high grades. Similarly, we see Quality as a function of Hotness, but it could be argued that Hotness is a function of Quality, where a brilliant professor, regardless of physical appearance, is considered sexy by his or her students. Again we wish that were the case, but most student comments point toward Quality as a function of Hotness when they focus on physical characteristics of their professors that could be captured in photographs.

The lesson that might be learned from this correlational study is that it does not hurt to dumb down your lecture content and hit the gym (well, the latter would be good regardless). 

Lecture fluency or welcome back, Dr Fox...

Now, let's enter study number 2. Shana Carpenter and colleagues from Iowa State University in a study recently published showed students a short video of the same teacher presenting the same material. The major difference was that in one video the prof acted in what was called a fluent way: upright, confident, with eye contact and speaking fluently without notes. In the other condition, the prof acted disfluent: slumped, looking away, speaking haltingly and relying on notes. In two experiments, students were tested on how much they actually learned and ratings of the prof were also obtained. The results very clearly showed that the fluent prof was rated much better (surprise surprise), but also that students thought that they had learned more and would remember more from the fluent prof compared to the disfluent prof. However, when later tested, there were no differences between the two groups. This means instructor fluency increases perceptions of learning but not actual learning! There were also some curious smaller findings. For example, for the disfluent group -when given the opportunity to read the transcript of the lecture, students who spent more time rehearsing had higher test scores. This was not the case for the fluent group. It is an ambiguous finding, but could indicate that fluent lectures may decrease the attention paid to study material when preparing for an exam. Not sure whether this is desirable.

This really sounds like the famous Dr Fox effect.  Talk nonsense as long as you are dynamic, engage the audience and are make jokes...

Some disturbing findings when using random assignment of students to profs

The most concerning study though used a controlled random assignment of students to courses that overcomes a lot of the shortcomings of previous studies (including self-selection of students to courses and professors). Scott Carrell and James West studied student achievement and course feedback as students moved through mandatory classes in maths, science and engineering. The unique aspect of their study is that professors rotated in sections of the course, assessment was not done by the professors themselves and students were randomly allocated to professors (but all studied the same content). A first finding that is of practical importance is that less academically qualified instructors got students more (erroneously?) interested in the topics which resulted in better immediate student performance, but then led to lower scores in follow-on related courses. More experienced and qualified professors in contrast had students that did not well in the introductory classes, but those students than excelled later on. Those students were able to build on what they had learned during the initial courses. 

What is even more important is that professors who were rated positively by students did better in the initial courses. However, the rating of the effectiveness of the professor did not predict later performance! In fact, in a number of cases the correlation flipped - students studying with the more highly rated professors did worse in half the courses than those who studied with a prof who was not rated as highly (note: only one of these correlations was significant - the point remains the same though: ratings of teacher effectiveness does not predict long-term student achievement). As Carreel and West argue:

'Since many U.S. colleges and universities use student evaluations as a measurement of teaching quality for academic promotion and tenure decisions, this finding draws into question the value and accuracy of this practice.' 

Are there alternatives? Yes! 

The reliance on student evaluations for courses and teachers is problematic, if these evaluations are not considered in a larger context of what is achieved in a course. In the business world, this has been long realized. Teaching is effectively training. In the organizational world, Donald Kirkpatrick developed a famous four stage model of training evaluation. The four criteria for the evaluation of the effectiveness of training are:

1. Student reactions - this is essentially equivalent of student evaluations, assessments of students thought they had learned and how they felt about the course/the teaching
2. Learning - this is measured by the increase in knowledge or capability after the course, we could consider the test performance in a test or exam as a good measure of this (of course only if the assessment is independent of the teacher - see above the problem with the easiness of a course)
3. Behaviour change - this refers to the changes in the behaviour outside the teaching environment that are a result of the teaching, including applications of what has been learned to new situations outside the teaching/training context
4. Results - this is the effect of the teaching on the business or the larger environment that results from the performance and the behaviour changes induced by the teaching/training

The 3rd and 4th points are what universities (and society) should be concerned about. There has been a lot of questioning of the value of tertiary education recently (see for example here, here and here). These criteria can help in re-adjusting both the focus of universities as well as criteria that are used to evaluate professors. 

Students and society deserve better, not just the profs ;)

Comments are welcome as usual :)