Monday, 28 January 2008

Handling misrepresentation

Marilyn and I do a reading from William Barclay’s daily Study Bible each day. In today’s reading based on II Corinthians 12:1-10, Barclay relates the following story.

“There is nothing so hard to face as misinterpretation and cruel misjudgement. Once a man flung a pail of water over Archelaus the Macedonian. He said nothing at all. And when a friend asked him how he could bear it so serenely, he said, “He threw the water not on me, but on the man he thought I was.”” (p 260).

The key here is to be confident about who and what we are. People do pour cold water on our ideas but if we have confidence in what we are doing then we should not allow their uniformed criticism to turn us from our path. Alternatively, maybe, they think they know what we are endeavouring to achieve and unintentionally cause more problems for our chosen direction. Because they think that they understand us, they act without really listening to what we say. Alternatively, maybe they simply have a different conception of the circumstances and act according to their belief without seeing that our conception of the circumstances is different and they are actually being an obstruction rather than assisting. Are we able, like Archelaus, to see that they are not throwing water on us but rather on their misconception? Can we continue to move forward with confidence?

We should not however be arrogant about our own conceptions. We too could be misrepresenting the situation. Are we prepared to listen and learn? Are we willing to evaluate and to see our own shortcomings?

Research is often like that. Different researchers have conceptions about what is valid research and they will write another’s work off without really seeing what has been done or the possible significance of the work. Taking time to listen and to try to understand can often bring fruits. We see things in a different light and learn.

My own research is about the conceptions that practitioners have of objet-oriented programming. It is quite surprising the differences even amongst very experienced people. However, it is not simply in the writing of the code that there are differences in perception. There are differences in understanding with respect to how to approach or manage the task. There are also differences in how to approach the teaching of programming. It is common to see people talking past each other rather than hearing what the other is saying. Often the problem is about particular ways of approaching the task rather than the principles that determine what needs to be done. Are we able to step back in the face of disagreement and see things from the others perspective. Let us not be caught pouring cold water on someone’s idea without first attempting to understand where he or she is coming from and what drives his or her thinking.

One of the nice things about the BRACELet research group is that we are learning to create ways to gather data to verify or disprove our assumptions about novice programmers. Instead of drawing a conclusion without evidence, we are looking for ways to verify our suspicions. It is all about verifying our conceptions and ensuring that we are correctly representing the situation.

The same applies to faith and a lot of life. If we judge without hearing then we miss the real need and alienate those that we think we are trying to help. Take the time to listen and not to prejudge. You might be surprised at the difference that it makes.


Barclay, W. (1975). The letters to the Corinthians (Revised ed.). Edinburgh: The Saint Andrews Press.

Sunday, 27 January 2008

Being encouraged

I have just returned from participation in the Australasian Computing Education conference where I presented some of the work being performed by the BRACELet project. With this project, we are investigating the knowledge of novice programmers and how we might be able to help them learn. Over the last few years, the project has conducted a number of data gathering exercises from a number of different tertiary institutions. Our results have provided some interesting insights and are beginning to attract some attention.

The conference proved to be a real encouragement for me. It is so easy to be focussed on our own work and forget that others are struggling with the same issues. The BRACELet project has involved a lot of collaboration, analysis, and writing. One of the strengths of collaboration is that each participant can work more to their strengths and we can learn how to deal with areas where we have uncertainty. I have learnt a lot about research through this collaboration and my major contribution has been in doing analysis and suggesting a possible analysis method.

With this conference, I began to see how our work has begun to impact others and is now being expanded into a number of Australian universities. The Australian senior researcher, involved in the project, being the keynote speaker for the conference, helped this as he promoted the work of the project.

The project has worked in cycles. We set small goals that were not very ambitious but designed to provide additional data about the way that novice programmers approach initially the reading of programs and now the writing of programs. After getting data for the each question, we analyse it and publish the results and then look at what questions might be answered next. We are not too ambitious about what we might learn but rather set targets that can easily be achieved. The result is a stack of papers that are beginning to influence the way members of the group teach and assess.

One thing about this project is that we aim to gather data through existing mechanisms (i.e. assessments such as final exams). The questions that we ask should help to identify whether the learners understand programming but they also provide us with additional data that can help us understand the novice programmer issues.

Below is a list of the papers that I know about although I am sure that there has been a least two others. It becomes difficult to keep track of what is happening as the group expands. It is certainly an exciting group to be part of.

Some of the BRACELet papers so far

Lister, R. (2008). After the gold rush: toward sustainable scholarship in computing. In Simon & M. Hamilton (Eds.), Tenth Australasian Computing Education Conference (ACE 2008) (Vol. 78, pp. 3-18). Wollongong, NSW, Australia: ACS. From

Thompson, E., Luxton-Reilly, A., Whalley, J., Hu, M., & Robbins, P. (2008). Bloom’s Taxonomy for CS Assessment. In Simon & M. Hamilton (Eds.), Tenth Australasian Computing Education Conference (ACE2008) (Vol. 78). Wollongong, Australia: Australian Computer Society Inc.

Clear, T., Edwards, J., Lister, R., Simon, B., Thompson, E., & Whalley, J. (2008). The Teaching of Novice Computer Programmers: Bringing the Scholarly–Research Approach to Australia. In Simon & M. Hamilton (Eds.), Tenth Australasian Computing Education Conference (ACE2008) (Vol. 78). Wollongong, Australia: Australian Computer Society Inc.

Thompson, E., Whalley, J., Lister, R. & Simon, B. (2006) Code Classification as a Learning and Assessment Exercise for Novice Programmers. The 19th Annual Conference of the National Advisory Committee on Computing Qualifications: Preparing for the Future — Capitalising on IT. Wellington. ISSN 1176-8082. (Received Citrus Award for Collaborate Research and highly commended in Best Paper Award)

Lister, R., Simon, B., Thompson, E., Whalley, J. & Prasad, C. (2006) Not seeing the forest for the trees: Novice programmers and the SOLO taxonomy. Innovation and Technology in Computer Science Education (ITiCSE 2006). Bolonga, Italy.

Whalley, J., Lister, R., Thompson, E., Clear, T., Robbins, P., Kumar, A., et al. (2006). An Australasian study of reading and comprehension skills in novice programmers, using the Bloom and SOLO taxonomies. In D. Tolhurst & S. Mann (Eds.), Eighth Australasian Computing Education Conference (ACE2006) (Vol. 52, pp. 243-252). Hobart, Tasmania, Australia: Australian Computer Society Inc. ISBN: 1-920-68234-1, ISSN: 1445-1336

Whalley, J. (2006). CSEd research instrument design: the localisation problem. In S. Mann & N. Bridgeman (Eds.), The 19th Annual Conference of the National Advisory Committee on Computing Qualifications: Preparing for the Future — Capitilising on IT (pp. 307-312). Wellington: National Advisory Committee on Computing Qualifications.

Monday, 14 January 2008

Cycling Commissaire

I spent my weekend commissairing at the the New Zealand Elite National Cycling Road Race championships. The commissaire is the referee and is responsible for ensuring that the rules of racing are upheld but even more importantly, they are responsible for the safety of the riders and ensuring that the public can enjoy the race.

By their nature, road time trials and road races means that the riders are out of site of the stationary public for a lot of the race. There becomes an issue of how you can keep them informed and try to encourage them from trying to continually get ahead of the race so they can see the riders pass another point on the road.

Riders blatantly ignoring the rules of racing and the rules of the road don't just put themselves in danger, they put other riders and other road users in danger. They also put at risk the future of road racing. When racing is held on open roads as they were over the weekend, the riders effectively have only the left-hand side of the road to race on (in New Zealand, we drive on the left). Crossing the centre line should be considered the same as taking to the grass on the side of the road. It hinders your progress and of course puts you in the path of oncoming traffic. This can irritate other road users who then put pressure on councils to ban road racing.

For road races, a major role of the commissaire is knowing the state of the race at all times. This can prove quite difficult when there are riders attacking off the front of the peloton (the main group of riders in the race) and others having mechanical failures or punctures. In all major road races, there is a convoy of officials, selectors, media representatives, and support vehicles that need to be managed. All this falls upon the team of commissaires to manage. The task can be quite demanding depending where you are on the road.

Even as a finish line judge or time keeper, you want to know where the riders are during the race. Each lap, you attempt to record all the race numbers and the time gaps. This makes it easier when the race finishes to check that you have accounted for all riders and that lapped riders are not accredited with incorrect placings. Lapped riders are often pulled from the event so that they don't hinder the main race leaders. You can also communicate with the on road commissaires about where riders are or which riders have withdrawn from the race. It is also possible that you can then update the public on the state of the race. Supporters always want to know where there rider is. Was (s)he at the front of the race or has (s)he been dropped and floundering at the back of the race. Fortunately, we had the support of a team focused on informing the public.

If you don't keep track of how the race is unfolding, when it comes to the finish, you have no idea how many riders to expect in the sprint or even to finish. We had a little bit of a problem with this on Sunday because of difficulties with radio reception. Still at the end of the race, we had a fairly clear idea of which riders had withdrawn and where key riders were on the road. If you don't know this you can end up standing on the side of the road waiting for the last rider and not being sure whether there is actually a last rider.

The commissaire's role is quite challenging and there is always something new to learn at each even. Unfortunately, there are also riders that you have to talk to because they are so focussed on getting a good time or finishing as high up as possible that they forget that there are rules for racing and of the road that they need to follow. As a commissaire, the first role with respect to rider safety is one of educating the riders. Most of this is done through attending club and regional events. Here when you see riders doing stupid things on the road, you can go and talk with them and help them to understand why their behaviour wouldn't be tolerated at the higher levels of the sport.

The top commissaires have a real love for the sport and also have a really good working relationship with the riders. The don't like having to be enforcers of the rules but know that if the racing is to be fair, safe, and enjoyable that they need to do so. In the same way, they enforce the rider uniform codes because these help the public, promoters, and sponsors. The right uniform helps the public know who is part of which team and for promoters and sponsors, it is all about getting their value for their dollar. All sponsored sports people are advertising billboards and to ensure that sponsorship is retained in the sport then keeping the sponsors and promoters happy is a key part of the sport persons activities. Portraying the right image through what you wear on the bike and off the bike helps portray a concern for your sponsor. The same applies for following the race and road rules. A sponsor doesn't want their name associated with bad publicity.

In the end, commissaires and all officials at events are interested in promoting the sport.

Wednesday, 9 January 2008

Why research?

I was reading, this morning, Barclay’s commentary on II Corinthians 9:1-5 where he is reflecting on the reasons why people give. It occurred to me that a similar reflection might apply to why people do research especially in an academic setting or at least why they do certain types of research. What do I mean?

Increasingly in an academic context, the pressure is on for a high quality publications record. In order to have such a record, there needs to be some backing research. Therefore there is a certain amount of compulsion to do research. If I don’t write x number of papers this year then I will lose my y research rating and I will …

From a personal perspective, I ask “why am I struggling with writing a PhD?” The answer is quite simple. I was employed as a university lecturer and if I wanted to continue as a university lecturer, I needed a PhD. Some would say “what about the prestige of being able to say you have a PhD?” Sorry but qualifications only have value to me if I can use them for something and by now in my life (mid 50s), I have found that I can learn most things that I need to in order to get a job done without the need for a qualification. The PhD would simply act as a certification that I could do research but I can do research without it and to some extent I was. I just wasn’t being as meticulous with gathering the data and never thought of writing it up. In fact, I wasn’t as rigorous about what constituted proof of my theories. Yes, there is value in having to publish the results of your research work.

The question is then why get involved in the research projects that I have. My choice of project and research technique is more to do with my personal interests and with what I needed to know in order to achieve in my work. Not surprisingly, my research revolves around the scholarship of teaching and with what it means to have learnt something. Even my definitions of “what it means to have learnt something” are influenced by my personal convictions.

This personal value of the research has also influenced my choice of research method. I couldn’t see the value of doing a survey. Past experience told me that they only confirmed or disconfirmed my suspicions. Also completing a surveys told me that often the questions didn’t ask me what I really wanted to say about the subject. They left things out that the researcher wasn’t interested in or maybe hadn’t considered as a possibility. Surveys aren’t of interest. I wanted to know what people were actually thinking and what they had really learnt. My data gathering had to gather much richer data than would be gathered form a survey or the administration of some test. The result is that I have conducted interviews. In another project, we are using multi-choice questions but we are actually finding that short answer questions are telling us more about the learner but that is different issue.

My research is about what I wanted to know and what I wanted to be achieving with my students. Yes, others are interested in that as well and it is good to share with others what I have learnt and to work with them to implement and test some of the findings.

Now as I write my thesis as a full time students, I am thinking whether I will ever get to apply these results or work with people that might want to. Fortunately, I am still involved in a research group (BRACELet) and this gives me an opportunity to talk with people who are still involved in teaching. I intend to remain involved as long as I can afford the time to participate and the group still wants me involved. Why? Well, the things that we are discovering about the learning of programming are proving of value and that interests me. As a result, the quality of the research is improving.

What I would like to contend is that when research is done simply to ensure that the publication count is maintained then there is no guarantee that the quality of that research is going to be there. Yes, there are review processes but even these are like a lottery but that is another story. The thing is that if I am under compulsion to publish then I will look for research projects that will deliver quick publications rather than results of value. This to me raises the question of whether we are the research measures put in place are meaningful measures. A measure causes certain types of behaviour. In this case, a compulsion to perform research in order to maintain a research outputs count. As long as the researcher can publish in appropriate books, journals, and conferences, the value of the research is meaningless. It makes me wonder how much research is performed that is really of little value other than to maintain the publication record. Is that a suitable research question for someone?

Thursday, 3 January 2008

How much do I believe in …

My thoughts today have been drawn to the idea of how much do I believe in the things that I claim to believe in? I am sure many of us have come across the fanatics who seem to be willing to die for their cause. Some quickly offend simply because they are so adamant about their beliefs that they argue for them without concern for the reaction of others.

However, there is something positive about believing in something enough that it acts as a driving force to achieve even when the obstacles seem insurmountable. There are many who are driven by a belief but who do not offend. They achieve things that others would easily have given up on a long time ago. We see their results and wonder whether we could ever be like them.

As I reflect on the subject, I have begun to see myself as lukewarm with respect to many of my beliefs or visions. Sure there have been times when I have put a lot of effort in but it seems that when the going gets a little tough or the demands of the workplace increase, I am inclined to put the vision or belief aside while I just try to stay on top of what seems to need to be done.

Having just taken redundancy so I can focus on completing the writing of my PhD thesis, I am having time to rethink my priorities and to consider what I might attempt to do next. Wouldn’t it be nice to achieve some of my dreams? But what hinders me from their pursuit? Inevitably such factors as the need for money. There is always maintenance required on the house or other chores that seem to rise in priority while the vision sits quietly dying in the background.

Maybe in the end, we don’t have enough belief in what we claim or in our visions. Maybe if we did then we would see more achieved with a lot less effort.