Month: July 2017

Pedagogy

Yet another great pedagogical course at Uppsala University

Jonas MollLeave a comment

Blåsenhus

During spring 2017 I wrote blog posts about pedagogical courses I took on supervision of oral presentations and methods for activating students as well as a leadership course – all given by Uppsala University. This blog post is about the third pedagogical course I took during the spring term – assessment, grading and feedback. This was a 1.5hp course which included four scheduled days, which mostly focused on group work, and one day devoted to own work with developing a course of one’s own choice.

In the individual work I once again chose to focus on developing the communication course for first year computer science students at KTH – the course I worked with for almost a decade before I started my postdoc in Uppsala. The individual task was to go through everything related to assessment, grading and feedback, starting with refining the intended learning outcomes. I thought everything was already in place regarding the learning outcomes, but when considering the big picture I suddenly realized that one of the most important aspects covered in the grading criteria – the ability to adjust the content of the reports to a particular target audience – was not brought up in the intended learning outcomes. There was also an intended learning outcome about being able to use different text production tools, but the only tool used in the course is Latex. After adding a new learning outcome about target audiences and narrowing down the learning outcome about text production to focus only on Latex there was a much better constructive alignment – clearer connection between assessment tasks, learning activities and intended learning outcomes. Constructive alignment was one of the key principles in the course and I can really see why it’s important.

After considering the intended learning outcomes, the next step in the individual task was to look over the assessment tasks – in my case different versions of the report and a critical review of another student’s report draft. When it came to this part I came to the conclusion that everything should be kept. Almost an entire day of the course was devoted to discussing means of providing feedback and peer feedback and assessment was brought up several times as highly beneficial especially when handing in different versions of e.g. a text while working towards a final deliverable. This was exactly the way we did it in my communication course – the students handed in different versions of reports which were discussed in small groups.

The last part of the individual task was devoted to refining the grading criteria and making sure that they were relevant with regard to the intended learning outcomes. This was probably the part where I learned the most from this course. I started out with qualitative grading criteria, regarding several aspects (e.g. content, structure, language,…), for each grade A-E. In most cases, the only thing that differed between different steps was a single word (e.g. ok, good, very good, excellent,…) and hence I used continuous grading criteria. I learned from the course that continuous grading criteria are often hard to relate to actual achievements and they are also hard to measure. Due to this I tried to change to discrete criteria in as a large extent as possible. Thus, instead of just varying single adjectives I tried to describe what they should actually do for a certain grade (e.g. instead of just writing “very good” I made it more explicit what should be accomplished).

One part of the course which I especially enjoyed was that we, several times during each of the scheduled days, left our groups to have a discussion with our “critical friends”. Before the course I had never heard about this concept. A critical friend was in this case another course participant who worked at the same department (in my case IT). The reason why we should discuss with our critical friends was that we should get the chance to discuss different problems with someone who is teaching within the same subject area. Quite often after a new topic had been introduced by the teachers, we discussed with our critical friend how the topic related to our area and our respective courses. Most of the time these discussions focused on the course we had chosen to focus on (in my case the communication course discussed above) and I definitely got quite a few ideas which I continued to work with in my individual assignment. The concept with critical friends was great and I’m quite sure that it can be used in quite a few courses on different levels. I will definitely try to incorporate the idea in my own courses when relevant!

I really enjoyed this course and once again I learned a lot. I definitely think the communication course is better now, especially when it comes to the grading criteria. I also think that I’m now well equipped for defining course plans and designing courses from scratch. I can definitely recommend this course both to inexperienced and experienced teachers!

 

DOME · eHealth · Medical Records Online

Seminar about the history of patient accessible electronic health records in Sweden

Jonas Moll1 Comment

A few months ago I participated in a very interesting session in the seminar series “Current challenges in biomedical information technology” at Uppsala University. The invited speaker was Benny Eklund, IT Strategist, Uppsala county council. Benny’s work and initiatives – starting about 20 years ago – is probably the main reason why we have patient accessible electronic health records in most county councils in Sweden today (the few ones remaining will allow access before we leave 2017). See link to the filmed presentation below!

Åsa Cajander, who has collaborated with Benny for years introduced him before he began his presentation. As she reveals in the introduction students in one of her courses, “IT in society” played a very important role when they, during one of the course rounds, investigated future possibilities for medical records online in Sweden (a topic provided by Benny). The students’ work resulted in a white paper, which was presented in Brussels! Their work certainly helped in attracting necessary funding.

During the session Benny talked mostly about the early history of patient accessible electronic health records in Sweden, like e.g. important decisions made, the many barriers encountered as well as identified enablers and functions included. After listening to the talk it became more clear to me than before that there were many bumps in the road towards the system in use today. I really recommend you to watch the video – I’m convinced that the barriers and enablers brought up to discussion are relevant in many other countries as well where similar systems have been or are about to be introduced.

Link to presentation

 

 

Academic leadership · Leadership

Interesting seminar about knowledge and academic leadership

Jonas Moll1 Comment

KTH

Yesterday, I attended a very interesting and inspiring seminar where professor Ingemar Ernberg, department of Microbiology, Tumor and Cell Biology at Karolinska Institutet, was the invited guest. The seminar series is called “summer talks” and covers not only science but other fields as well. So far, it has been held in Lövstabruk, a really tiny place in the northernmost, forested part of Uppsala county, Sweden, once a week during the period June-August each year. It’s a local initiative mostly for members of the Cultural Association in Lövstabruk. The theme for all the talks this year is “Academic leadership at the boarder of knowledge” and Ingemar focused especially on what is needed to establish a creative academic environment. As usual, Ingemar got about one hour to present, whereafter we all had a small soup dinner. During the second hour the audience discussed the presented content with Ingemar.

Ingemar began by discussing the three main tasks of the University; conducting research, teaching and sharing knowledge with society. He highlighted several times that research should be the foundation in that the other tasks should be based on it. Research should be the university’s engine so to speak.

He then went on to discuss what is needed, in terms of academic leadership, to make an environment facilitating the creation of new knowledge (which could then be used in e.g. front line education and expanding scientific fields). Of course, creative people are needed but a research environment that enables the researchers to fully make use of this creativity is also very important. A good academic leader builds a research environment where the researchers can blossom, which in turn gives the academic environment creative injections. It is of utmost importance that such an environment is inspirational, enables focusing and that it supports engagement and motivation.

Two words that were especially highlighted by Ingemar were creativity and flow. Creativity is of course a prerequisite when conducting research, especially at the boarder of what is current knowledge (the “adjacent possible”). Flow is a kind of mental state in which you are fully emerged in a task not caring about anything else, or even space and time. E.g. if you are writing an article, and happen to enter flow state, writing can go on for hours, provided you have the inspiration and an environment which is e.g. free from interruptions. Creativity and flow are definitely connected and it is easy to understand why it is of importance to enable both creativity and flow in a research environment.

This is where Ingemar mentioned a problem that many universities struggle with today – they are run more and more like government agencies. There are as a consequence many rules and procedures and an overall increasing administrative burden on the employees. Such circumstances do certainly not support creativity! Related to this Ingemar showed a rather simple scale with “Order” and “Chaos” at opposite ends. He believed that the academic leadership should be in pursuit of an environment somewhere in the middle of the scale. Of course we cannot have an environment where everyone does what they want and no one checks what is going on (total chaos), but we should not have an environment where everything is controlled in detail either. Today’s problem is that many research environments are getting closer and closer to the order extreme on the scale. To enable creativity (e.g. bring in somewhat more “chaos”) we need to move away, at least to some extent, from the control structures and procedures of today.

Ingemar also brought up some specific success factors for academic environments. Some of these were:

  • Openness
  • Encourage critical questions
  • Tolerance for errors
  • Curiosity
  • Diversity
  • Courage to break paradigms

Ingemar did not only talk about academic leadership but also about knowledge in general. He showed a very pedagogic image, about different types of knowledge, which I don’t want to reproduce here due to copyright reasons. Anyway, I will try to explain the key points. Imagine a sphere including everything we currently know – thus, the results of all research conducted so far. This sphere is, of course, constantly increasing in volume at a forever higher pace due to e.g. that we know still more and get still better equipment. Learning from historical discoveries we also know that a small part of this sphere also contains incorrect knowledge. Enclosing the knowledge sphere is another, larger, sphere containing what we may call “the adjacent possible” – this is the new knowledge within reach by building on e.g. our current technologies and what we know today. At present, most researchers seem to work with research mostly within the knowledge sphere – hence not reaching very far into the yet unknown. According to Ingemar we need to build research environments which accept that more creative people conduct research outside this “comfortable” (and rather failsafe) sphere. Working in the unknown (but still reachable) is risky in many respects – most of the work performed here will fail and it is not uncommon that it can take up to a decade before any publishable results are achieved. Nevertheless, taking risks here can lead to very important and possible life-altering discoveries which can greatly advance the research!

The last part of the talk was devoted to leadership styles. I already mentioned the government agency style above. Another common leadership style is the enterprise style. Both of these styles do not really promote a genuinely creative atmosphere. Ingemar would instead like to see a third option which is based more on shaping creativity. How such a leadership style should be formed was discussed with the audience after the soup break but we did not really end up with a definite answer – I guess this calls for some digging into the adjacent possible…

I really enjoyed Ingemar’s presentation – I definitely see why he has now been invited to the seminar series for the fifth time! I really hope he shows up again next year. The basic idea with the seminar series is to let the public meet and discuss with influential people of different fields – thus it is an initiative that enables university researchers to fulfill the task of spreading knowledge. I think it is great that the participants (mostly from outside academia) get the possibility to interact with renowned researchers in this way, discussing important topics!

 

conference · Haptics · Multimodality · sonification

Got a new paper published, on the effects of auditory and haptic feedback on gaze behaviour!

Jonas Moll3 Comments

SMC_published

About a month ago I wrote a blog post about a conference paper with the title “AN EXPLORATORY STUDY ON THE EFFECT OF AUDITORY FEEDBACK ON GAZE BEHAVIOR IN A VIRTUAL THROWING TASK WITH AND WITHOUT HAPTIC FEEDBACK” that had just been accepted for the Sound and Music Computing 2017 conference. Now, that paper has been formally published! You can find our paper here and the full conference proceedings here. The study leader, Emma Frid presented the paper last Thursday (6/7 2017) afternoon in Espoo, Finland. The other authors are Roberto Bresin, Eva-Lotta Sallnäs Pysander and I.

As I wrote in the earlier blog post, this particular paper is based on a small part of an extensive experiment. The experiment, which 20 participants took part in, was based on a simple task – picking up a ball and throwing it into a goal area at the opposite side of a virtual room. After 15 hits the task had been solved. The same task was solved in several different conditions of which some included haptic rendering and some included movement sonification (two different sound models were compared) of the throwing gesture. During all interaction with the interface, different parameters, including gaze data collected through an eye-tracker, were continuously logged. In the part of the experiment on which the published paper is based we wanted to find out if the participants’ visual focus in the interface changed depending on experiment condition (e.g. if participants looked more at the goal when haptic and/or auditory feedback was presented). Due to bad quality of the sampled gaze data for some of the participants (< 80% of the gaze points had been registered), only gaze data from 13 participants could be used in the analysis.

Much due to large inter-subject variability, we did not get any significant results this time around, but some interesting patterns arose. Results e.g. indicated that participants fixated fewer times on the screen when solving the task in visual/audio conditions compared to a visual-only condition and fewer times on the screen when solving the task in the visual/haptic/audio conditions than when doing it in the visual/haptic condition. The differences between haptic conditions were, however, small especially regarding one of the sound models presenting a swishing sonification of the throwing gesture. When considering total fixation duration (for how long the participants focused on the screen) the tendency was that participants focused less on the screen when this sound model was used (indications were stronger when haptic feedback was not provided). Even though these results were not significant they indicate that movement sonification has an effect on gaze behaviour. When looking at gaze behaviour for each participant individually we could also see that the participants could be divided into a few clusters in which the participants showed similar behaviour. Although the large inter-subject variability did not make it possible to find any general patterns, we could find indications of effects of auditory feedback within the clusters. See the article linked above, for a more detailed analysis, illustrations and discussion.

Even though we did not get any significant results, the indications we got that movement sonifications can affect visual focus are still interesting. If it is true that you look more on the screen when you do not have access to movement sonification, this can mean that you can focus on different parts of an interface, maybe solving different tasks in parallel, when having access to movement sonification in this kind of environment. It is definitely worth conducting similar studies with a lot more participants in order to see if the indications we got would become significant. Experiments with more users could also show if participants focus more on the goal when having access to movement sonification and/or haptic feedback – if so, this would indicate that the information provided by haptic and audio feedback, respectively, is enough to understand that you are performing an accurate throwing gesture (you don’t need to look at the ball to confirm it). Results from interviews held at the end of the test sessions already indicate this!

This is the very first paper Eva-Lotta and I have gotten accepted to the Sound and Music Computing conference. Emma and Roberto, however, have gotten papers accepted to that conference numerous times. Check out their Researchgate accounts for their earlier contributions to this conference and so much more related to e.g. sound design.

conference · eHealth · Interact

Paper on critical incidents and eHealth design accepted to Interact 2017!

Jonas Moll2 Comments

Interact_accept

Months ago I wrote a blog post about a workshop at NordiCHI 2016, to which I submitted my first ever research contribution from the patient’s perspective. You can find the workshop position paper here. After that workshop the participants decided that we should continue our discussions and also do research together when possible. The first result of our collaboration, a short paper submitted to Interact 2017, has now been accepted for publication and presentation at the conference (was conditionally accepted about a month ago)!

Christiane Grünloh is the lead author of this paper, and the others are (in order) Jean Hallewell, Bridget Kane, Eunji Lee, Thomas Lind, Jonas Moll, Hanife Rexhepi and Isabella Scandurra. The title of the Interact paper is: “Using Critical Incidents in Workshops to Inform eHealth Design”.

The paper is focused on the workshop and especially on how this kind of workshop, gathering both researchers, practitioners and patients (me, in this case) who all contribute with a critical incident related to eHealth, can be used to generate ideas that can inform future eHealth design. More details about the format can be found in the paper when it’s published and in the blog post which I linked to above. Christiane will present the paper at the conference and it seems like the presentation (as well as most other presentations) will be broadcasted!

Here is the abstract, summarizing the main points:

Demands for technological solutions to address the variety of problems in healthcare have increased. The design of eHealth is challenging due to e.g. the complexity of the domain and the multitude of stakeholders involved. We describe a workshop method based on Critical Incidents that can be used to reflect on, and critically analyze, different experiences and practices in healthcare. We propose the workshop format, which was used during a conference and found very helpful by the participants to identify possible implications for eHealth design, that can be applied in future projects. This new format shows promise to evaluate eHealth designs, to learn from patients’ real stories and case studies through retrospective meta-analyses, and to inform design through joint reflection of understandings about users’ needs and issues for designers.

 

 

eHealth · Summer school

EHealth summer school in Dublin, day 5

Jonas Moll8 Comments

Today was the last day of the eHealth summer school in Dublin. The lectures, that were of high quality, were fewer today since the whole afternoon was devoted to the projects. This day also contained a talk about the second summer school week in Stockholm. 

My blog posts about the other days:

The three lectures focused on a depression intervention case study, implementation science and ethical design in eHealth. The case study concerned a depression intervention, gNAT’s Island, which started as a research project and is now used by over 1200 mental health professionals! Thus, this is clearly a case of implementation done right. The application, targeting children 10-16 years, comes in numerous variants and the core application is a game focusing on Cognitive Behavioural Therapy. Needless to say, good collaboration with all stakeholders involved was a prerequisite for the success. It must be very unusual that health professionals use a game as a part of the therapy! 

The implementation science lecture focused on real examples of where implementation had gone wrong – sadly enough quite a few can be found within healthcare. Poor design can have fatal consequences within this area! As many as 70% of systems in healthcare fail, often because of poor implementation. Many things are needed for successful implementation. It’s not enough that you have run a successful pilot trial – you also need to be able to scale up the solution, provide necessary training for care professionals (and maybe also patients), have continous evaluation (not just develop something and leave), etc. 

In the talk about ethical design a rather unusual interactive component was used – the participants should fill in a 4×4 “Ethical Bingo” during the talk! As soon as a word on the Bingo card was mentioned by the speaker, Marguerite Barry, it should be marked. There were no prices, but it was an interesting way to keep the audience alert. Among other things, the talk showed how the view of ethics in design has evolved through the years and how to apply wisdom in design. The presented view on ethical design was very interesting, since most discussions about ethics focus on privacy, data protection and similar. 

The entite afternoon was devoted to the projects. Directly after lunch we got 1,5 hours to prepare a 8 minutes presentation which should contain the elements covered during the project sessions the earlier days:

  • A user scenario introducing the intervention and giving it a clear context
  • The user-centered design process and methods
  • The trial protocol proposed for evaluating the intervention

After a coffee break every project was presented with minimal gaps in between (just pauses for switching computers) and then the jury chose a winning team. Before the jury presented their decision the patient representatives got a chance to talk and they were all very pleased with the results and that they had gotten the opportunity to be a part of the summer school. Apparantly, this is the first time patients have been invited to this kind of summer school. 

This last paragraph about (this week of) the summer school relates to the blog post picture. Jan Gulliksen and Åsa Cajander presented, as organizers for the Stockholm week, what would happen during the second summer school week in August. This time we were asked to prepare 6 slides for a Pecha Kucha during the first day in Stockholm! Parts of that day will focus on the participants getting to know each other a little better. The second day we will be in Uppsala and one of my tasks will be to make sure that everyone gets there in the morning and home again in the evening! Medical practitioners will be among the speakers this day. I was really happy to see that we will have a full day workshop in the very cool visualization studio! I have actually never performed any research tasks in that room despoter the fact that I worked at KTH for years. It will be interesting to find out what will happen during the last two days – the participants was asked to write down ideas for possible themes on post-its and hand them in just before lunch. The week in Dublin has been great and I’m sure we will have a super week in Stockholm/Uppsala as well!