Delft University of Technology Exploring the effectiveness and the transversal competency retention of a game-based learning activity one year after student participation

Most studies in engineering education focus on the effectiveness of the teaching interventions and what transversal competencies engineering students gain from those interventions immediately after they take place. This study explores the effectiveness of an existing game-based learning activity and the competency retention of the participating students one year after its implementation. Semi-structured interviews were conducted to nine students who participated in the activity at Delft University of Technology, The Netherlands. The findings showed that students remembered and benefited from the activity. This exploratory study indicated that this activity is valid to retain students transversal competencies because it provides active learning, engagement, rules, risk-taking, cooperation and competition, and it can be plugged-and-played in any engineering context. This study encourages researchers in the field of engineering education to conduct transversal competency retention studies, which are fundamental to understand how the teaching practices shape students’ competencies in the period after their participation.


Materials and Methods
This study is an extension of the study of  to explore the characteristics that make the Chinese Whispers with a Twist activity effective to retain the transversal competencies of the participating students. The sampling strategy used in this chapter is convenience sampling (Cohen, Manion, and Morrison 2007). The researcher has purposely selected the cohort of engineering students who attended the Chinese Whispers with a Twist activity at Delft University of Technology (TU Delft) in March 2018. There were two reasons for this choice: first, these students belonged to the same university as the first author, which facilitated the interviewing process and second, the interviewing period fitted into the time frame of the doctoral thesis of the first author. As this study is only exploratory and no generalizations will be done, this choice does not hinder the outcomes (Cohen, Manion, and Morrison 2007). This cohort of students will be named cohort TU Delft 2018 from here on. All twenty-one students of the cohort TU Delft 2018 were contacted by email to take part in a semi-structured interview. The use of interviews was selected to explore the characteristics previous identified in the study of  and to gather a deep understanding of what students retain of the game-based learning activity (Creswell 2003;Cohen, Manion, and Morrison 2007). Students were offered a chocolate bar of Tony's Chocolonely for their participation. Without selecting the students, out of the twenty-one students, nine accepted to be interviewed. This sample consisted of five males and four females, of which five were Dutch students and four were international students coming from the United Kingdom, Greece, Spain and India. The interviews were conducted in English and took place between 25 March and 21 April 2019, one year after students participated in the Chinese Whispers with a Twist activity. The period of one year between students' participation and the interviews was chosen for this exploratory study to investigate the validity of this activity to stimulate students' practice and reflection of their communication competencies, although other periods would be interesting to investigate. The interviews lasted approximately one hour. The following questions were asked to students: of coding methods was applied to the data transcribed: previous codes used in the data analysis in Leandro  and descriptive codes that emerged from the data (Saldaña 2016). This coding approach (Saldaña 2016) intended to corroborate and possibly complement the findings of Leandro . The previous and new themes are explained with students' quotes. They are between quotation marks and are associated with each interviewee. All data was analyzed anonymously and students were given a random identification code from S1 to S9.

Results
In this section, the findings that emerged from the interviews conducted one-year after students participated in the Chinese Whispers with a Twist activity are presented. This section starts by showing that all students remembered the activity. This is followed by an analysis of students' communication performance perceived by the students. Finally, the benefits that students reported they gained from the activity are described as well as how students think they best learn communication competencies.

Students' memory about the activity
The first goal of the interviews was to understand whether students remembered the activity and what they remembered. The author showed the image that students were asked to communicate and draw during the Chinese Whispers with a Twist activity to the students and all of them immediately remembered the activity and the role they played in it. Students showed some of their memories as follows: "I remember that someone had to look at this thing and the rest of us didn't know and then we had to draw what someone else's told us to draw and then it had to look like this." (S2) "I think, I had to pass on the information. Yes, like between the first one and then the third person." (S5) "We were a team of four people, four or five people and, I think, it was mostly about transfer of information, of how information is transmitted." (S9)

Students' perceived performance one-year after the activity
Students were also asked about how they performed during the activity. Two categories emerged from the interviews: C1) effective communication competencies and C2) ineffective communication competencies. This division was made because the emerging themes focused on the ineffective and effective communication competencies perceived by students and how that hindered or benefited their performance during the activity. The themes and the students who mentioned them are shown in Table 1. These categories and themes are used to summarize students' perceptions of their communication competencies. The themes of effective and ineffective communication competencies will be presented together because they were reported by students for the same themes.

Conveying/explaining/answering+
In this theme, students referred they described well and used the information they observed or heard to convey the message. However, for S1 effective transmission was not possible right from the start. S1 mentioned that the message was conveyed after some refinements. Students' expressions were: "But the guy was very good in explaining, the guy who explained to us." (S2) "In how far I could bring the message across, it was OK." (S5) "But eventually they refined the way they were communicating so that I could understand what they meant and put it in context and tried to make a figure out of it." (S1) Conveying/explaining/ answering Remembering Effective (+) Ineffective (-)  S1-S9: student codes Conveying/explaining/answering-Almost all the students mentioned this theme. Students realized they could have described the image differently. Some students mentioned they should have given fewer details and more critical information, others said they could have described more accurately. They also realized that describing the image was not an easy task and that they omitted certain aspects that they assumed the receiving party would know implicitly. Students also referred to the difficulties they experienced when answering the questions because either they did not have enough information to respond or they did not understand the information transmitted. Examples under this theme were: "The implicit details that I assumed I didn't get across." (S8) "There were some answers they couldn't give us." (S3) "Because when you're actually describing it you realize it's not as easy." (S7)

Asking questions+
Students in this theme mentioned what questions they asked (including detailed and clarifying questions) and how they asked questions (for instance creatively) to communicate effectively. Examples provided by students were: "We did ask questions. I did ask. They were like clarifying questions rather than more general kind of questions." (S4) "Then we had to kind of creatively ask around to still find an answer." (S3) Asking questions-Under this theme, students mentioned they were not able to ask the right questions or asked insufficient questions. Other students remember they were too impatient to wait for the answers. Examples of these were: "I wanted to say things like they were not asking." (S9) "I and the other person in the last role were both a little impatient waiting for the whole answer." (S3)

Details+
Students in this theme referred specifically to the details they described, asked and draw. Examples of these were: "I think we had most of the things." (S2) "So I was really looking for specific information." (S4) "We distributed the work between basically the large overall part and the details." (S8) Details-Students indicated that they did not describe certain aspects of the image because they assumed they were implicit in the description or they forgot details that were told. Also, S4 expressed the need to describe not all the details but the essential details to convey the overall picture. Examples provided by students were: "Because when you have the picture in your head and you just describe the things that you see. You're not necessarily describing all the details but implicitly in your head they are included in what you say." (S8) "We said three trees. But we didn't get the height right." (S9) "Going back, they probably could have done differently such that in maybe explaining less they would have conveyed the idea a bit better and would have given like the essential details or the essential points of the picture." (S4) Planning/strategy+ Students revealed two strategies they adopted during the activity: the distribution of the work between the team members and the planning in advance of what to ask. Students expressed these as follows: "Then we come up with already some questions that we had and we could ask." (S2) "We were two people describing the thing and, I think, we distributed the work." (S8) Planning/strategy-However, the strategy of planning in advance of what to ask did not work well as stated by S2: "So, we didn't really anticipate something like this. So, all the questions were out of the window."

Listening+
Students in the code "listening+" mentioned that they listen attentively and carefully. Examples in this theme were:

"And then, of course, you have to carefully listen to what they say, filter the information and see: Is this what I need or is this what I want to know?" (S1) "I listen a lot." (S7)
Listening-However, students also found difficulties when listening to each other. They express these as follows: "And there you were trying to say something but they were listening but it was not entering in their minds what you were saying." (S5)

Managing+
Only S2 expressed this theme. She mentioned her group had shown efficient management skills because they were fast and efficient by asking lots of questions, as follows: "Quite efficient. Just because we asked a lot of questions, I guess, about what to do." Managing-Also, only one student referred to this theme. S9 mentioned inefficient management skills in his group including difficulties to stick to the time given and with the amount of information, as follows: "The information was just too much for the person to handle (…) Issues with the time frame."

Taking notes+
In this theme, S6 referred that they were efficient to write down the information they observed from the image. She expresses this as the following: "We wrote what we saw." Noise-This theme was expressed by all students, except S6. Comments in this theme referred to barriers in student's communication such as the difficulties to understand the descriptions and the big picture, the losses of information along the communication chain and the assumptions made. Most of these issues happened because students thought it was easy to convey information and that information was not clearly transmitted. Students realized it was harder than it seemed. Also, in this theme, another communication issue was pointed out by S3. She mentioned that her group experienced language issues due to the different backgrounds of the team members who have English as a second language. Examples in this theme were: "We didn't have an accurate representation at the end because I didn't really have any idea what it was that we were supposed to do. I mean, I didn't actually have the big picture." (S4) "The implicit details that I assumed I didn't get across." (S8) "We also had some language difficulties because we had a Spanish girl and then the others were Dutch but not all of them as fluent in English." (S3)

Remembering+
This theme was only referred to by S5. This student was in role B and showed frustration because she could not tell everything she memorized from the information given by role A.
"But it was frustrating. Because the first person told me a lot of things. So yes, all the 10 things in your mind and you want to tell them all 10 and then the next person is only asking about two so you want to bring the others also."

Benefits of the activity by students
Students were asked in the interviews about their experiences during the activity and the takeaways and important aspects they perceived they gained from it. This was inquired to investigate the benefits of the activity according to students. Three categories emerged from the interviews: C1) Awareness, C2) Experience and C3) No gain. This division was based on comments in which students mentioned they were aware of their communication competencies, explicitly stated they experienced certain communication competencies in the activity or reported that they did not gain anything from the activity. The themes that arose from the analysis were grouped into one of these categories (Table 2) and were used to summarize the benefits of the activity perceived by the students. The findings showed that this activity created awareness and experience of communication competencies for all the students.

C1) Awareness
Have awareness already  Nothing new Have experience already S1-S9: student codes (N): new themes that emerged from the anal ysis of the interviews C1) Awareness: This category concerns the awareness students gained during the communication activity.

Barriers in communication
In this theme, students mentioned the awareness they gained of the difficulties to change people's mind and to get the idea across when not everyone has access to the same information and the right questions are not asked. Also, students gained awareness of the need to be precise and clear and not rush when describing, asking and responding to questions, otherwise, the flow of information is disrupted or information gets lost. Students showed this awareness as follows: "

Details
All the comments that refer to awareness for details were considered in this theme. Students mentioned the details they missed, took for granted, did not ask about and assumed they had said or that the other would think the same way as they did. Another student mentioned that only after participating in the activity students realized the details they needed to convey the message. Examples of these were: "The implicit details in your head." (S8) "We rush and miss details because they were not told yet." (S3) "I think we would get better results because now we'd at least know how precise you have to be and how much details you have to give." (S9) Conveying/explaining/answering Students mentioned that this activity helped them to realize the importance of transmitting information clearly and concisely in a short time to get the message across, avoid misinterpretations and information loss. Students expressed the following: "You realize the importance of transmitting information and things like that." (S1) "You have to be precise when giving information because then it can turn out to be like a completely different thing." (S6) Asking questions In this theme, students referred to the need to ask clear and concise questions to be understood and the right questions to get the necessary information to complete successfully the tasks. Students also mentioned they gained awareness of the difficulties to pass information when the questions were not asked. Students showed these as follows: "You have to ask questions so that you can get it." (S2) "It was also difficult. You want to say there was a tree and someone is asking the wrong questions and then you have to sort of hint that there was a tree without really saying it." Understanding Students mentioned that this activity helped them to understand that information should be carefully passed and make sure the person on the receiving end understands the message. "You really have to be careful about how you formulate it and make sure that the way you formulate it helps them understand exactly what you mean. Which is as this exercise showed, revealed even for simple things." (S1) "You actually have to be really precise when you describe something and ask and make sure they understood the same as you understood." (S6)

Strengths and points for improvement
This theme was only mentioned once by S3. She referred that she is an extrovert and realized that she needs to improve her listening skills because during the activity she asked questions but did not listen attentively. She expressed this as follows: "I have the tendency then to already go ahead and continue on the next question while sometimes it's better to wait for the whole answer and then ask that question." (S3)

Assumptions
Under this theme, students mentioned they assumed certain aspects that were not told and realized the importance of being clear and make sure all the information is conveyed. Examples in this theme were:

Barriers in communication
Students in this theme referred to the difficulties they experienced during the activity in terms of changing people's ideas and putting visual information into words. Examples given by the students were: "

It did make me see that if someone has like a fixed idea or something then it's hard to change it." (S5) "It shows that you have a certain mental picture in your head about the image in this case and that communicating that exact image is very difficult." (S8)
Conveying/explaining/answering In this theme, students mentioned they experienced pitching skills, i.e., they practiced how to convey a message in a short time and what happens when the information is not clearly transmitted. S3 also experienced that this activity allowed introverted students to practice: "Communicating in a shorter way or concise and then still be clear enough that a person can draw the drawing basically that's, I think, something that when looking back I practiced here." (S3) "It's interesting to see that you try to explain something and people interpret what you are saying." (S7) "They were in the beginning kind of uncomfortable having to ask questions or describe something.

(…) For introverts is hard to describe something and ask the questions. That' a good practice." (S3)
Listening Students expressed that this activity was a good practice of listening skills. S3 mentioned that especially for extroverts, this activity is a great exercise to learn to attentively listen to others, take on the information and convey the message. "It was a good exercise in realizing the importance of asking questions and listening attentively." (S1) "For an extrovert, it's the listening that's a good practice." (S3)

Managing/planning/organizing
In this theme, S3 mentioned students experienced pitching information in a short time with this activity and the need to prioritize the information, i.e., go from the general to the particular. She expressed these as follows:

Engagement/fun
The comments of the students in this theme showed they enjoyed the activity a lot. One student was more specific and mentioned that simple tasks as asking questions can be a fun exercise. Also, in this theme, students mentioned they liked the activity beca use it was a hands-on activity. Examples of these were: "Pretty. All of that was just interesting. So no matter if I was good or bad but for me, it was just a learning exercise. So, I enjoyed it." (S9) "So that showed them that having to ask questions or describe something that it's also fun I think that was the good part of the process." (S3) "This one was more dynamic." (S6) Interaction between people In this theme, students mentioned that this activity allowed them to get to know their peers better and engage with the team members, as in a team-building exercise. Students showed evidence of this as follows: "So, like being in a competition. It's like a game let's say that really involves communicating skills." (S6) C3) No gain: This category includes comments where students mentioned that they did not gain any awareness, experience or anything else by taking part in the activity.

Have awareness already
Only one student mentioned this theme. Although S9 expressed that he had awareness of the barriers in the flow of information and the need to be clear, he also mentioned that this was a takeaway from the activity, as follows: "I've always been aware that information has to be very precise or there's always distortion. (…) "This is something which I've seen online as well. Where they show how information is gonna break from one person. So, I think that was the biggest take away here." (S9)

Learning communication competencies according to students
The last question of the interviews intended to investigate what the best way to learn communication competencies is according to students. The themes that arose from the analysis of the interviews (Table 3) were used to complement the previous qualitative data in the search for the characteristics of the effective game-based learning activity.

Hands-on
This theme showed comments of students about the need to practice communication competencies during their studies. Students mentioned that learning of communication competencies happens when they are exposed to situations in which they have to actively communicate and be confronted with their effective and ineffective communication competencies and where they can improve. They referred that this activity is one of those situations. Examples under this theme were: " Comfort environment Only one student mentioned this theme. S4 referred that students need to gain awareness or practice transversal competencies including communication competencies at the university so that they can take risks and learn from the failures. He expressed this as follows: "So, I think, it's nice to learn them at university or in this environment because it's kind of safe and the consequences are low." (S4)

Discussion
This study is an extension of the study of  to explore the effectiveness of the game-based learning activity to retain the transversal competencies students' practiced and reflect on one year after they participated in the Chinese Whispers with a Twist activity.
To do so, the characteristics that make this activity valid were explored with semi-structured interviews carried out with a cohort of engineering students. In this section, these characteristics will be compared to those found in Leandro .
In the study of , the game-based learning activity was proven valid because of six characteristics: active participation, engagement, rules, reflection, risk-taking and cooperation. The research in the current paper not only corroborates the characteristics found in Leandro  but also complements their findings, highlighting two other characteristics that are competition and plug-and-play. The first two characteristics, active participation and engagement, were identified by the participating students under the existing theme "engagement/fun" and the new theme "hands-on". One year after students took part in the activity, they remembered the activity because it was fun and engaging. Different from the study of Leandro  is the new theme hands-on, in which students mentioned that the game-based learning activity is a great exercise to practice and think about their communication competencies. The third characteristic was the rules. One year after students participated in the activity, they still remembered the roles they had and the constraints of the activity, including the limited time to describe the image, listen, ask and answer to questions, and the fact that they could not ask questions for clarification nor give information that was not asked. This was expressed in both the study of Leandro  and the current sunder the themes "planning/strategy+/-" and "managing+/-", in which some students mentioned they dealt and adapted well to the rules, they planned and managed the time but others students experienced difficulties, they failed to plan or did not manage the limited time. An additional finding from the interviews which was not present in the study of Leandro  was concerning the two last rules in which students in role B could not ask questions and could only respond to the questions posed to them. These rules triggered some frustration in one student (S5) from the fact that she could not tell all the information she had received because her peers were asking the wrong questions. In summary, rules are a characteristic of the gamebased activity that allow students to deal with uncertain events that would not normally happen in the real world but that prepare students to be ready to react promptly.
The fourth characteristic, reflection, was shown by students in the interviews under the theme "reflection". Students highlighted the usefulness of having to reflect on their individual and team performance after the activity because they stated that the learning happens not only when they experience something but after reflecting on it. This is supported by Kolb's theory (Kolb 1984) which "learners need to reflect on what they experienced, conceptualize and draw conclusions from their experiences and critically use the knowledge gained from the experiences in future scenarios". The next characteristic was risk-taking. This study showed this with the themes "comfort environment" and "embedding". Students perceived that activities like the Chinese Whispers with a Twist should be embedded in existing courses and connected to the engineering context to become a safe environment. This way students can make links to the engineering world without being afraid to make mistakes and fail. This finding shows an extra characteristic found in this exploratory study that demonstrates the effectiveness of this activity. This characteristic is plug-and-play. As mentioned in Leandro  and proven in the current study by the theme "embedded", this activity was effective because it was designed to be implemented in existing engineering courses where students have a reflection moment at the end of the activity to contextualize the content of communication in teams to the engineering environment of the course.

Conclusions
Unlike most studies in engineering education, this study explores the retention of transversal competencies over a long time rather than focusing only on evaluating the acquisition of transversal competencies immediately after the interventions. This study complements the results presented in the study of  and indicates that students retained the communication competencies gained during the game-based learning activity after one year. This seems to be due to the following characteristics of the activity: active participation, engagement, rules, reflection, cooperation with team members, competition between teams and plug-and-play. This exploratory research provides a platform for further investigation in the field of transversal competency retention in engineering education by showing that it is worthwhile and feasible to investigate the retention of transversal competencies of participating students one year after the intervention conducting interviews. The evaluation of the transversal competencies after a period of time is considered a fundamental requirement to understand how the teaching practices shape students' competencies in the period after their participation and whether students retain the necessary competencies for employment. Thus, although time and resource consuming, it is recommended that researchers and educators in the field of engineering education do not only focus on course evaluations but go beyond that, and follow students in the years after they took part in these courses . The limitations of this study are the limitations of most exploratory studies. The interviewed participants consisted of a limited number of students who participated in the activity in only one university. These students were not randomly selected and it is not possible to exclude that the students who participated in the study were those who had a better recollection of the teaching activity. However, this sample is considered a representative group of the entire original sample joining in the activity (n = 393) because it is a heterogenic population in terms of gender and nationality. The other limitation of this research is related to the interaction of the participants with biological, environmental and interventions that were not investigated and may have influenced the memories of participants' experiences in the activity. Finally, the lack of a control group was a limiting factor in this research, which hindered the investigation of the gain and retention of the transversal competencies of students who did not participate in this activity. It would be of interest to explore whether students who did not attend the activity gained the transversal competencies in other moments of the curriculum and whether they retained those competencies throughout the years. As future work, a phase of quantitative data collection may be carried out to build on the results of the qualitative phase with the cohort of TU Delft 2018 (Creswell 2003). This way, the quantitative data may assist in the interpretation of the qualitative results and the qualitative findings can be generalised to different samples (Creswell 2003). This is called sequential exploratory research and its advantages are that it is easy to implement and straightforward to analyse and report, and expands on the qualitative outcomes.