WEBVTT 00:00:03.730 --> 00:00:11.140 I am Christian Stecker, I am head of the research area "Comparative Politics and German Government" at the TU Darmstadt. 00:00:11.140 --> 00:00:19.360 This is based at the Institute of Political Science, and in the context of digital examinations, mainly my lectures in the Bachelor's 00:00:19.360 --> 00:00:27.220 play a role and these lectures "Introduction to the Government System of the FRG" are particularly about 00:00:27.220 --> 00:00:31.720 basic knowledge. So, for example, how is the Federal Chancellor elected? 00:00:31.720 --> 00:00:39.350 But it's also about testing certain transfer skills and seeing how far students can apply concepts they've learned. 00:00:39.350 --> 00:00:45.430 For example: How could you change the electoral system in Germany if you wanted fewer parties in the Federal Parliament? 00:00:45.430 --> 00:00:56.140 That would be such a classic question. "How was the exam structured before Corona?" Before Corona, I think I did it like most of my colleagues. 00:00:56.140 --> 00:00:58.930 I wrote a classical paper exam. 00:00:58.930 --> 00:01:07.930 That is, the students got a 7-page exam, mostly where different questions were listed, 00:01:07.930 --> 00:01:16.570 with a certain multiple choice part and with a part where you had to fill in certain answers and solutions in diagrams and figures. 00:01:16.570 --> 00:01:27.500 And there was a proportion of open questions, where you could deal with a certain topic in detail in a limited space. 00:01:27.500 --> 00:01:31.370 "What were the key challenges in the digital transition?" I'll start with the administration of the exam. 00:01:31.370 --> 00:01:33.650 After all, this is where you have to do everything in the digital room, 00:01:33.650 --> 00:01:43.070 what you have done in direct exchange and presented in the lecture or examination room. 00:01:43.070 --> 00:01:46.670 That starts with the registration for the exam, what then, 00:01:46.670 --> 00:01:52.940 because it is simply still a pilot phase, is associated with a much greater need for clarification with the students. 00:01:52.940 --> 00:01:56.970 Then there are questions about what happens if the camera on the computer at home is broken. 00:01:56.970 --> 00:02:02.750 Whether one can then also participate on a mobile phone or what happens if the examination is interrupted for technical reasons? 00:02:02.750 --> 00:02:06.260 A concern that is also new in the digital space. 00:02:06.260 --> 00:02:13.340 That doesn't happen to you in the present exam room. You have to organise entry controls in the digital space. 00:02:13.340 --> 00:02:19.940 You have to organise the identity check and finally also everyday things like going to the toilet, that you have to administer. 00:02:19.940 --> 00:02:26.630 You simply need procedures, pre-planned procedures, which are then used in the digital space. 00:02:26.630 --> 00:02:34.190 This changeover is connected with all kinds of legal questions, which lecturers and examiners don't like to deal with, 00:02:34.190 --> 00:02:37.640 because you already have to build the exam and check the exam. 00:02:37.640 --> 00:02:44.270 But then we also had support from the relevant departments at the TU and the HDA. 00:02:44.270 --> 00:02:53.270 Important questions here are simply: How do you guarantee data protection when you often look into students' bedrooms 00:02:53.270 --> 00:03:01.340 if you have to check their identity without being able to look directly into their identity card? 00:03:01.340 --> 00:03:07.250 To the second aspect: the technical implementation of the written exam on Moodle test. 00:03:07.250 --> 00:03:10.700 This is a challenge, but really pays off when it is accomplished. 00:03:10.700 --> 00:03:17.360 You simply have to transfer the questions that you were asked in the paper exam 00:03:17.360 --> 00:03:21.080 to the Moodle test. There was also very good support from the HDA and I would say, 00:03:21.080 --> 00:03:25.160 it has also extended the range of possible questions. You then ultimately have to 00:03:25.160 --> 00:03:30.800 transfer the multiple choice questions into Moodle's multiple choice format. However, Moodle 00:03:30.800 --> 00:03:38.600 also offers interesting question types such as cloze texts, where students can be invited to give short answers, 00:03:38.600 --> 00:03:46.940 which the computer also corrects automatically. You can programme questions where diagrams can be edited and texted directly. 00:03:46.940 --> 00:03:50.630 Of course, you first have to familiarise yourself with the basic functions of Moodle. 00:03:50.630 --> 00:03:54.440 It's not just putting together a Word document and typing in the questions, 00:03:54.440 --> 00:04:00.710 but you also have to familiarise yourself a bit with the syntax if you want to ask more demanding cloze questions. 00:04:00.710 --> 00:04:06.170 But once you have that, you also have a really good exam and it's very attractive in terms of 00:04:06.170 --> 00:04:15.120 the range of competences tested as well as in terms of correctability. "How was the exam scenario structured?" 00:04:15.120 --> 00:04:21.840 How does such an examination proceed from the examinee's point of view, a remote supervision examination? 00:04:21.840 --> 00:04:26.760 Well, it starts quite classically with the registration via TuCan. 00:04:26.760 --> 00:04:29.010 But then it gets a bit more complicated, 00:04:29.010 --> 00:04:35.220 because we also have to transfer the registrations into the examination Moodle, where the examination is administered. 00:04:35.220 --> 00:04:40.440 As a student, you also have to sign various consent declarations digitally, 00:04:40.440 --> 00:04:47.460 so that you agree in principle to this change from face-to-face exams to distance exams. 00:04:47.460 --> 00:04:52.380 And ultimately you have to create the technical conditions on your own side so that you can take part in this examination. 00:04:52.380 --> 00:04:56.040 And then the procedure from the students' point of view is like this: 00:04:56.040 --> 00:05:04.410 that they appear at Zoom at an agreed time in the digital room and then in front of a fairly large team, i.e. also at the beginning 00:05:04.410 --> 00:05:08.670 they are looked after and guided through the process. 00:05:08.670 --> 00:05:15.810 This means identity control, assignment to breakout rooms and opening and closing of the exam. 00:05:15.810 --> 00:05:20.580 I will go into this in more detail when we talk about the examination procedure. 00:05:20.580 --> 00:05:27.900 You also have to establish an additional communication channel between the supervisory staff. 00:05:27.900 --> 00:05:32.280 In a presence examination, you can go up to your colleague and whisper in his ear. 00:05:32.280 --> 00:05:35.670 I have a suspicion that this one is cheating on the left. 00:05:35.670 --> 00:05:41.160 Please have a look too. If they would speak in a digital room in Zoom, then everybody would hear that. 00:05:41.160 --> 00:05:44.580 If they write it in the chat and the wrong channel, then everybody reads it. 00:05:44.580 --> 00:05:51.430 And in the end, it has also been important to have such a support infrastructure. 00:05:51.430 --> 00:05:57.510 That is, if someone has technical problems, he needs a contact person to turn to, 00:05:57.510 --> 00:06:04.930 who then perhaps also clarifies directly with him on the screen how this question and the problem can be solved. 00:06:04.930 --> 00:06:11.160 And that is the technical structure, which is really much more demanding than in a face-to-face examination. 00:06:11.160 --> 00:06:17.550 It will be interesting to see how this can be further simplified in the future or to what extent this can be done by 00:06:17.550 --> 00:06:21.450 the central office or to what extent chairs are then also more competent, 00:06:21.450 --> 00:06:35.670 to administer this for themselves. "How are the digital exams graded?" Grading a digital exam or a Moodle test just has many advantages. 00:06:35.670 --> 00:06:43.310 So, if you compare that with the face-to-face exam. There you collect these exams, then you carry a few kilos of paper to the office, 00:06:43.310 --> 00:06:48.320 sit down and have to count crosses for hours and days 00:06:48.320 --> 00:06:55.850 uand decipher moderately readable handwriting. When the Moodle test is handed in 00:06:55.850 --> 00:07:02.120 the computer corrects the multiple choice part and also the cloze texts at the same moment. 00:07:02.120 --> 00:07:09.140 So you just get a big time gain, which you can invest elsewhere, 00:07:09.140 --> 00:07:17.750 be it in the careful control of the open questions or in being able to concentrate on another counselling service. 00:07:17.750 --> 00:07:23.420 So I would say, for example, exams have to be efficient, 00:07:23.420 --> 00:07:30.710 because we as lecturers should simply invest more time in good homework guidance of students, 00:07:30.710 --> 00:07:37.070 because that is simply the more important kind of competence that one acquires in the studies or let's say such a competence, 00:07:37.070 --> 00:07:45.620 which simply has to be accompanied better than the recall of basic knowledge. 00:07:45.620 --> 00:07:51.080 In a nutshell, I really want to keep the digital Moodle exam. 00:07:51.080 --> 00:07:59.910 But I also want to have the face-to-face exam back. So a face to face Moodle exam is my dream. 00:07:59.910 --> 00:08:07.700 A big computer pool at the TU, where I can test my students with my staff, in presence. 00:08:07.700 --> 00:08:14.210 MTo be able to interact with them directly and at the same time also strictly and go with a wary eye through 00:08:14.210 --> 00:08:24.460 the rows and ensure without privacy concerns that these are just fair examination conditions for everybody. "In your opinion, where is the potential for electronically supported testing?" 00:08:24.460 --> 00:08:28.720 So a very big potential, which was related to the current situation, was, 00:08:28.720 --> 00:08:37.500 that a remote supervisory examination has actually enabled us to ensure an examination requirement and to ensure health protection. 00:08:37.500 --> 00:08:45.770 So I also changed it at relatively short notice at that time, because I simply had problems with the idea at the beginning of the wave 00:08:45.770 --> 00:08:50.340 to direct 200 students in public transport into closed rooms, and 00:08:50.340 --> 00:08:54.210 that can have really insane implications. If there is a positive case, 00:08:54.210 --> 00:08:59.090 200 people sit in quarantine for 14 days and they are just also worried, 00:08:59.090 --> 00:09:04.970 that they might have been infected somehow. So just being able to make a small contribution to 00:09:04.970 --> 00:09:10.640 make sure that it doesn't spread further and that students can still be validly examined. 00:09:10.640 --> 00:09:15.080 That was simply a potential that we had with these electronic examinations. 00:09:15.080 --> 00:09:20.300 And I also found that this was possible at the TU at such short notice. 00:09:20.300 --> 00:09:27.440 Now the pandemic will hopefully be over at some point and even then there will be further potential for electronic examinations. 00:09:27.440 --> 00:09:30.440 I could imagine that all in all there will be more flexibility, 00:09:30.440 --> 00:09:35.690 also with appointments and perhaps also with the question of where students take examinations. 00:09:35.690 --> 00:09:40.550 I can imagine if you have a very large pool of questions, 00:09:40.550 --> 00:09:53.780 which can also circulate, it is in principle possible that if someone is in a semester abroad, 00:09:45.920 --> 00:09:53.780 but still has to write this exam, that he logs on to a computer and then questions are generated for him at random 00:09:53.780 --> 00:10:01.050 which he then has to answer in a limited time. "What was the feedback from the students?" We received mixed feedback from the students. 00:10:01.050 --> 00:10:07.770 Purely context specific, there was a lot of understanding especially with the short term change, 00:10:07.770 --> 00:10:10.540 that we did this, and it was also supported by everyone. 00:10:10.540 --> 00:10:15.960 So in my case, in the specific situation at that time, I announced the deadline two weeks before the exam: 00:10:15.960 --> 00:10:23.710 There is no way around a presence exam because, to be honest, I couldn't imagine how it would work. 00:10:23.710 --> 00:10:27.420 That was a very quick learning process for me, which was also triggered. 00:10:27.420 --> 00:10:31.740 With the HDA and also with colleagues who also wanted to try it out, I was also happy, 00:10:31.740 --> 00:10:36.420 wthat I did this. The students were simply grateful for this examination opportunity, 00:10:36.420 --> 00:10:48.450 without the worry of infecting themselves and then also infecting risk patients in their environment. 00:10:48.450 --> 00:10:54.870 But there is also criticism. What the students miss is really the exam atmosphere 00:10:54.870 --> 00:11:03.270 and we have been communicatedthat it is really difficult to get into examination mode in their own youth room. 00:11:03.270 --> 00:11:10.010 I also found it interesting that some of them really dressed like they did during the examination. 00:11:10.010 --> 00:11:12.060 Just to construct a little bit for themselves: 00:11:12.060 --> 00:11:17.460 This is now a test, this is important and I have to construct this somehow for myself mentally, 00:11:17.460 --> 00:11:19.860 even if I am not in the examination room now. 00:11:19.860 --> 00:11:26.280 The students have to deal with additional worries, especially concerning the technical implementation. 00:11:26.280 --> 00:11:29.970 I find the comparison quite amusing. Let's imagine we are writing a presence paper exam. 00:11:29.970 --> 00:11:33.340 Suddenly the pen doesn't write any more 00:11:33.340 --> 00:11:38.640 or the paper on which the questions were just written suddenly turns white again. 00:11:38.640 --> 00:11:41.400 And that is ultimately the analogy of what can happen to you in principle, 00:11:41.400 --> 00:11:47.520 if the Internet connection goes down, if maybe something is wrong with Moodle. 00:11:47.520 --> 00:11:52.440 And of course you go into this test with a certain amount of concern as to whether your own Internet is sufficient, 00:11:52.440 --> 00:11:58.210 because maybe you are writing in a rural area or you have to share the bandwidth with family members. 00:11:58.210 --> 00:12:04.500 Because it's just additional worries that students have that they don't have when they write a presence exam. 00:12:04.500 --> 00:12:11.110 And there was just also feedback that this is an additional burden. "What are the lessons learned?" 00:12:11.110 --> 00:12:17.470 I would start with a slightly more abstract point that I've noticed so with myself and also with colleagues, 00:12:17.470 --> 00:12:24.430 that there was a pleasing agility from all places and I must now compliment myself on this. 00:12:24.430 --> 00:12:30.760 So we were all very attached to the paper exam and could not imagine anything else. 00:12:30.760 --> 00:12:36.370 And once that was pushed, we were very quickly with support in a position to 00:12:36.370 --> 00:12:43.000 very quickly inbring the examination into a completely different format and transfer it into a digital space. 00:12:43.000 --> 00:12:50.050 And I just thought that was a nice experience, that you found out that you were actually already a bit stuck, 00:12:50.050 --> 00:12:54.670 but then you can break away from it and then just be solution-oriented in the sense of the 00:12:54.670 --> 00:13:02.650 health protection and the examination requirements. 00:13:02.650 --> 00:13:10.420 Lessons Learned concretely refers to the amazing, varied, diverse possibilities, 00:13:10.420 --> 00:13:16.480 to test wider competence space with Moodle and to simply acquire the skills here as well. 00:13:16.480 --> 00:13:22.120 To programme some clever tasks that are perhaps also a bit interactive. 00:13:22.120 --> 00:13:33.090 For me, I would say, these are the two most important lessons, abstract and concrete, that I take away from it. 00:13:33.090 --> 00:13:40.480 "Tips and tricks" Looking also at the support that is available from the HDA, the changeover is well manageable and you have investment costs. 00:13:40.480 --> 00:13:44.500 You have a bit more to do at the beginning and you have to deal with new things. 00:13:44.500 --> 00:13:54.860 ABut once that is in place, it can really be the basis for valid and efficient examination, especially if you have quite a lot of students. 00:13:54.860 --> 00:13:58.650 I wouldn't recommend it so much for smaller courses. 00:13:58.650 --> 00:14:09.450 But if I really have to examine 200, 250 or even more students efficiently, this is a very good possibility. 00:14:09.450 --> 00:14:14.820 Which I wouldn't necessarily advertise now that we should all stick to these remote supervision exams. 00:14:14.820 --> 00:14:19.770 If that is no longer necessary pandemically at all, then I would really advertise that we should all commit to it, 00:14:19.770 --> 00:14:27.540 that the TU provides us with a large computer pool, where we can then administer it. 00:14:27.540 --> 00:14:33.620 And yes, otherwise tips and tricks: 00:14:33.620 --> 00:14:41.390 I would say that it is important to appreciate again that we demand a lot from the students, 00:14:41.390 --> 00:14:45.800 and also from us a lot is demanded, because we have costs for changes and have to get involved in it. 00:14:45.800 --> 00:14:50.660 Suddenly completely different institutions are talking to us about the administration. 00:14:50.660 --> 00:14:54.980 Before, you did it a bit more autonomously in the lecture hall, you knew the rules. 00:14:54.980 --> 00:15:00.230 And now there are very precise rules about what you are allowed to do in this zoom room and what you are not allowed to do. 00:15:00.230 --> 00:15:08.690 Whether the micros have to be employed or not. In the past, you had a little more autonomy in the presence examination. 00:15:08.690 --> 00:15:14.330 But you get a lot of freedom through digital exams,, 00:15:14.330 --> 00:15:31.050 which you can then focus on more interesting and important support services, especially in terms of time.