WEBVTT 00:00:03.730 --> 00:00:11.140 I am Christian Stecker, I am head of the research area "Comparative Politics and German Government" at the TU Darmstadt. 00:00:11.140 --> 00:00:19.360 This is based at the Institute of Political Science, and in the context of digital examinations, mainly my lectures in the Bachelor's 00:00:19.360 --> 00:00:27.220 play a role and these lectures "Introduction to the Government System of the FRG" are particularly about 00:00:27.220 --> 00:00:31.720 basic knowledge. So, for example, how is the Federal Chancellor elected? 00:00:31.720 --> 00:00:39.350 But it's also about testing certain transfer skills and seeing how far students can apply concepts they've learned. 00:00:39.350 --> 00:00:45.430 For example: How could you change the electoral system in Germany if you wanted fewer parties in the Federal Parliament? 00:00:45.430 --> 00:00:56.140 That would be such a classic question. "How was the exam structured before Corona?" Before Corona, I think I did it like most of my colleagues. 00:00:56.140 --> 00:00:58.930 I wrote a classical paper exam. 00:00:58.930 --> 00:01:07.930 That is, the students got a 7-page exam, mostly where different questions were listed, 00:01:07.930 --> 00:01:16.570 with a certain multiple choice part and with a part where you had to fill in certain answers and solutions in diagrams and figures. 00:01:16.570 --> 00:01:25.390 And there was a proportion of open questions, where you could deal with a certain topic in detail in a limited space. 00:01:25.390 --> 00:01:27.730 And when the exam was collected, 00:01:27.730 --> 00:01:35.110 you just withdrew with different staff members for two to three days, tried to count, 00:01:35.110 --> 00:01:41.500 where the crosses were put in the right place and in the wrong place and tried to decipher the students' handwriting, 00:01:41.500 --> 00:01:52.580 to assess the open questions. "What were the key challenges in the digital transition?" So in the transition to digital examinations, there are ultimately two main challenges that 00:01:52.580 --> 00:01:57.230 had to be mastered. One concerns the administration of the examination in the digital space, 00:01:57.230 --> 00:02:00.740 replicating everything that is usually done in the political space. 00:02:00.740 --> 00:02:08.180 And the second aspect concerns the technical conversion of the paper exam to a Moodle test. 00:02:08.180 --> 00:02:14.330 I'll start with the administration of the exam. After all, this is where you have to do everything in the digital room, 00:02:14.330 --> 00:02:23.750 what you have done in direct exchange and presented in the lecture or examination room. 00:02:23.750 --> 00:02:27.350 That starts with the registration for the exam, what then, 00:02:27.350 --> 00:02:33.620 because it is simply still a pilot phase, is associated with a much greater need for clarification with the students. 00:02:33.620 --> 00:02:37.650 Then there are questions about what happens if the camera on the computer at home is broken. 00:02:37.650 --> 00:02:43.430 Whether one can then also participate on a mobile phone or what happens if the examination is interrupted for technical reasons? 00:02:43.430 --> 00:02:46.940 A concern that is also new in the digital space. 00:02:46.940 --> 00:02:54.020 That doesn't happen to you in the present exam room. You have to organise entry controls in the digital space. 00:02:54.020 --> 00:03:00.620 You have to organise the identity check and finally also everyday things like going to the toilet, that you have to administer. 00:03:00.620 --> 00:03:07.310 You simply need procedures, pre-planned procedures, which are then used in the digital space. 00:03:07.310 --> 00:03:14.870 This changeover is connected with all kinds of legal questions, which lecturers and examiners don't like to deal with, 00:03:14.870 --> 00:03:18.320 because you already have to build the exam and check the exam. 00:03:18.320 --> 00:03:24.950 But then we also had support from the relevant departments at the TU and the HDA. 00:03:24.950 --> 00:03:33.950 Important questions here are simply: How do you guarantee data protection when you often look into students' bedrooms 00:03:33.950 --> 00:03:41.090 if you have to check their identity without being able to look directly into their identity card? 00:03:41.090 --> 00:03:43.850 Of course, the fairness of the examination is a very important point. 00:03:43.850 --> 00:03:50.180 So how is it guaranteed in the digital space that everyone has equal examination conditions. 00:03:50.180 --> 00:03:54.890 One possible solution, which I have not chosen, would be to switch the whole thing to Open Book, 00:03:54.890 --> 00:03:59.180 so that students can use resources, can use slides. 00:03:59.180 --> 00:04:04.560 But there it becomes a bigger challenge that in the end you have a bigger correction load. 00:04:04.560 --> 00:04:10.790 So when I let students write essays, it is not a problem when they look at their transcripts. 00:04:10.790 --> 00:04:13.730 But then I sit for several weeks to read these essays. 00:04:13.730 --> 00:04:20.420 And if I use simpler multiple choice questions in the end I also have to guarantee, 00:04:20.420 --> 00:04:25.100 that you can't just copy the slides for some questions. 00:04:25.100 --> 00:04:31.340 That's why I decided to take a distance examination, where aids are excluded. 00:04:31.340 --> 00:04:38.240 But then of course I also have to prevent attempts to cheat and you can imagine that very well. 00:04:38.240 --> 00:04:43.140 The students sit at their home computers and ,without imputation, 00:04:43.140 --> 00:04:49.070 the temptation and opportunity is clearly there to have a look, 00:04:49.070 --> 00:04:56.180 whether the term was really used in the lecture. 00:04:56.180 --> 00:05:03.120 Of course, the supervisors also sit at their home computers and just see a picture of the student, ultimately they don't see, 00:05:03.120 --> 00:05:09.110 what is happening below the camera above and to the right and left and also not what is happening on the screen. 00:05:09.110 --> 00:05:15.170 There are technical solutions that are more sophisticated now, e.g. a Safe Exam Browser, 00:05:15.170 --> 00:05:22.820 which the students can install, which guarantees that nothing else can run on this computer than this test. 00:05:22.820 --> 00:05:27.300 So that, for example, they can't maybe talk to each other via Discord. 00:05:27.300 --> 00:05:33.410 But at the end of the day, you also have to admit quite honestly that supervision problem in the digital space is a very big one. 00:05:33.410 --> 00:05:39.200 And it is then also so with things only good to solve, which you actually don't want to use that way, 00:05:39.200 --> 00:05:44.930 because they interfere very much with the privacy of the students, for example Proctorio. 00:05:44.930 --> 00:05:49.430 And that, in my view, is also one of the great challenges of this remote supervision. 00:05:49.430 --> 00:05:52.640 We will come back to that later. 00:05:52.640 --> 00:06:00.710 That's also one of the things that I didn't want to continue, but rather to make the supervisory attendance. 00:06:00.710 --> 00:06:06.620 To the second aspect: the technical implementation of the written exam on Moodle test. 00:06:06.620 --> 00:06:10.070 This is a challenge, but really pays off when it is accomplished. 00:06:10.070 --> 00:06:16.730 You simply have to transfer the questions that you were asked in the paper exam 00:06:16.730 --> 00:06:20.420 to the Moodle test. There was also very good support from the HDA and I would say, 00:06:20.420 --> 00:06:24.500 it has also extended the range of possible questions. You then ultimately have to 00:06:24.500 --> 00:06:30.140 transfer the multiple choice questions into Moodle's multiple choice format. However, Moodle 00:06:30.140 --> 00:06:37.970 also offers interesting question types such as cloze texts, where students can be invited to give short answers, 00:06:37.970 --> 00:06:46.390 which the computer also corrects automatically. You can programme questions where diagrams can be edited and texted directly. 00:06:46.390 --> 00:06:50.020 Of course, you first have to familiarise yourself with the basic functions of Moodle. 00:06:50.020 --> 00:06:53.800 It's not just putting together a Word document and typing in the questions, 00:06:53.800 --> 00:07:00.070 but you also have to familiarise yourself a bit with the syntax if you want to ask more demanding cloze questions. 00:07:00.070 --> 00:07:05.530 But once you have that, you also have a really good exam and it's very attractive in terms of 00:07:05.530 --> 00:07:14.480 the range of competences tested as well as in terms of correctability. "How was the exam scenario structured?" 00:07:14.480 --> 00:07:21.200 How does such an examination proceed from the examinee's point of view, a remote supervision examination? 00:07:21.200 --> 00:07:26.120 Well, it starts quite classically with the registration via TuCan. 00:07:26.120 --> 00:07:28.370 But then it gets a bit more complicated, 00:07:28.370 --> 00:07:34.580 because we also have to transfer the registrations into the examination Moodle, where the examination is administered. 00:07:34.580 --> 00:07:39.800 As a student, you also have to sign various consent declarations digitally, 00:07:39.800 --> 00:07:46.820 so that you agree in principle to this change from face-to-face exams to distance exams. 00:07:46.820 --> 00:07:54.210 And ultimately you have to create the technical conditions on your own side so that you can take part in this examination. 00:07:54.210 --> 00:07:59.150 So you have to install the current Zoom licence. You might also have to organise 00:07:59.150 --> 00:08:03.020 that the bandwidth of the Internet is as wide as possible at the time of the test 00:08:03.020 --> 00:08:07.160 for example maybe other family members are not watching Youtube videos at that time. 00:08:07.160 --> 00:08:11.060 If colleagues wanted to do that, as I said, they didn't, 00:08:11.060 --> 00:08:20.300 then you also have to install the Safe Exam Browser and maybe test it or set up Proctorio. 00:08:20.300 --> 00:08:24.530 So you really have to give credit to the students for that. They did that very well. 00:08:24.530 --> 00:08:30.770 We expect much more from the students in a distance examination than in the classroom examination, 00:08:30.770 --> 00:08:37.760 where they just come into the lecture room and everything else is there. So there have to be conditions created by the students. 00:08:37.760 --> 00:08:41.450 And then the procedure from the students' point of view is like this: 00:08:41.450 --> 00:08:51.290 that they appear at Zoom at an agreed time in the digital room and then in front of a fairly large team, i.e. also at the beginning 00:08:51.290 --> 00:08:54.050 they are looked after and guided through the process. 00:08:54.050 --> 00:09:01.220 This means identity control, assignment to breakout rooms and opening and closing of the exam. 00:09:01.220 --> 00:09:07.230 I will go into this in more detail when we talk about the examination procedure. 00:09:07.230 --> 00:09:13.530 Perhaps a few words about the technical structure. You also have to say that in principle structurally. 00:09:13.530 --> 00:09:19.290 Individual work areas can certainly ensure a lot of this themselves at some point. 00:09:19.290 --> 00:09:24.330 But especially in this initial phase we were also dependent on the support of the HDA, 00:09:24.330 --> 00:09:28.960 which, thankfully, we also received in many ways. 00:09:28.960 --> 00:09:35.460 So the HDA designs the Zoom environment, the meeting really has to be thought through in detail. 00:09:35.460 --> 00:09:41.160 Who are the supervisors? Who has the rights to move students? 00:09:41.160 --> 00:09:50.700 What are communication channels like? You also have to establish an additional communication channel between the supervisory staff. 00:09:50.700 --> 00:09:55.110 In a presence examination, you can go up to your colleague and whisper in his ear. 00:09:55.110 --> 00:09:58.470 I have a suspicion that this one is cheating on the left. 00:09:58.470 --> 00:10:03.960 Please have a look too. If they would speak in a digital room in Zoom, then everybody would hear that. 00:10:03.960 --> 00:10:12.060 If they write it in the chat and the wrong channel, then everybody reads it. That's why we then set up parallel communication channels. 00:10:12.060 --> 00:10:17.280 Via teams, for example, and Discord, which perhaps the students also use more often for gambling, 00:10:17.280 --> 00:10:24.220 we then used, so to speak, to supervise the students in the end or to coordinate among ourselves. 00:10:24.220 --> 00:10:32.190 In the lecture hall, a student comes up with a question. One goes over, 00:10:32.190 --> 00:10:35.970 answers the question. If in some breakout room where I am not at all, 00:10:35.970 --> 00:10:39.780 this question comes up, I first have to be informed that there is this question. 00:10:39.780 --> 00:10:47.010 And accordingly, we need communication channels that are confidential for us, but at the same time do not disturb the students. 00:10:47.010 --> 00:10:52.140 And then the HDA also set this up and maintained it. 00:10:52.140 --> 00:10:58.990 And in the end, it has also been important to have such a support infrastructure. 00:10:58.990 --> 00:11:05.070 That is, if someone has technical problems, he needs a contact person to turn to, 00:11:05.070 --> 00:11:12.490 who then perhaps also clarifies directly with him on the screen how this question and the problem can be solved. 00:11:12.490 --> 00:11:18.720 And that is the technical structure, which is really much more demanding than in a face-to-face examination. 00:11:18.720 --> 00:11:25.110 It will be interesting to see how this can be further simplified in the future or to what extent this can be done by 00:11:25.110 --> 00:11:29.010 the central office or to what extent chairs are then also more competent, 00:11:29.010 --> 00:11:40.340 to administer this for themselves. Let us move on to the procedure of the examination. 00:11:40.340 --> 00:11:44.510 A little bit stronger also from the point of view of the examiner and the supervisor. 00:11:44.510 --> 00:11:55.280 So, in the end, we were even the first to do this at the TU at the beginning of the second wave. 00:11:55.280 --> 00:12:00.360 And there were still a few problems that could not be avoided. 00:12:00.360 --> 00:12:08.450 So we ordered students three quarters of an hour before the exam start in Zoom, because we still had the bottleneck problem at that time. 00:12:08.450 --> 00:12:15.820 For data protection reasons, we had to pass students through breakout rooms one by one and check their identity cards. 00:12:15.820 --> 00:12:22.320 This prevented other students from seeing the identity documents. 00:12:22.320 --> 00:12:27.440 And that was just a huge bottleneck that cost a lot of time. That has now been solved quite well. 00:12:27.440 --> 00:12:32.510 The students have the possibility to upload their identity documents and then you can 00:12:32.510 --> 00:12:39.080 go through the breakout rooms during the exam and check against the identity document, 00:12:39.080 --> 00:12:43.880 whether this is the right student. 00:12:43.880 --> 00:12:51.920 After the students have been admitted, there is a general address, which is also given in the present setting, 00:12:51.920 --> 00:12:58.400 where you encourage the students again and tell them by default if they are really well prepared, 00:12:58.400 --> 00:13:04.280 that nothing can happen to them and then they are allocated to the breakout rooms. 00:13:04.280 --> 00:13:05.630 Here is the idea, 00:13:05.630 --> 00:13:13.820 that in such a huge zoom meeting with up to 130 students in some cases, of course, you hardly have a chance, 00:13:13.820 --> 00:13:16.790 to carry out visual supervision. 00:13:16.790 --> 00:13:26.450 That's why we also sorted the students into a maximum of 25 in breakout rooms where there are two supervisors each. 00:13:26.450 --> 00:13:30.910 Supervising the students once, but also being approachable during the exams 00:13:30.910 --> 00:13:39.860 when questions arise. And once that's done, you can never plan exactly when that will be, 00:13:39.860 --> 00:13:47.480 you sit for a while and stare at each other, as is often the case, in the digital room. In the lecture hall 00:13:47.480 --> 00:13:53.270 you might be able to have another casual conversation and then maybe relax a bit better. 00:13:53.270 --> 00:13:58.520 And then at a point in time will have to give the sign to everybody that the rooms are ready, 00:13:58.520 --> 00:14:08.330 release the exam. Then ultimately the students click on the exam to edit and edit the Moodle test. 00:14:08.330 --> 00:14:12.770 And then there are the typical things to administer during an exam. 00:14:12.770 --> 00:14:17.510 These are questions, content-related questions, technical problems. 00:14:17.510 --> 00:14:22.370 Toilet visits have to be recorded and actually also to make sure that nobody goes at the same time, 00:14:22.370 --> 00:14:29.540 because you could then perhaps coordinate with someone on the toilet visit, for example via mobile phone. 00:14:29.540 --> 00:14:34.910 So it is about ensuring fair examination conditions for all, 00:14:34.910 --> 00:14:38.990 lose the students as little as possible from the field of vision. 00:14:38.990 --> 00:14:44.810 The field of vision is just very very small, unlike in a face-to-face exam. 00:14:44.810 --> 00:14:53.490 Last but not least, the students can hand in the test themselves or it is taken from them by the computer, so to speak, 00:14:53.490 --> 00:15:02.410 when the processing time has expired. Which is perhaps also a small advantage if you are particularly fast. 00:15:02.410 --> 00:15:06.520 In a face-to-face exam you often have to wait until everyone can leave. 00:15:06.520 --> 00:15:16.300 And with us it is that you can finally leave the room when you have handed in the exam. "How are the digital exams graded?" 00:15:16.300 --> 00:15:23.750 Grading a digital exam or a Moodle test just has many advantages. 00:15:23.750 --> 00:15:31.390 So, if you compare that with the face-to-face exam. There you collect these exams, then you carry a few kilos of paper to the office, 00:15:31.390 --> 00:15:36.400 sit down and have to count crosses for hours and days as I said 00:15:36.400 --> 00:15:43.930 and decipher moderately readable handwriting. When the Moodle test is handed in 00:15:43.930 --> 00:15:50.200 the computer corrects the multiple choice part and also the cloze texts at the same moment. 00:15:50.200 --> 00:15:57.220 So you just get a big time gain, which you can invest elsewhere, 00:15:57.220 --> 00:16:05.830 be it in the careful control of the open questions or in being able to concentrate on another counselling service. 00:16:05.830 --> 00:16:11.500 So I would say, for example, exams have to be efficient, 00:16:11.500 --> 00:16:18.790 because we as lecturers should simply invest more time in good homework guidance of students, 00:16:18.790 --> 00:16:25.150 because that is simply the more important kind of competence that one acquires in the studies or let's say such a competence, 00:16:25.150 --> 00:16:30.190 which simply has to be accompanied better than the recall of basic knowledge. 00:16:30.190 --> 00:16:35.740 I think that is a huge advantage that we can just let the computer correct exams, 00:16:35.740 --> 00:16:38.710 which it can do much better and faster than humans. 00:16:38.710 --> 00:16:46.640 Namely, looking to see if crosses are put in the right place or if a word is put in the right place. 00:16:46.640 --> 00:16:52.330 The great thing about Moodle is that you can also correct cloze text and also create tolerances. 00:16:52.330 --> 00:16:59.590 So, for example, there is a correct answer which says that voting systems have a psychological effect because I know as a voter, 00:16:59.590 --> 00:17:04.220 that if I vote for a very small party, my vote is often lost because they don't get any seats. 00:17:04.220 --> 00:17:12.670 That is a psychological effect. Then I can define when the students make various orthographic mistakes 00:17:12.670 --> 00:17:15.970 that I as a human being would accept when reading, 00:17:15.970 --> 00:17:24.790 then I can accept that with regular expressions. And then Moodle can also correct such cloze texts relatively flexibly on its own. 00:17:24.790 --> 00:17:32.500 What is also great about the assessment is that I can always see in a table how the points are distributed for the individual tasks, 00:17:32.500 --> 00:17:39.280 if these tasks have already been corrected automatically. And I can look at task 3, for example, where there are 5 points. 00:17:39.280 --> 00:17:44.680 I can look at the ones that have zero points or 3 points and see if there is a programming error. 00:17:44.680 --> 00:17:50.320 Does anyone perhaps have another orthographic variant of psychologically written that I would like to accept? 00:17:50.320 --> 00:17:54.700 Then I can still change the regular expression afterwards in the programming 00:17:54.700 --> 00:17:58.750 and this is simply taken into account afterwards. 00:17:58.750 --> 00:18:02.770 That's also a great functionality that I would never have in a paper exam. 00:18:02.770 --> 00:18:08.680 Then I would have to leaf through everything individually and that would take several minutes. 00:18:08.680 --> 00:18:12.760 With Moodle I can do it in a few seconds. 00:18:12.760 --> 00:18:21.610 What remains in comparison to the face-to-face exam or the paper exam is that you read the open answers and grade them classically. 00:18:21.610 --> 00:18:29.560 Then you enter points into a mask. Which, thankfully, is different from the paper examination, is 00:18:29.560 --> 00:18:38.770 that you type in the answers and the interesting variations in handwriting don't cost you very much time. 00:18:38.770 --> 00:18:45.260 Last but not least, Moodle adds up the points automatically. You have a tabular overview, 00:18:45.260 --> 00:18:50.020 which can be read out relatively quickly into Excel or, I do it with the statistics programme. 00:18:50.020 --> 00:18:53.410 You can look at the distribution of the points and then also find out again, 00:18:53.410 --> 00:18:56.710 whether the grading scale should be adjusted because the examination was a bit more difficult 00:18:56.710 --> 00:19:01.030 than intended and then use 00:19:01.030 --> 00:19:10.370 the distribution of points to assign grades and enter them in TuCan. "How do you handle the viewing of examination scripts?" 00:19:10.370 --> 00:19:17.360 Examination viewing is an important issue. Students often want to see where they have made mistakes. 00:19:17.360 --> 00:19:20.090 Some might also want to look again to see if they can still find that one point, 00:19:20.090 --> 00:19:25.130 that would give them a better mark. In attendance mode 00:19:25.130 --> 00:19:32.000 you would have administered it in such a way that people come by, you pick the exam and they can look at it again. 00:19:32.000 --> 00:19:38.450 In the digital space, you could actually publish the Moodle test. 00:19:38.450 --> 00:19:44.870 To be honest, I'm not that interested in that because the pool of questions is still limited and I would like 00:19:44.870 --> 00:19:50.660 to not have them circulating freely and to be able to prepare oneself completely with these questions. 00:19:50.660 --> 00:19:55.700 But I suspect that we might also get to the point where we then have such a large pool that it's perfectly okay, 00:19:55.700 --> 00:20:02.940 if the pool circulates and you then simply draw questions randomly from this pool and then still have fair examination conditions. 00:20:02.940 --> 00:20:09.590 So that would be a possibility. Just publish the test and everybody can look at it if they want to. Another option 00:20:09.590 --> 00:20:17.060 which we have used, is that you arrange to meet on Zoom and then go through the exam with shared screens. 00:20:17.060 --> 00:20:21.200 And if not many students want to do that, that's also good 00:20:21.200 --> 00:20:27.010 administrable. "How was the result compared to face-to-face exams?" I am quite open about that. 00:20:27.010 --> 00:20:31.450 What were the results like? They were flexible, adapted to the situation. 00:20:31.450 --> 00:20:39.130 So I had actually written the very first distance supervision exams at the TU Darmstadt at that time and there was a dress rehearsal, 00:20:39.130 --> 00:20:47.500 where the students were the test subjects for the infrastructure for 2 hours and then they were bored, 00:20:47.500 --> 00:20:53.120 because we didn't manage to do the identity control very well. 00:20:53.120 --> 00:20:57.310 They were the first ones with a view to the questions: What do you do in case of technical problems? How does 00:20:57.310 --> 00:21:07.510 the Moodle test work? I gave them a lot of credit for their compliance and constructive cooperation. 00:21:07.510 --> 00:21:10.960 From my point of view, that simply had to be taken into account. 00:21:10.960 --> 00:21:15.130 Accordingly, this examination turned out to be slightly better than the face-to-face 00:21:15.130 --> 00:21:17.080 examinations that I 00:21:17.080 --> 00:21:26.350 wrote at the TU Darmstadt under hygiene conditions during the first wave. With the further examinations 00:21:26.350 --> 00:21:31.240 this normalises and settles into the usual average. 00:21:31.240 --> 00:21:34.990 So I perceive in the end, 00:21:34.990 --> 00:21:40.870 that with these remote supervision examinations or with these digital tests, you have the same 00:21:40.870 --> 00:21:44.410 valid differentiation performances 00:21:44.410 --> 00:21:47.800 as with such a face-to-face examination. I don't have that impression now, 00:21:47.800 --> 00:22:00.580 that the students find it more difficult overall than in a face-to-face exam and that they also do not have any great problems in retrieving their performance. "Which aspects are continued in the post-Corona time?" 00:22:00.580 --> 00:22:06.040 In a nutshell, I really want to keep the digital Moodle exam. 00:22:06.040 --> 00:22:12.670 But I also want to have the face-to-face exam back. So a face to face 00:22:12.670 --> 00:22:14.860 Moodle exam is my dream. 00:22:14.860 --> 00:22:22.660 A big computer pool at the TU, where I can test my students with my staff, in presence. 00:22:22.660 --> 00:22:27.640 To be able to interact with them directly and at the same time also strictly and go with a wary eye through 00:22:27.640 --> 00:22:32.290 the rows and ensure without privacy concerns, 00:22:32.290 --> 00:22:36.490 that these are just fair examination conditions for everybody. 00:22:36.490 --> 00:22:40.720 Why the digital Moodle test? As I said, I find the questions 00:22:40.720 --> 00:22:46.750 variety is a big one. You have more scope, you also come up with new ideas again 00:22:46.750 --> 00:22:53.170 to test certain types of competences, which cannot be tested very well in the paper examination. 00:22:53.170 --> 00:23:02.590 And this insane gain in time through the quick correction, which can then be focused on more meaningful counselling services 00:23:02.590 --> 00:23:06.430 than counting crosses to see if they are in the right place. 00:23:06.430 --> 00:23:14.470 But why do I want the face to face test back again? Well, for one thing, I think it's very important for the students. 00:23:14.470 --> 00:23:19.540 The students also reported back to us that it is very difficult to get into such an examination mode. 00:23:19.540 --> 00:23:27.700 If you don't sit down at the computer in your teenager's room, where you maybe play computer games or watch videos from time to time, 00:23:27.700 --> 00:23:36.520 not sitting, so to speak, in this large community of examinees in a room that is really there to be examined. 00:23:36.520 --> 00:23:39.550 And it is also about the social element of students waiting in front of the room. 00:23:39.550 --> 00:23:46.060 They talk to their fellow students, encourage each other again, explain last questions, maybe they can still pass on, 00:23:46.060 --> 00:23:53.530 what has leaked out, what will not be covered or what will be covered. It is also important for the community experience of studying. 00:23:53.530 --> 00:23:58.330 Afterwards you go out relieved, have a coffee, chat together, 00:23:58.330 --> 00:24:05.350 what you do in the semester break or say again where you were unsure whether it is right. 00:24:05.350 --> 00:24:09.610 I would just like to have that back as an examination experience, so to speak. 00:24:09.610 --> 00:24:13.030 So not that the exams are necessarily pleasant 00:24:13.030 --> 00:24:20.050 from the students' point of view at all, but they can just get pleasant elements if you make it a bigger community experience again. 00:24:20.050 --> 00:24:28.570 And that's what I would like to have back. It also works within the framework of a Moodle test and if you complete it in a large computer pool. 00:24:28.570 --> 00:24:37.660 I would like to have it back with us as well, independent of these social components, just to be able to solve this supervision problem well. 00:24:37.660 --> 00:24:41.620 In the digital space, as I said, it's just a big struggle to recreate. 00:24:41.620 --> 00:24:46.600 And such tools like Proctorio I don't really want to use. 00:24:46.600 --> 00:24:56.080 Because that is very intrusive, I think, into the privacy and I don't want to use this Safe Exam Browser for remote supervision examinations, 00:24:56.080 --> 00:25:03.640 because that could cause new technical problems and then the exam is cancelled because this browser really doesn't work. 00:25:03.640 --> 00:25:10.660 But this Safe Exam Browser could work great if you do it in a big computer pool at the TU. 00:25:10.660 --> 00:25:17.020 So that's it in a nutshell: Keep Moodle Test by all means, but also presence by all means. 00:25:17.020 --> 00:25:28.320 Have it back as soon as possible. "In your opinion, where is the potential for electronically supported testing?" So a very big potential, which was related to the current situation, was, 00:25:28.320 --> 00:25:37.100 that a remote supervisory examination has actually enabled us to ensure an examination requirement and to ensure health protection. 00:25:37.100 --> 00:25:45.370 So I also changed it at relatively short notice at that time, because I simply had problems with the idea at the beginning of the wave 00:25:45.370 --> 00:25:49.960 to direct 200 students in public transport into closed rooms, and 00:25:49.960 --> 00:25:53.810 that can have really insane implications. If there is a positive case, 00:25:53.810 --> 00:25:58.690 200 people sit in quarantine for 14 days and they are just also worried, 00:25:58.690 --> 00:26:04.570 that they might have been infected somehow. So just being able to make a small contribution to 00:26:04.570 --> 00:26:10.240 to make sure that it doesn't spread further and that students can still be validly examined. 00:26:10.240 --> 00:26:14.680 That was simply a potential that we had with these electronic examinations. 00:26:14.680 --> 00:26:19.900 And I also found that this was possible at the TU at such short notice. 00:26:19.900 --> 00:26:27.040 Now the pandemic will hopefully be over at some point and even then there will be further potential for electronic testing. 00:26:27.040 --> 00:26:30.040 I could imagine that all in all there will be more flexibility, 00:26:30.040 --> 00:26:35.290 also with appointments and perhaps also with the question of where students take examinations. 00:26:35.290 --> 00:26:40.150 I can imagine if you have a very large pool of questions, 00:26:40.150 --> 00:26:45.520 which can also circulate, it is in principle possible that if someone is in a semester abroad, 00:26:45.520 --> 00:26:53.380 but still has to write this exam, that he logs on to a computer and then questions are generated for him at random 00:26:53.380 --> 00:26:56.200 which he then has to answer in a limited time. 00:26:56.200 --> 00:27:01.240 I had already addressed for me and colleagues as the ones who then have to correct, 00:27:01.240 --> 00:27:08.140 a huge potential is a time saving, which we can use for other things. 00:27:08.140 --> 00:27:12.850 And finally, I also see now with a very concrete look at Moodle, 00:27:12.850 --> 00:27:17.950 that you can use clever task formats and a larger variety of task formats, 00:27:17.950 --> 00:27:23.000 which can then also better probe the competences that are actually in demand. 00:27:23.000 --> 00:27:27.910 So maybe to make this more vivid, these time savings with multiple choice questions, 00:27:27.910 --> 00:27:37.300 because Moodle corrects that, I can then perhaps distribute it within the exam to a somewhat more demanding, more open question, 00:27:37.300 --> 00:27:43.630 because then I have more time to read through the text, which is also easy to read because it is not handwritten. 00:27:43.630 --> 00:27:52.210 And from my point of view, these are the most important potentials that lie in electronic testing. 00:27:52.210 --> 00:28:05.570 I see the potential in digital writing, but less in digital administration, digital supervision and digital implementation. 00:28:05.570 --> 00:28:08.540 "What was the feedback from the students?" We received mixed feedback from the students. 00:28:08.540 --> 00:28:15.230 Purely context specific, there was a lot of understanding especially with the short term change, 00:28:15.230 --> 00:28:17.990 that we did this, and it was also supported by everyone. 00:28:17.990 --> 00:28:23.450 So in my case, in the specific situation at that time, I announced the deadline two weeks before the exam: 00:28:23.450 --> 00:28:31.190 There is no way around a presence exam because, to be honest, I couldn't imagine how it would work. 00:28:31.190 --> 00:28:34.880 That was a very quick learning process for me, which was also triggered. 00:28:34.880 --> 00:28:39.770 With the HDA and also with colleagues who also wanted to try it out, I was also happy, 00:28:39.770 --> 00:28:44.540 that I did this. The students were simply grateful for this examination opportunity, 00:28:44.540 --> 00:28:55.940 without the worry of infecting themselves and then also infecting risk patients in their environment. 00:28:55.940 --> 00:29:03.800 So there was first of all a great agreement and also a great flexibility among the students and willingness, 00:29:03.800 --> 00:29:09.080 to work constructively to ensure that the first remote supervision examinations simply 00:29:09.080 --> 00:29:13.160 become a success. Now there is almost a certain routine. 00:29:13.160 --> 00:29:20.660 Well, more exams have been written and the feedback is still positive. 00:29:20.660 --> 00:29:28.400 But there is also criticism. What the students miss is really the exam atmosphere and we have been communicated, 00:29:28.400 --> 00:29:35.480 that it is really difficult to get into examination mode in their own youth room. 00:29:35.480 --> 00:29:42.210 I also found it interesting that some of them really dressed like they did during the examination. 00:29:42.210 --> 00:29:44.240 Just to construct a little bit for themselves: 00:29:44.240 --> 00:29:49.670 This is now a test, this is important and I have to construct this somehow for myself mentally, 00:29:49.670 --> 00:29:52.040 even if I am not in the examination room now. 00:29:52.040 --> 00:29:58.460 The students have to deal with additional worries, especially concerning the technical implementation. 00:29:58.460 --> 00:30:04.040 I find the comparison quite amusing. Let's imagine we are writing a presence paper exam. 00:30:04.040 --> 00:30:10.850 Suddenly the pen doesn't write any more or the paper on which the questions were just written suddenly turns white again. 00:30:10.850 --> 00:30:13.610 And that is ultimately the analogy of what can happen to you in principle, 00:30:13.610 --> 00:30:19.730 if the Internet connection goes down, if maybe something is wrong with Moodle. 00:30:19.730 --> 00:30:24.650 And of course you go into this test with a certain amount of concern as to whether your own Internet is sufficient, 00:30:24.650 --> 00:30:30.410 because maybe you are writing in a rural area or you have to share the bandwidth with family members. 00:30:30.410 --> 00:30:36.710 Because it's just additional worries that students have that they don't have when they write a presence exam. 00:30:36.710 --> 00:30:40.820 And there was just also feedback that this is an additional burden. 00:30:40.820 --> 00:30:44.300 And some also simply articulated that this kind of supervision, 00:30:44.300 --> 00:30:50.270 that we, to put it bluntly, look into their youth room and ultimately also have the right, 00:30:50.270 --> 00:30:55.760 to inspect the room if we have a suspicion that there are spickers there. 00:30:55.760 --> 00:30:58.970 They just found it very unpleasant. 00:30:58.970 --> 00:31:03.590 And that is also really a lot to ask of the students. They have to have technical equipment ready. 00:31:03.590 --> 00:31:08.570 They have to be prepared to let us invade their privacy a little bit in the end. 00:31:08.570 --> 00:31:22.270 And therefore, as I said, I would be happy if we could do digital tests in the public university TU space presence. 00:31:22.270 --> 00:31:28.630 "What are the lessons learned?" I would start with a slightly more abstract point that I've noticed so with myself and also with colleagues, 00:31:28.630 --> 00:31:35.590 that there was a pleasing agility from all places and I must now compliment myself on this. 00:31:35.590 --> 00:31:41.920 So we were all very attached to the paper exam and could not imagine anything else. 00:31:41.920 --> 00:31:47.530 And once that was pushed, we were very quickly with support in a position to 00:31:47.530 --> 00:31:54.160 very quickly inbring the examination into a completely different format and transfer it into a digital space. 00:31:54.160 --> 00:32:01.210 And I just thought that was a nice experience, that you found out that you were actually already a bit stuck, 00:32:01.210 --> 00:32:05.830 but then you can break away from it and then just be solution-oriented in the sense of the 00:32:05.830 --> 00:32:13.810 health protection and the examination requirements. 00:32:13.810 --> 00:32:21.580 Lessons Learned concretely refers to the amazing, varied, diverse possibilities, 00:32:21.580 --> 00:32:27.640 to test wider competence space with Moodle and to simply acquire the skills here as well. 00:32:27.640 --> 00:32:33.280 To programme some clever tasks that are perhaps also a bit interactive. 00:32:33.280 --> 00:32:44.250 For me, I would say, these are the two most important lessons, abstract and concrete, that I take away from it. 00:32:44.250 --> 00:32:51.640 "Tips and tricks" Looking also at the support that is available from the HDA, the changeover is well manageable and you have investment costs. 00:32:51.640 --> 00:32:55.660 You have a bit more to do at the beginning and you have to deal with new things. 00:32:55.660 --> 00:33:06.020 But once that is in place, it can really be the basis for valid and efficient examination, especially if you have quite a lot of students. 00:33:06.020 --> 00:33:09.810 I wouldn't recommend it so much for smaller courses. 00:33:09.810 --> 00:33:20.610 But if I really have to examine 200, 250 or even more students efficiently, this is a very good possibility. 00:33:20.610 --> 00:33:25.980 Which I wouldn't necessarily advertise now that we should all stick to these remote supervision exams. 00:33:25.980 --> 00:33:30.930 If that is no longer necessary pandemically at all, then I would really advertise that we should all commit to it, 00:33:30.930 --> 00:33:38.700 that the TU provides us with a large computer pool, where we can then administer it. 00:33:38.700 --> 00:33:44.780 And yes, otherwise tips and tricks: 00:33:44.780 --> 00:33:52.550 I would say that it is important to appreciate again that we demand a lot from the students, 00:33:52.550 --> 00:33:56.960 and also from us a lot is demanded, because we have costs for changes and have to get involved in it. 00:33:56.960 --> 00:34:01.820 Suddenly completely different institutions are talking to us about the administration. 00:34:01.820 --> 00:34:06.140 Before, you did it a bit more autonomously in the lecture hall, you knew the rules. 00:34:06.140 --> 00:34:11.420 And now there are very precise rules about what you are allowed to do in this zoom room and what you are not allowed to do. 00:34:11.420 --> 00:34:19.850 Whether the micros have to be employed or not. In the past, you had a little more autonomy in the presence examination. 00:34:19.850 --> 00:34:25.490 But you get a lot of freedom through digital exams, 00:34:25.490 --> 00:34:42.218 which you can then focus on more interesting and important support services, especially in terms of time.