Who wrote this, you or DeepL?

A Goethe-Institut research project has developed a self-learning computer program that does more than just detect fake text submissions. Victoria Engels spoke to scientist and researcher Leo Sylvio Rüdian, who explains how the program supports teachers in their daily work tutoring German online courses.

By Victoria Engels

For this research project, the Goethe-Lab Sprache, a Goethe-Institut innovation unit, collaborated with scientists and researchers Professor Niels Pinkwart and Leo Sylvio Rüdian from the Humboldt-Universität Berlin between 2019 and 2020.  Based on cooperative case studies the team’s overall goal is to verify to which extent natural language processing techniques combined with machine learning can be utilised for automated feedback for open writing tasks in online German courses.

Leo Sylvio Rüdian is a scientist at the Weizenbaum Institute, a doctoral student at the Humboldt-Universität Berlin, and a member of the AI Campus project of the Educational Technology Lab at the German Research Center for Artificial Intelligence (DFKI). He spoke to Goethe-Institut's Victoria Engels about the project.

What is the difference between artificial intelligence, neural networks and self-learning programs?

Leo Sylvio Rüdian | Photo (detail): © Leo Sylvio Rüdian We use the term artificial intelligence to describe processes that learn through data and form decisions accordingly. The underlying idea is to show processed data from the real world, such as pictures of cats and dogs for instance. A neural network can then be trained to recognise which images depict cats and which show dogs so that for any new images it will recognise which of the two animals it is ‘seeing’. Structures like neural networks can help us to make predictions. However, there is one pitfall: they only show us an answer they do not reveal how they come to their conclusions.
 
You are researching automated feedback for open writing tasks. What exactly is at the centre of the joint project between Humboldt-Universität Berlin and Goethe-Institut?

During numerous joint workshops with the Goethe-Institut we observed a huge gap between our theoretical research and its practical application. We have been working with many different types of technology which, despite showing great results, still haven’t been practically introduced in the German online courses. One of the areas we have looked at is automated text analysis, where right from the beginning our goal was to transfer our research findings to the online courses of the Goethe-Institut. In the first phase, we are focusing on supporting tutors with providing feedback for open writing tasks.
 
Which problem could an automated text analysis solve in this context?

One of the main duties performed by teachers is evaluating and giving feedback for open writing tasks. Teachers must check whether a text is appropriate for the relevant course section, whether students have used the acquired vocabulary and the correct grammatical structures. To do so, teachers require extensive meta-knowledge of a course. We wanted to find out whether we could develop a process, which is capable of giving feedback for free writing exercises, thus, providing a support system for teachers. In fact, there are several objective parameters, which could also be processed by algorithms.
Detecting fake submissions is part of language teaching these days | Credit: Dom Fou / Unsplash Which data did you use to train the program?

We work on two levels; using both the course content and already marked free writing tasks submitted in the past. One difficulty we face is that for each writing task, we have to retrain the process from scratch. This is very time consuming as for each exercise we need hundreds, perhaps even thousands, of previously marked text submissions.
 
The self-learning program is well equipped to detect fake submissions. Could you explain more about this specific use case?

Sometimes students look to automated translation tools such as Google Translate or DeepL to complete their writing tasks. Thanks to our objective criteria we are usually able to detect these fake submissions. Those texts often contain linguistic structures and words that are not included in the relevant course. Additionally, they often show very few mistakes. Our algorithm will flag any combination of very few mistakes and high linguistic complexity in a text and subsequently send an alert to the teacher. The warning prompts the teacher to take a closer look at the relevant submission.
 
What does the future of language learning look like?

The future of language learning will match individual needs based on each student’s previous knowledge and preferences. Where, up until now, a personal tutor was needed for an individual learning experience, technology-based solutions will soon provide personalised, much more efficient approaches to learning a foreign language. These solutions will still not replace teachers any time in the near future - instead they will support them in their work.

This interview first appeared on the website of Goethe-Lab-Sprache in November 2020.