Ethel for your Course

In Spring semester 2024, there are already several opportunities for lecturers to get involved with developing Ethel. Note that Ethel is not a service; it is a practical and oftentimes pragmatical experiment. Thus, we also do not have customers, but would welcome collaborators.

Custom Chatbot for your Course

What is it? Your course could have a custom chatbot, based on your course materials. As students interact with the bot, answers will be based on the materials you provided. This could include lecture scripts, slides, exercises, etc. We also experiment with multimedia, such as your annotated lecture slides or lecture recordings. We can handle much larger volumes of data than commercially available custom bots like https://ethel.ethz.ch/

What are the risks? The underlying strategy of Retrieval Augmented Generation is reasonably well-established, but we found that it can also fairly quickly reach its limits with, for example, highly mathematical content. We are trying out different methods to overcome these challenges, but we cannot guarantee stellar results.

What do we need? Any material you can give us and your patience. We do not have the resources to process materials purely for curiosity's sake, but would limit this opportunity to courses which would actively use the bot in a given semester. We would also be thankful if you could ask your students to use the bot and give us their feedback.

When would we need this? Basically anytime. It will likely take a week to process your materials, and we are currently working on mechanisms to add materials as the course progresses.

Grading Assistance for Handwritten Exercises and Exams

What is it? For much of what we teach, not only the final results, but also its derivation is important. When it comes to mathematical content, there is no way around handwriting, as typesetting extensive derivations or proofs is unreasonably time consuming. We are working on the pipeline for converting handwritten mathematical expressions into a machine-readable format and subsequent annotation or assistance in grading by Ethel.

What are the risks? There are too many risks to do this online and in production. With IRB-approval, we are running asynchronous, offline experiments to optimize the process, gather experiences, and evaluate results.

What do we need? Your students would need to agree to participate and sign informed consent. We would then need scans of the ungraded assignments of the participants, as well as electronic documents with the problem, model solution, and grading rubric. In the end, we would need the results and comments of your regular grading process for comparison.

When would we need this? We are currently working with a high-stakes exam in a large-enrollment course, and this will likely keep us busy for a few weeks. However, we would welcome participation of courses with low-stakes exercise sheets, where we could also make the AI-feedback available to your students (formative assessment).

Grading Assistance for Jupyter Notebooks

What is it? The gold standard for assessing the correctness of coding exercises are unit tests, where the properties of code segements are explored using carefully crafted statements with inputs and expected outputs. Writing good unit tests can be cumbersome, and for low-stakes exercises that embed computation in non-computer-science courses, might not be needed. A frequently used tool for such low-stake exercises are Jupyter notebooks, and the idea is to provide feedback on those using natural language rubrics.

What are the risks? Essentially the same as for feedback on handwritten assignments; Ethel might provide less-than-helpful hints or code evaluations.

What do we need? Also essentially the same as for feedback on handwritten assignments, except in this case the ipynb files instead of scanned sheets of paper.

When would we need this? We do not have IRB approval for this study yet, but would love to work with you on setting it up.