How long before Respondus introduces an education equivalent of BattlEye or other kernel-level anticheats as a result of stuff like this?
And I don’t mean the Lockdown browser, I mean something beyond that, so as to block local AI Implementations in addition to web-based ones.
Also, I’m pretty sure there’s still plenty of fields that are more hands-on and either really hard or impossible to AI-cheat your way through. For example, if you’re going for carpentry at the local vo-tech, good luck AI-cheating your way through that when that’s a very hands-on subject by its nature.
I would argue that in person exams with no resources to do research goes against how the world works for most white collar workers.
Few are unable to research on the internet to verify information, or at least look at say a man page for coding or look up past stuff on stackoverflow, if they are working through a problem.
Standardized testing is just not as useful as-is. I do great at it and can typically pass exams without really studying the material, but others are not so lucky.
I’ve met people who can flunk exams but talk about the problems, go into how they would fix it, and work through a problem to implementation and testing in the real world.
Oh, and LLMs are the new typewriter, for better or worse. It’s unlikely we are going to have a future where they are not readily available. We already have models that run locally and do not transmit data anywhere, and AI customized to your own data that is not shared is already a service provided by Microsoft.
Education needs to evolve with technology. It’s always been 5-10 years behind the curve.
Maybe we should be using LLMs to proctor tests and generate interactive testing. Grading can be verified by a professor reading a transcript to verify hallucinations didn’t occur or influence the results. We can even have LLMs monitor the working process of people to help determine what are the most efficient ways to work custom tailored to individuals. This is just one idea of many potential options.
Those are all very nice ideas, and we’ll see if they pan out in the future. But universities need ways to stop (or, fine, reduce) cheating that can be implemented right now. A class in English literature and composition should test how well you can read and interpret the source material to then express something about it in your own words in a coherent way. This is a useful life skill to have, and students should learn to do it without AI assistance. Giving them a pen and paper and a quiet room to work in has been a good enough method of assessment for at least the last 50 years which is reasonably cost effective.
Yes, there are problems with standardized testing. Yes, you can cheat on a paper test. But the way to improve the evaluation process is to first establish a stable baseline, and then try new things that might work better to see if they actually work better. Not to throw out everything we knew before and haphazardly try every random idea that pops into someone’s head in a panic.
Back when I was in grade school in the mid 1990’s, we were one of the first families to have a computer. We weren’t allowed to ANY schoolwork on it. If you had to write a paper, it had to be written by hand. Which, as someone who could type much faster and used bigger words, was REALLY fucking annoying.
But yeah, I imagine we need to go back to dumb, disconnected computers in exam halls to keep things above board. It’s depressing to see how lazy this tech makes students.
Or even actually show what they learned in a practical sense. In a vo-tech, for example, have the students fix up a car or get a small LAN set up, or even in the case of an art school, have the class do a mural outside as their end-of-instruction project (which sounds like a really fun end-of-instruction project, btw), with admin approval, of course.
I like this idea, but I also think that we should keep in mind that the time of university staff is expensive, and with the already outlandish cost of education we need to strike a balance
How long before Respondus introduces an education equivalent of BattlEye or other kernel-level anticheats as a result of stuff like this?
And I don’t mean the Lockdown browser, I mean something beyond that, so as to block local AI Implementations in addition to web-based ones.
Also, I’m pretty sure there’s still plenty of fields that are more hands-on and either really hard or impossible to AI-cheat your way through. For example, if you’re going for carpentry at the local vo-tech, good luck AI-cheating your way through that when that’s a very hands-on subject by its nature.
Or, ya’know, they could just have students take tests on paper in a lecture hall.
That’s what we used to do, 15 years ago though
Because nobody ever cheated on a paper exam before.
perfect not being the enemy of the good and all that
I would argue that in person exams with no resources to do research goes against how the world works for most white collar workers.
Few are unable to research on the internet to verify information, or at least look at say a man page for coding or look up past stuff on stackoverflow, if they are working through a problem.
Standardized testing is just not as useful as-is. I do great at it and can typically pass exams without really studying the material, but others are not so lucky.
I’ve met people who can flunk exams but talk about the problems, go into how they would fix it, and work through a problem to implementation and testing in the real world.
Oh, and LLMs are the new typewriter, for better or worse. It’s unlikely we are going to have a future where they are not readily available. We already have models that run locally and do not transmit data anywhere, and AI customized to your own data that is not shared is already a service provided by Microsoft.
Education needs to evolve with technology. It’s always been 5-10 years behind the curve.
Maybe we should be using LLMs to proctor tests and generate interactive testing. Grading can be verified by a professor reading a transcript to verify hallucinations didn’t occur or influence the results. We can even have LLMs monitor the working process of people to help determine what are the most efficient ways to work custom tailored to individuals. This is just one idea of many potential options.
Those are all very nice ideas, and we’ll see if they pan out in the future. But universities need ways to stop (or, fine, reduce) cheating that can be implemented right now. A class in English literature and composition should test how well you can read and interpret the source material to then express something about it in your own words in a coherent way. This is a useful life skill to have, and students should learn to do it without AI assistance. Giving them a pen and paper and a quiet room to work in has been a good enough method of assessment for at least the last 50 years which is reasonably cost effective.
Yes, there are problems with standardized testing. Yes, you can cheat on a paper test. But the way to improve the evaluation process is to first establish a stable baseline, and then try new things that might work better to see if they actually work better. Not to throw out everything we knew before and haphazardly try every random idea that pops into someone’s head in a panic.
Doesn’t even need to be paper. Have locked-down, internet-disconnected computers in the exam hall bas glorified typewriters.
Why not a middle ground? Have them only access a local network version of Wikipedia + a verified library to search
Back when I was in grade school in the mid 1990’s, we were one of the first families to have a computer. We weren’t allowed to ANY schoolwork on it. If you had to write a paper, it had to be written by hand. Which, as someone who could type much faster and used bigger words, was REALLY fucking annoying.
But yeah, I imagine we need to go back to dumb, disconnected computers in exam halls to keep things above board. It’s depressing to see how lazy this tech makes students.
Exactly, that’s how it works in my country. I think the PCs are connected to a local server that then matches the results to your id and email.
deleted by creator
Or even actually show what they learned in a practical sense. In a vo-tech, for example, have the students fix up a car or get a small LAN set up, or even in the case of an art school, have the class do a mural outside as their end-of-instruction project (which sounds like a really fun end-of-instruction project, btw), with admin approval, of course.
Academia isn’t really that practical
I like this idea, but I also think that we should keep in mind that the time of university staff is expensive, and with the already outlandish cost of education we need to strike a balance
tools like that were going big in the pandemic for online exams. Basically rootkits that fully compromise your machine