Examining AI Detection Reliability

Questions about whether AI detectors trigger false positives, particularly for average students whose writing resembles AI output. Concerns about logs revealing copy-paste behavior and sophisticated notation never taught in class.

← Back to Failing grades soar with AI usage, dwindling math skills in Berkeley CS classes

While AI detectors are frequently criticized for their unreliability and tendency to flag average students whose writing mimics AI patterns, instructors often rely on more concrete behavioral evidence to identify academic dishonesty. Digital logs frequently provide "smoking gun" proof by revealing students who copy and paste complex, perfectly formatted solutions mere seconds after opening an assignment for the first time. Beyond technological detection, educators in STEM fields note a suspicious shift toward hyper-formal notation and sophisticated coding techniques that were never taught in the classroom. Ultimately, there is a growing concern that this low-effort access to advanced tools is diminishing students' fundamental ability to perform during live assessments.

4 comments tagged with this topic

View on HN · Topics
Or how many are normally caught cheating? Did they use AI to detect AI using cheaters?
View on HN · Topics
And if cheating was triggered using AI detectors, was it real? AI detectors are pretty mid in practice - they tend to have a lot of false positives for "B" students who are okay, but can still be struggled to be more coherent than AIs are. There are some specific triggers that AIs are way more likely to do than students, but a lot of AI detectors will trigger on this "almost there, but you're still struggling" level of essay writing that might get a B, B-. I could expect the same might be true for CS students even though I haven't seen how AI detectors work for CS/math homework.
View on HN · Topics
You'd be amazed at how many students we know are obviously cheating because the logs reveal that they copy pasted a long, complete answer within seconds of opening a problem for the first time, full of sophisticated code constructs that we didn't teach them, and lot's of nicely formatted comments. Sometimes they even copy/paste the entire GPT output and then format it down.
View on HN · Topics
This has been my wife’s experience as a college math professor. Instead of code it’s extremely formal problems with way more steps than the student normally performs using notation never taught in class. It’s not that students didn’t cheat before, LLMs have just lowered the bar so far many can’t complete a live test in a class that requires effort.