Solution:
Warn the students throughout the year “If you use AI all year for your schoolwork, you will fail the exams every month or two”. Teach AI literacy, how it works, and the dangers of not learning the material and how it’s a snowball effect. Teach the “why” it’s important to use your brain to solve the problem, and state that they should be using their own words/math/etc. Students suspected of using AI should have a note sent to their parents early on to hopefully correct their path early.
Exams are paper and pencil only. No computers. No phones. No smart glasses. When students are getting an A on their homework and flunking their exams, it’ll be pretty obvious. Even a student who has anxiety about exams can get a C.
I wonder if it would be suitable to adjust people’s grades each year on their final exam.
For example instead of having assignments worth 25%, attendance 10%, quizzes 20%, midterm 20%, and final 25%
Have it as assignments 75% and attendance as 25%
Then adjust all of that for by midterm and finals
Get 100% on your assignments due to chatgpt and show up to class every time despite not paying attention, if you get 50% on your quizzes, midterm, and final you get 50% overall, not 67.5%
Even before AI was a big thing, I really wish we had some kind of class, or at least an explanation from our teachers on the basics of how learning works. So much unnecessary drama could have been avoided if the kids had a better understanding of why their teachers were asking them to do things this way instead of just saying “do it this way because if not I’ll give you a bad grade.” Obviously younger kids aren’t going to be equipped to handle all the neuroscience, but I’m sure there is some simplified explanation that could be given that would get the point across.
This was an extreme example of this, but it was emblematic of the general way my teachers handled students who didn’t understand the point of the assignments or teaching methods: I forget which grade I was in, but in one of my math classes there was a day in class where I was solving the problems but not doing it exactly the way the teacher was teaching it. When he insisted, I asked why I had to do it that way if it works either way. He said something like “Because I have the big desk.” Basically just an appeal to authority without any further explanation. “You’re a dumb kid and I’m an adult, so do what I say.”
That would require a number of teachers that actually understand or care about FoI which we don’t have.
What is FOI in this context? The closet thing I found to something that makes sense would be Freedom of Information, but that doesn’t seem quite right, or I’m not understanding the connection.
Fundamentals of Instruction.
Ah. Thanks!
Not every uni teaches stuff like how to study, fortunately one of them does and the lecture is on the Internet. I wish I had found this earlier. https://youtu.be/IlU-zDU6aQ0
That would be great if the system wasn’t rewarded for passing students.
they went from failing students often, to starting with participation to barely passing in 2000s, to unable to fail students in HS now. remember referrals they were bad mark on a students record,.
Yeah it really needs to be on leadership to help implement these kinds of things at the state, local, and school level.
Maybe the bigger problem is that the education system is teaching things that AI is already able to ace. Love it or hate it, AI isnt going away and it also has serious and persistent shortfalls that probably wont be resolved without further breakthroughs. We should stop feeding students busy work and pretending its an education.
My wife is a teacher, she has shown me vibed handed in assignments abd its incredibly obvious.
Right off the bat, if she gives an assignment to make, say, a slideshow on “Topic” and they talk about a examples A, B, and C in class, and the assignment goes off on tangents about topics F, G, and H instead, it’s an instant red flag.
This happens cuz the student just copy paste the assignment blurb into gpt, but gpt has no context for what was discussed in class… so it goes off the rails instantly.
Its also easy to include poison pills in the middle of an assignment if they copy paste it straight into gpt.
Also theres all the usual markers. Emoji, em dash, and the assignment having way higher verbosity than you know damn well the kid has the vocabulary for. Suddenly they’re speaking at a grade 7~8 levels higher than usual? Uh huh. .
From her and her teacher friends, Ive been told its extremely obvious to spot still. And its pretty trivial to setup the assignment to poison pill the AI.
Imagine handing in a paper with emojis… I just can’t
What if the kid lies and says they didn’t use AI? How successful have they been in convincing admin and parents of their ai usage? While I agree its all damning, its still circumstantial evidence.
She just failed the assignment, because it didnt actually meet the requirements lol
Thats the thing, if you simply design your assignment well, AI will just… fail at it, and you dont have to prove they used AI, they simply just didnt pass.
Back in the day just one instance of plagerism was very serious. If you got caught doing it more than once you could get expelled.
Now apparently everyone is using the plagerism machine including the professors. So much for academic integrity.
and then professors using AI to sniff out your AI. resumes are being done by AI, and employers are also using AI TOO. Also research papers are apparently done with AI too, some how sneaking into journals.
I had a sustainability class where the professor used AI to write the course syllabus, assignments, and feedback. a fucking sustainability class.
I contacted the office of the president about it at my university but nothing ever happened of it. academia in general has gone off the rails with AI recently. I used to assume those with doctorates we’re bright enough to avoid AI but evidently that’s not the case.
professors are often to busy with thier labs or research, so they relegate thier writing/teaching to AI. before it was only done with power-points that barely gave substance to a lectures those were the bad teachers when they just read off or give little context to the powerpoint slides, comes time for a midterm, students were like wtf was she teaching in those lectures.
Never underestimate how fucking lazy humans can be.
What if the kid lies and says they didn’t use AI?
Have them re-do the assignment in a classroom with the teacher (or any other procror) present.
I don’t teach kids, so I don’t know the answer to this, but I imagine what you’d do is add guidelines to the assignment that cause them to either lose significant points or fail if they don’t specifically mention things discussed in assignments and the classroom.
I’d also like to point out that, yes, we know when kids and adults lazily insert a prompt and lazily paste its response, but anybody with half a brain knows they only need to spend an extra 15 minutes re-prompting and editing it to make it nearly unnoticeable.
The answer is probably to test them in person with no computer of any kind in front of them.
Amateurs, everyone knows you record classroom discussion, translate it to text, and then feed it into the LLM for context.
I was unable to get Mistral’s AI to output an emoji recently. They forbid it in the system prompt and it wouldn’t give one out for a pretend life or death situation.
In some schools, teachers look the other way, so their pay doesn’t suffer even more. The administration refuses to admit reality, so their funding doesn’t suffer.
Sure as hell ain’t my students; it’s been a steady decline since ChatGPT came out and I think I may have failed more students than ever over unfinished projects. You can’t GPT the semester long project, there’s a paper trail and data to collect can it becomes super clear who is AI brained now…
Edit: PS, grade inflation has been a thing for a few decades now, btw; the As aren’t the problem so much as the mush brain.
You are unfair, you don’t let them use ChatGPT during tests 😉
They’re not grading ChatGPT performance either
I think they are saying add chatgpt in because if everyone adopts it outside and then they cant do it on the test then the test now does not effectively tell how the person is able to think to solutions because the test itself is “out of spec” of the typical person needing to do the things on the test.
Its like when calculators were allowed then disallowed then allowed again and now we can’t go without them a lot of the time while everyone now does more complex math and gets into higher level data sooner as a result of technological adoption.
The nature of the tests and teaching/learning now must change since we are looking at people who can do the combo of themselves plus chatgpt.
Ah. Good old days in the 80’s when teacher didn’t even read what you wrote. Grade was given according who you were, did the teacher like you and what your previous grades were. No sudden inspirations to do better.
Replace “teacher” with “management” and you’re describing every workplace.
This happened to me. I was a pretty good kid; brainy too, but one history test kicked my ass. The teacher was the husband of my grandfather’s sister. A distant family cut me a break.
This phenomenon can continue into adulthood too.
However, after a certain point it’s not about the grades you make. It’s about the hands you shake.
Education needs to change. Including punishment for using LLMs.
Before you can punish for using LLMs, you need to be able to reliably detect the use of LLMs, including guarding against false positives.
Current AI checkers are woefully inadequate and prone to errors.
A teacher I know says it is easy to determine if a student wrote their paper if you interview them about it. You’re right that automated methods are risky.
That’s it. As a teacher who has been dealing with this in the last 2-3 years, the only reliable way I have found is to do short interviews.
Students hand in their work, I grade it, then I ask them verbally a few easy questions about what they mean in specific sections of their work. How they score on these questions is used as a coefficient that I apply on the grade to get the final score.
So they can use LLMs, but they have to understand its output.
Interviewing every student would take a lot of time.
Before you can punish for using LLMs, you need to be able to reliably detect the use of LLMs, including guarding against false positives.
You can tell they’re using an LLM if they have a computer out during the pen-and-paper test.
How is that allowed?
Hell, back in my day, teachers were even very picky about what kind of calculator you could use. And if it was a graphing calculator, you had to show them yourself wiping the memory at the beginning of the test.
(Except for one algerbra teacher, who was really cool about it. He’d allow custom programs to stay on the calculator if you programmed it yourself. On the theory that if you can write a computer program that reliably solves these math problems, then you must have a very good understanding of how to solve these math problems. And, yes, I was one of the few kids who actually did that. Ah, writing my own custom software for the TI-83 on the TI-83, because that seemed easier than actually doing the math problems by hand … good times.)
Not US, but there’s a tendency of focusing more on the work during the semester than in the exam itself
LLMs are going to be a massive headache for me when they get older
but there’s a tendency of focusing more on the work during the semester than in the exam itself
Perhaps this tendency needs to be reversed?
If you have one big exam (or a few of them spread over the year) that it’s impossible to use LLM help for and those exams carry enough weight to make the student fail the class if they completely bomb it … then you’ll be stopping the LLM-cheaters dead in their tracks. Sure, they can be lazy and do much of their coursework that way, but if they’re being lazy like that, they’re likely not actually learning anything, and that will show up during the big exams. And when they fail those big exams and then fail the class, hopefully they’ll learn their lesson about relying on LLMs to get them through classes.
Right, and that’s what’s going to have to change: a bigger focus on things like in-person tests (including in-person bluebook essays), oral presentations/thesis defenses instead of other project deliverables, etc.
As someone who works in ed tech these days, I’m kind of down for them as a study tool. For example, synthesizing notes and turning them into flashcards, practice tests, etc. I find that stuff to be suuuper handy if I’m trying to learn something.
But for cheating, yah, fuck that noise. A lot of classes are moving back to pencil and paper because of this, and I totally support that.
I feel like synthesising notes and turning them into flash cards how i learn things.
I’m also in this camp. That said, I’ve learned that everyone studies differently, and some people learn by synthesizing, and others do better when they have something structured from the jump.
I often run to weird little study teams where one person studies by taking everyone’s notes and creating sets in something like quizlet, and someone else studies by using the flashcards / tests that get spit out.
Exactly. Taking notes in class during a lecture. Copying something the instructor wrote on the board. This is all part of the learning process. The act of doing these things helps you learn.
The only skills or learnings I really seem to have retained from University are the ability to collect, and collate information and then apply it to a problem. The actual information collected and problems solved are lost to me now.
We are allowing LLMs for all of our homeworks. As long as you can solve the problems in the indicated way with a reasonable answer.
In case you are not sure about the “indicated way”, there are practice questions with detailed step-by-step solutions for each hw problem that you just have to change the numbers/equations a bit and you’ll get points.
What we’ve noticed is that the year-after-year averages are significantly higher, especially this year. However, students are bringing in details that we explicitly didn’t go over in lecture and putting that on the homework (e.g. Delayed branching in Computer Architecture, because it’s a random quirk of MIPS that even assembly programmers don’t have to deal with). None of these details are ever mentioned in lecture or the practice homeworks (in a few cases, they are mentioned with the explicit wording “do not worry about this now”)
We can only assume people are copying the homework into LLMs and copying the results straight down. The latest exam had a question where students were asked to analyze a specific chunk of assembly code to deduce certain properties about it. Approximately 20-30% of the students didn’t know the FORMAT to answer it, despite it literally being item 1 on last week’s homework.
And when I say format, I don’t mean exactly “you must write these exact words or you lose points”. It’s literally just point out “line A and B have this property X because of attribute Y”. Just including ABXY as shown in the practice homework is enough. But apparently people are too lazy to read a 10 bullet point answer…
Then those people will go and fucking vote. Fucking hell
Let’s be honest, with their attendance rate in class, I don’t think these students actually vote…
and they’ll be assured they’re deserving of a tech sector job while everyone else is already losing their everything.
Then why issue homework at all?
Because the goal is to get people to learn/think about something. We don’t care what you use as long as you retain knowledge taught in the course. If what helps you learn is LLMs, then go for it.
Problem right now is there is a significant amount of people that are using these tools to do the thinking for them. And this is when Office Hours, Homework feedback, Email (I guarantee all students emails are responded to within 24hrs. Most are handled within 30 minutes) are all available and paid for (by tuition). I am even happy to schedule one-on-ones if privacy is a concern, but none of this is being utilized.
Yeah this is having a huge negative impact on industry. New grads expect step by step process for every task. Every year there is less ability to solve problems. I don’t know what the solution is but even if we remove llms from the conversation we have an issue with graduates not being able to problem solve. Add on top of that the influence of social media and the constant fear of being wrong and you end up with paralysis once these individuals enter the workforce and are required to solve bespoke problems.
I dunno, they’re here to stay. Cat’s out of the box. Educators and education need to adapt. In person assessment is probably the ideal way to gauge progress and learning, but due to resources I don’t see it being practical.
they’re here to stay. Cat’s out of the box.
People keep saying this as though it’s true. The odds that this current era of free and ubiquitous access to these frontier LLMs lasts forever are pretty slim.
How do you figure? There are open source self host able solutions right now.
Already, very few middle schoolers have the tech savvy to self-host anything. If it’s not a tablet, they have trouble using it.
Add to that the possibility that the data center run on memory and processors could mean that local computing power will disappear, to be replaced with devices like Chromebooks that use corporate cloud services for everything.
You can’t run anything like a frontier model on a self hosted solution. To get anywhere close you’d have to spend thousands of dollars on hardware which obviously isn’t free, or even a viable solution for the vast majority of people, let alone these students. And the quality of output you’d get from a model running on off the shelf consumer hardware like a MacBook is much more noticeably AI generated and trivial for AI detection tools to flag.
My point exactly. Same thing with just looking up the answer on Google or whatever
Cat’s out of the box.
The monkey’s out of the bottle.
Deserved. 🤣
Except the whole point of education is to LEARN how to do it without these tools. If you’re just turning your brain off and handing in the output, you are literally missing the point.
It’s like using calculators on steroids. There are times to use calculators and times to force mental math. You can teach kids AI literacy and usage habits, but letting them just use no thinking makes the entire exercise pointless. We might as well close schools, because having the AI generated your math homework or essay is fucking pointless.
You wildly misunderstood my post. Firstly, I’m not suggesting students turn their brains off. Secondly, how you learn isn’t relevant to the demonstration and application of that knowledge, which is precisely why I said in person assessment is the optimal way. You ask the student and watch them, live. This is how you defend a thesis, in front of a live panel. No tools, no cheating, just your knowledge.
What I am suggesting is that the system should adapt to the reality of the technology.
Isn’t that how it is suppose to work?
At the end of class you get a grade.
so you regularly handed in work that was not your own?
No. But I always got a grade.
‘A’ grades are suddenly everywhere, as introduction of stochastic parrot as a service reveals the education system is geared towards training parrots instead of teaching humans. What a surprise.
so I need you to explain to the class why 2+2 = 4 and why it can’t be any other answer but 4.
if you can’t then it means your opinion is wrong based on the fact that you are clearly dumber than a “stochastic parrot”.
if you can it also means your opinion is wrong based on the fact that your " stochastic parrot" education was more than just memorization and regurgitation and you learned how to think critically(but clearly not critically enough).
so, which tier of “wrong” do you want to belong in?
The good students are still getting A grades naturally. And the bad students are getting A grades with ChatGPT. A grades for everybody! (Until we get to the closed-book, in-person test at the end of the year…)
Surely this won’t cause any problems at all
Shouldn’t the education system change then? If it is easy to get an A with a machine, should we then not focus on learning something that can’t be done by a machine. I mean, it has value to know things and know what to do insituations where AI is not available.
Are you suggesting that kids should stop learning basic arithmetic?
so your answer is to regulate education? one of the most heavily regulated industries, education, you want to further regulate because of the influence of a brand new unregulated industry.
Regulate? Who is talking about regulating? I’m talking about teaching and the way we grade students. As a teacher myself I know that a lot of students (and people in the workforce) use AI. AI itself is just a tool. You can set it to full auto but that only leads to you being not needed anymore. What I teach my students is to use it responsible and that the AI is being used so it adds value to you. And not let it replace you.
In my opinion the AI is better at some tasks than humans are. Some tasks the human is better. Make yourself useful in the spot that AI is underperforming.
Edit: like some other comment was saying, “you can’t chatgpt through a whole semester”. And if you could, you are on the road of being replaced. If you can’t, congratulations. You learn something the machine can’t do.
I’m talking about teaching and the way we grade students.
Who is talking about regulating?
clearly, you are. I just don’t think you understand how it works yet. you must not be a teacher in the US.
you can’t just go and make changes to how you grade/teach your students in the midst of session. if you wait for a new year, your grading criteria has to go through review and approval.
come the fuck on.
I seriously doubt you are a teacher, and if you were I doubt a PE teacher has much to say about AI worth hearing.
I’m not a US teacher but Europe. University. A lot of teachers work in the field part time and part time teacher. We need to have a regulated outcome for our students. How we teach is up to the teacher. If I see that students can just get an A by using AI, then I’m not doing my job. So yeah, I’m not only expected to change how I teach, I’m required.
European university is a huge difference from US general education. the department of education is a staunch supporter of bureaucracy and hierarchy. as a teacher, if you aren’t doing “approved” things in “approved” ways in your classroom, you’ll be fired and stripped of your teaching license, students be damned.
this is why most US teachers are burnt out and just leave the career entirely.
the USDOE is far from functional, but it’s still better to have something (even the turd it is now) than nothing.
I did not know this. That really sucks.
