ChatGPT passes graduate-level law and business exams: Check its grades
What's the story
How would ChatGPT perform in a law school or business school? Will it ace the tests or will it be an average student? We now have the answers to these questions. Per studies conducted at the University of Minnesota's School of Law and the University of Pennsylvania's Wharton Business School, OpenAI's chatbot passed graduate-level law and business exams. Spoiler alert: Its performance was average.
Context
Why does this story matter?
There are concerns about ChatGPT being used by students to cheat in exams and assignments. The reason for concern is the chatbot's ability to generate original content. As great as it is, the chatbot's abilities have given rise to some never-heard-before ethical conundrums. Public schools in New York City and Seattle have already banned the usage of ChatGPT.
Law school
The chatbot passed law exams with a C+ grade
The law professors at the University of Minnesota subjected ChatGPT to exams in four courses. The AI answered 95 multiple-choice questions and 12 essay questions in the tests. It passed all four exams but performed at the level of an average C+ student. The professors noted that the bot was good at answering questions about basic legal rules and summarizing legal doctrines.
Issue
ChatGPT couldn't pinpoint relevant issues to a case
ChatGPT's major issue in answering legal questions was that it failed to pinpoint relevant issues to a case. It also failed to conduct a deep analysis by applying legal rules to the facts of a case. According to Jonathan Choi, one of the University of Minnesota law professors, ChatGPT could be helpful at "producing a first draft that a student could then refine."
Business school
The chatbot performed better at the business school
OpenAI's chatbot performed much better during a business management course at Wharton. Per the study conducted by Christian Terweisch, a Wharton professor, ChatGPT earned a B to B- grade. The researcher said that the AI performed well at answering "basic operations management and process-analysis questions." However, it struggled when it came to more advanced prompts. It also made "surprising mistakes" with 6th-grade math.
Concerns
ChatGPT could have a place in the classroom
Both camps expressed their concerns about ChatGPT. Terweisch said that there should be restrictions on the usage of the chatbot considering how well it performed in his tests. However, both study groups believe that ChatGPT has a place in classrooms. It could help professors save time that could be spent on students. Professors will be also able to use the tool to grade exams.