The Generative AI Learning Penalty: Evidence from Chinese Secondary Education

Using 30 months of panel data on 26,811 Chinese students in grades 7-12, we study how generative AI affects homework productivity and learning. The data combine monthly closed-book exams, high-school and college entrance exams, and homework scores and completion time across nine subjects. We exploit staggered AI adoption in a difference-in-differences design. AI adoption raises homework scores by 18% and reduces completion time by 30%, but lowers monthly exam scores by 20% within six months. High-stakes entrance-exam scores fall by 18 and 24%, with the full penalty emerging only after about two years. The losses are largest in social science subjects, followed by STEM and languages, and are especially large for junior students, high-achieving students, and boys. The learning losses are concentrated among roughly 80% of AI users whose behavior is consistent with homework outsourcing, as indicated by exceptionally short homework completion time coupled with high homework scores. AI users who maintain similar homework completion time as non-AI users experience small learning losses.

Edit: moving my comment up here

Just in case as it’s formatted a bit weirdly

X-Axis: Homework scores

Y-Axis: Exam scores

  • mnemonicmonkeys@sh.itjust.worksEnglish
    26·
    3 days ago

    This is actually a very good chart to explain outcomes. It shows that there’s a positive linear relationship between homework and exam scores for students that don’t use ai, and a terrible relationship for students that do use ai

            • Viceversa@lemmy.world
              2·
              2 days ago

              Oh god.

              Y label is bold
              has the prefix “China*,”
              has additional row of text between the label and the diagram.

              X label is regular
              doesn’t have the prefix “China*,”
              but has the suffix “, average without Al=100”

              Complete disaster of a diagram.
              Was it made by AI, perchance?

                • Viceversa@lemmy.world
                  1·
                  2 days ago

                  The Economist makes great charts because they keep things simple and effective. They aim to ensure that anyone looking at their charts can quickly understand what’s being shown, whether it’s numbers or trends.

                  LOL

                  @[email protected] , but why do you think the diagram was made by The Economist though?

                  • Knock_Knock_Lemmy_In@lemmy.world
                    1·
                    1 day ago

                    The graph is not from authors because the paper graphs are of a different style. So the bad graph creators needed access to the raw data. This implies it’s not just any random person.

                    I went for the Economist because the fonts and style look very similar.

                    chart.

                    I don’t subscribe, but I bet there is an online article.