Challenges of Using Artificial Intelligence in the Academic Analysis of Social Science Texts

Živa Kos and Jasna Mažgon

Summary:  The increasing use of artificial intelligence (AI) in education, in particular large-scale language models (LLMs) such as ChatGPT and Microsoft Copilot, offers both challenges and opportunities. These LLMs can perform tasks that require natural language processing, from answering questions to solving problems. The integration of LLMs in education raises a number of dilemmas regarding the credibility and reliability of the information, the role of the teacher, the nature of learning, etc., with the need for a guided use of LLMs in educational processes being at the forefront. This paper focuses on the use of LLMs in the analysis of social studies and humanities texts. In a pilot study, we compared the epistemological and methodological approaches of LLMs with traditional, stand-alone literature analysis (deep reading) and compared the differences in results. The study, conducted with 17 first-year MA students in Sociology, reveals that while LLMs can optimise the creation of summaries and the provision of background information, they also call into question the depth and impartiality of the content. In doing so, we have explored how a hybrid approach, combining AI tools with traditional methods of reading and in-depth analysis, offers a promising way to improve learning and teaching strategies and can enhance critical analytical skills that are crucial for didactics in the digital age.