生成式人工智能科普写作能力评估 ——基于“微生物”主题的科普创作

Evaluation of Generative AI"s Popular Science Writing Ability:

中文关键词: 生成式人工智能  科普写作? 科学传播  人工智能创作能力

英文关键词: Generative artificial intelligence  Popular science writing  Science communication  AI creation ability

中文摘要:

      生成式人工智能已经成为科普作品创作的重要工具,但与人类创作者相比,其科普写作能力仍然受到质疑。为了评估生成式人工智能的科普写作能力,构建了易读性、趣味性、科学性和传播效果的四维度评价指标,以“微生物”为主题,分别考察了人类创作者、ChatGPT与文心一言创作的科普作品。结果显示,ChatGPT初步提示与深度提示版本在整体得分上与人类创作版本无显著差异,ChatGPT深度提示版本在“易读性”指标上显著优于人类创作版本,文心一言初步提示版本在整体均分、趣味性和科学性上均表现最差；评价者对所有生成式人工智能创作作品的甄别正确率均不足55%,且评价者更倾向于将高分作品认为是人类创作。实验结果表明,生成式人工智能具备替代人类科普创作者的潜力,进而提出了“人机合作科普创作”这一科普创作新模式,并呼吁学界关注“人类能力幻觉”现象。

英文摘要:

      Generative artificial intelligence has emerged as a crucial tool in creating science popularization works, yet its proficiency in this field remains questionable compared to that of human creators. To assess the science popularization writing capabilities of generative artificial intelligence, a four-dimensional evaluation index encompassing readability, interest, scientific rigor, and communication effectiveness is established. Focusing on the theme of ‘microorganisms’, we examine science popularization works crafted by human authors, ChatGPT and ERNIE Bot. The findings reveal that there is no significant disparity in the overall scores between the preliminary prompt version of ChatGPT and the deep prompt version, both of which align closely with the human-created version. Notably, the deep prompt version of ChatGPT excels in readability, surpassing the human-created version. Conversely, the ERNIE Bot preliminary prompt version performs poorly in terms of overall average score, interest, and scientific rigor. Evaluators have an accuracy rate of less than 55% in discerning the creations of generative artificial intelligence, with a tendency to attribute higher scores to these works as if they were human-crafted. These experimental outcomes suggest that generative artificial intelligence has the potential to supplant human science popularization creators. Consequently, we propose a novel model for science popularization creation termed ‘human-machine collaborative science popularization’, and urge academia to address the phenomenon of ‘Hallucination of Human’s Ability’.