The article discusses the methodology and results of researching the fine-tuning of the GPT-3.5 model for identifying praise in tutor responses. Utilizing the M-IoU score as a metric, the study showed that, despite content restrictions on the GPT-4 model, the accessible version could still effectively assess tutor performance under low-resource conditions. With training sample sizes as low as 13, the GPT-3.5 demonstrated promising results in distinguishing between effort-based and outcome-based praise, underscoring the potential of AI in educational feedback mechanisms.
The fine-tuned GPT-3.5 model's performance was evaluated using M-IoU scores across multiple random seeds, demonstrating its efficacy in identifying praise in tutor responses with limited training data.
Despite limited resources, the GPT-3.5 model maintained adequate performance levels, achieving M-IoU scores of 0.5 and 0.65 for effort-based and outcome-based praise, respectively.
Collection
[
|
...
]