The agreement results between different pairs of human annotators and the LLM annotator indicate that misalignment occurs primarily in interpreting user intent categories.
The iterative approach to lexicon development, combining manual reviews and semi-automatic methods, significantly enhances the identification of suicidal ideation from clinical notes.