Publications

View publication

Title Attitude Analysis in Systemic Functional Linguistics: A New Corpus and Benchmark Task
Authors Jorge Ortiz, Felipe Bravo-Marquez, Beatriz Quiroz
Publication date 2026
Abstract We introduce a corpus for Systemic Functional Linguistics
(SFL)
Appraisal analysis in Chilean Spanish, together with TULIO, a BERT model
adapted to Chilean Spanish through continual pre-training. The corpus
contains 2,546 texts annotated for the Attitude subsystem, with a focus on
negative Judgment, and supports two span-level sequence labeling tasks:
Attitude Types, which distinguishes Affect, Judgment, and Appreciation, and
Judgment Subtypes, which identifies five finer-grained forms of social
evaluation. To reduce the boundary ambiguity reported in prior Appraisal
annotation work, we use a two-stage protocol that first marks structural
spans and then assigns Attitude labels. Three expert annotators reach almost
perfect agreement at the general level (Krippendorff's alpha = 0.834) and
substantial agreement for Judgment Subtypes (alpha = 0.647). We benchmark
supervised sequence labelers and structured LLM agents. The highest observed
F1 scores are 0.554 on Attitude Types from a Flair BiLSTM-CRF with TULIO
embeddings and 0.410 on Judgment Subtypes from Qwen 3.6 Plus. Under
Twitter-only training, supervised performance is genre-sensitive, with
higher F1 on Twitter than on other genres. All model scores remain below
expert agreement.
Pages 28000-28023
Conference name Empirical Methods in Natural Language Processing
Publisher Association for Computational Linguistic
Reference URL View reference page