One of the most prestigious functions of literary fiction is to criticize contemporary social affairs (see Jannidis et al. 2009, 26). Many writers publicly contend to take a critical stance on social developments by means of their fictions and lay people as well as literary critics regularly claim that certain novels or other fictional texts are “socially critical”, “critical of contemporary affairs”, or “critical of culture”. However, there is neither a widely acknowledged technical term for this kind of literature – “littérature engage” is bound to a specific aesthetic/philosophical tradition (see Hucke/Kutzmutz 2003; Ilić 2017; Wegmann 1996) – nor a corresponding field of research. Against this background, our contribution aims at identifying on the one hand ascriptions of such “fictional critique” by literary critics and on the other hand textual correlates of such “fictional critique” that are meaningful to literary scholars. The main hypothesis is that “literary evaluations” (Hunt/Vipond 1986; Winko 1991; Prinz/Winko 2013), i.e. evaluations of characters, spaces, time periods etc. that occur in fictional works of literature (e.g., "the courageous savior", "a wonderful road", but also more implicit forms) play a crucial role for the expression of fictional critique.
To analyze ascriptions of fictional critique (e.g., “the socially critical novel X”) we extract paragraphs that contain relevant compounds with "critique" from a German-language corpus of literary studies reference works and literary histories (> 100 books, 2005-2023) using regular expression. These paragraphs are annotated for ascriptions of fictional critique (2 annotators, Cohen’s κ = 0,69), including a gold standard creation. In a second step, we use the resulting metadata (the literary works the secondary literature is referencing, Gittel 2025) to build a digital corpus of works of fictional critique and annotate a subset of it for literary evaluations (including evaluation target, evaluated aspect – “courageous savior” for example being a moral evaluation –, and linguistic trigger) using the platform INCEpTION (de Castilho 2018). According to the annotation guidelines, minimal spans are NP’s and clauses.
Since sentiment analysis is often used for large-scale analysis of “evaluations” (Liu 2017, 12) in the non-literary domain (e.g., product reviews) and even literary researchers sometimes describe its merit in terms of analyzing “evaluation[s]” (Grisot/Herrmann 2023, 5), we hypothesize that it can be used in the literary domain to detect literary evaluations (Hunt/Vipond 1986; Winko 1991; Prinz/Winko 2013). Although a growing body of sentiment analysis studies has determined sentiment scores for whole literary texts or calculated affective variation throughout a narrative (Kim et al. 2017; Reagan et al. 2016; Jacobs 2019; Mohammad 2012; Klinger et al. 2016), it is not clear whether sentiment analysis is appropriate to measure literary evaluations. Thus, we evaluate which of the most popular sentiment analysis tools for German – like SentiWS (Remus et al. 2010), SentiMerge (Emerson/Declerck 2014) or German-Sentiment-BERT (Guhr 2020) – works best using the gold standard annotation of literary evaluations as ground truth (6 finalized texts of an ongoing annotation with 2 annotators). Text passages both annotated as evaluative and not annotated as such are run through sentiment analysis to check whether a matching (negative / positive / neutral) sentiment score is assigned.
In a third step, we use the best performing model to compare the fictional critique corpus and corresponding subsets of this corpus for social critique, cultural critique (germ. “Kulturkritik”), contemporary critique (germ. “Zeitkritik”) and others to the German ELTeC Reference Corpus (ELTeC 2020, 100 texts) regarding frequency and distribution patterns of literary evaluations. If the main hypothesis, that literary evaluations play a crucial role for the expression of fictional critique, is true, the fictional critique corpus / its subsets should show characteristic patterns in comparison to the ELTeC corpus. Among others, we test the hypotheses, that i) fictional critique (or one of its subtypes) contains on average more negative evaluations than non-critique fiction, and ii) that fictional critique (or one of its subtypes) contains stronger evaluative contrasts between positively and negatively evaluated entities than non-critique fiction. To the second end, we average sentiment scores in the surrounding text of mentions of named entities and calculate the difference between the means of positively and negatively evaluated entities of one text.