Social scientists have been manually coding (labeling text segments to capture their essence, and clustering these labels into groups) large textual corpora for decades. Time that could have been spent doing research, annotating fire hydrants for Google Streetview, or spending time with loved ones.

For this reason and more, in our latest paper we turn to LLMs for automated axial coding of lengthy transcripts (political debates). We extend an ensemble-based open coding pipeline with two axial coding (grouping) strategies: “traditional” clustering with subsequent LLM labeling, and direct LLM-based grouping.

We find a clear trade-off: traditional clustering methods achieve high coverage and structural separation, and direct LLM grouping produces more concise, interpretable labels that are more similar to human-assign group labels, but with much lower coverage. Traditional clustering ensures broad representation; LLMs supply the interpretive layer that makes categories human-readable.

Get the preprint here:

• A. Parfenova, D. Graus, and J. Pfeffer, “From quotes to concepts: axial coding of political debates with ensemble lms,” in European conference on information retrieval (ecir), Delft, The Netherlands, 2026.
  [Bibtex]

  @inproceedings{parfenova2026quotes,
  title = {From Quotes to Concepts: Axial Coding of Political Debates with Ensemble LMs},
  author = {Parfenova, Angelina and Graus, David and Pfeffer, Juergen},
  booktitle = {European Conference on Information Retrieval (ECIR)},
  year = {2026},
  address = {Delft, The Netherlands},
  note = {To appear}
  }

This was work led by Angelina Parfenova. Our full dataset of 5k Dutch parliamentary debate utterances with LLM-assigned codes and categories are publicly available here: https://github.com/Likich/axial_coding_dataset.

Super happy with the acceptance of our ECIR 2026 Findings paper: “From Quotes to Concepts: Axial Coding of Political Debates with Ensemble LMs“

In this work we apply fine-tuned LLMs for qualitative data analysis (axial coding) of Dutch political debates. This is work led by Angelina Parfenova (Technical University of Munich), who I met at last year’s ECIR in Lucca 🇮🇹. Love it when conference chats turn into papers!

From Quotes to Concepts: Axial Coding of Political Debates with Ensemble LMs

Abstract. We introduce the first method to perform axial coding using large language models (LLMs), transforming raw debate transcripts into concise, hierarchical categories. Axial coding is a qualitative analysis technique that organizes codes (labels) representing text into broader categories, enhancing document understanding and analysis. Starting with sentence-level labels (open codes) generated by an LLM ensemble with a moderator, we introduce an axial coding step that groups these codes into higher-order categories. We compare two strategies: (i) clustering (code + utterance) embeddings using density-based and partitioning algorithms followed by LLM labeling, and (ii) direct LLM-based grouping of codes and utterances into categories. We apply our method to Dutch parliamentary debates, converting lengthy transcripts into compact, hierarchically structured codes and categories. We evaluate our method using extrinsic metrics aligned with human-assigned topic labels (ROUGE-L, cosine, BERTScore), and intrinsic metrics describing code groups (coverage, brevity, coherence, novelty, JSD divergence). Our results reveal a trade-off: density-based clustering achieves high coverage and strong cluster alignment, while direct LLM grouping results in higher fine-grained alignment, but lower coverage (∼20%). Overall, clustering maximizes coverage and structural separation, whereas LLM grouping produces more concise, interpretable, and semantically aligned categories. To support future research, we publicly release the full dataset of utterances and codes, enabling reproducibility and comparative studies.

This is also my first full (and OpenGov-themed) co-authored paper since rejoining academia and running the OpenGov Lab!

Stay tuned for the preprint.