Μεταπηδήστε στο περιεχόμενο

Τμήμα Χρηματοοικονομικής και Τραπεζικής Διοικητικής

Academic Seminar Series

LLMs and Systematic Measurement Error

William Grieser
Oklahoma State University
28/05/2026

–  Time:

16:15-17:30

  – 

Online

Abstract

Researchers increasingly construct text-based empirical measures with Large Language Models, often treating model outputs as observed data rather than generated regressors. Using S&P 500 Item 1A risk disclosures from 2005–2024, we show that LLM-derived measures contain non-classical measurement error driven by two distortions: compression and injection. Compression is substantial: summaries retain only 40–63% of source vocabulary and selectively omit source-text language. Injection is also pervasive and not merely editorial: summaries introduce novel terms absent from the source that account for 16–42% of their most discriminative vocabulary, often changing salience, tone, and substantive content. Together, compression and injection directionally reshape narrative emphasis and predict outputs from downstream LLM tasks, including categorization, scoring, and ordinal classification. These distortions predict severity, specificity, complexity, and boilerplate measures, and generate firm-outcome associations consistent with non-classical measurement error. We document these patterns across models and textual constructs and propose diagnostics for structured LLM measurement error.

William Grieser is an Associate Professor of Finance and the Paul C. Wise Chair for Excellence in Finance at Oklahoma State University’s Spears School of Business, where he also coordinates the Finance PhD program. He studies how firms’ decisions ripple through production and geographic networks, with research spanning network economics, corporate investment, financial constraints, commodity markets, corporate risk management, and most recently the measurement properties of Large Language Models. His research has been published in journals like Journal of Financial Economics, Journal of Financial and Quantitative Analysis, Review of Financial Studies and Management Science. For more information please visit his website https://billygrieser.com/.

This seminar will be online on Teams. Click here to attend

Το Πανεπιστήμιο

Πληροφορίες για το Πανεπιστήμιο Πειραιώς.