Title

SEMANTIC MATCHING FOR REVIEWER ASSIGNMENT: AN ABLATION STUDY OF DENSE RETRIEVAL AND LLM REASONING

Abstract

Abstract

Assigning suitable academic reviewers to research projects is a complex task that requires understanding both the semantic content of project descriptions and the diverse research backgrounds of academicians. Manual reviewer assignment or simple keyword-based methods often fail to capture these relationships. This thesis proposes a semantic, data-driven framework that integrates multi-encoder dense retrieval with large language model (LLM) based reasoning to improve project–academician matching. Academic profiles consisting of publications, supervised theses, research keywords, and project descriptions are first encoded using several pretrained embedding models, including BGE-M3, SPECTER2, MiniLM, and UAE. Cosine similarity is computed to retrieve the top candidate reviewers, and encoder performances are compared to identify which models and textual components best capture semantic expertise. The top-ranked candidates are then provided to an LLM such as ChatGPT, Gemini, or Claude for reasoning-based assessment, allowing the model to refine reviewer suitability using semantic evidence and contextual signals. An ablation study systematically varies textual features, encoder selections, and LLM selections to assess their impact on matching performance. The proposed method aims to enhance reviewer assignment accuracy and interpretability by combining quantitative semantic similarity with qualitative reasoning.

Supervisor(s)

Supervisor(s)

IREM DERELI

Date and Location

Date and Location

2026-01-22 10:00:00

Category

Category

MSc_Thesis