Mechanistic Interpretability of LLMs
Research
Hello, I’m Favour Umejesi, a third-year Computer Science student at Grambling State University with a minor in Mathematics.
My current research focuses on the mechanistic interpretability of large language models, where I investigate how internal model components contribute to factual recall, hallucinations, and biased outputs. Under the supervision of my Deep Learning professor, Dr. Vivya Kalindindi, I am exploring techniques such as layer ablation and game-theoretic methods to better understand how information is represented and processed within LLMs.
Research
Conducting ongoing undergraduate research under the guidance of Prof. Vivya Kalidindi at Grambling State University, investigating how information is represented and processed across layers of transformer-based language models. Using GPT-2, I conduct layer-wise zero-ablation experiments on feed-forward network (FFN) components to measure how individual layers affect model predictions and help localize factual knowledge. I also use TransformerLens and PyTorch to analyze logit attribution across the residual stream, identifying potential “source” and “inhibitor” layers associated with model predictions and latent hallucinations. This work explores how understanding layer-level behavior can provide greater insight into the internal mechanisms behind factual recall, hallucinations, and model confidence.
Project
EcoTrack is an open-source climate platform that helps users calculate and visualize their carbon footprint, track progress over time, and share sustainable actions through a community feed. Beyond the application, EcoTrack serves as a research testbed for exploring retrieval-augmented generation (RAG), alternative retrieval approaches, and methods for improving semantic search and information retrieval. Its Climate AI Assistant applies these techniques to answer climate questions from authoritative research with grounded, source-cited responses.
Research
As part of a research team, investigated machine learning and transfer learning approaches for ship vs. iceberg classification using Synthetic Aperture Radar (SAR) imagery. Evaluated a baseline MLP classifier trained on 1,604 labeled SAR images against models using pretrained ViT and ConvNeXT embeddings, analyzing representation quality through PCA and t-SNE. The baseline achieved 96% validation accuracy and 0.9855 ROC-AUC, outperforming the embedding-based approaches, while ConvNeXT produced stronger class separation among the pretrained representations. Our findings highlighted the limitations of transferring representations learned from natural images to specialized SAR imagery with limited labeled data.
Project
Collaborated with a team to develop a data-driven MVP for Rescuing Leftover Cuisine, addressing inefficiencies in matching surplus food donations with recipient organizations. We designed an intelligent matching system that applied natural language processing to food descriptions to categorize donations and improve donor-recipient matching, alongside geospatial routing to identify nearby recipients and reduce inefficient cross-ZIP travel. The broader solution incorporated simplified donor onboarding and verification, automated food matching, and proximity-based routing to help increase food rescue opportunities while reducing transportation inefficiencies and food waste.
Project
As part of a five-person team, developed an end-to-end Named Entity Recognition (NER) pipeline for extracting structured attributes such as make, model, year, and part type from 2M+ unstructured German automotive listings. Our solution combined XLM-RoBERTa and GELECTRA in a transformer ensemble, using weighted cross-entropy to address class imbalance and temperature scaling to calibrate model confidence. We optimized the pipeline for batched, large-scale inference, achieving a 0.933 F1 score and 0.955 accuracy and placing in the Top 10% of the eBay 2025 University Machine Learning Challenge.
Project
Collaborated with a team to analyze community data for The Concilio, a Dallas nonprofit supporting Latino and other diverse families. We developed an unsupervised K-Means clustering model to identify geographic areas with high community needs and segment households within each ZIP code into three levels of need based on demographic and socioeconomic factors such as age, household size, income, insurance status, and benefits. We integrated the model into an interactive web application for classifying new household data, helping The Concilio visualize community needs and make more informed decisions about resource allocation.
Project
Led a four-person team investigating the use of biosignal data to predict smoking status through binary classification. As team captain, coordinated the project while contributing to feature engineering, model development, and evaluation. We compared multiple machine learning approaches and optimized a Random Forest classifier that achieved 79% accuracy and a 0.87 AUC-ROC, approximately a 10% improvement over the Logistic Regression baseline. Our solution placed 4th among 10 competing teams, demonstrating the effectiveness of feature engineering and nonlinear models for classification on physiological data.