Keunyoung Kim

Ph.D. Student in Artificial Intelligence
Seoul National UniversityAdvisor: Prof. Nojun Kwak

Hi, I’m Keunyoung Kim, a Ph.D. student in Artificial Intelligence at Seoul National University. My research focuses on improving the efficiency of large language model (LLM) training under memory or time constraints. My long-term goal is to develop broadly applicable training methods that extend beyond LLMs to a wider range of AI models and application domains. I studied Systems Engineering and Military Science at the Republic of Korea Air Force Academy and earned my master’s degree in Artificial Intelligence at Seoul National University.

Email CV LinkedIn Scholar

Research

Featured work

EMNLP 2026Main Conference

MoARa: Module-Aware Rank Allocation and Structure-Preserving Decomposition for Low-Rank LLM Pre-training

Keunyoung Kim, Nojun Kwak

Transformer modules respond differently to low-rank projection. MoARa assigns projection ranks according to module sensitivity and preserves local gradient magnitudes to improve training convergence.

01Allocate rank by module

Reallocate rank according to module sensitivity A schematic example moves rank budget from Query and Key projections to the Down projection. Matrix widths are illustrative, not measured ranks. Projection rank budget reallocation Q / K Other modules Down

Give more rank to modules that are more sensitive to projection.

02Preserve local magnitudes

Preserve scale while projecting direction Separate each gradient block into its magnitude and direction. Retain the magnitude, apply low-rank projection to the normalized gradient, then recombine them. Gradient Local magnitude retain Direction Low-rank projection recombine

Keep local scale information through low-rank gradient projection.

In our Llama 2 7B experiment, MoARa reaches the GaLore baseline’s final perplexity with about 34% less wall-clock training time.

Paper Poster

Contact