Gyuwan Kim

Gyuwan Kim

Ph.D. Candidate in Computer Science

University of California, Santa Barbara

Hi, I am a fifth-year Ph.D. candidate in Computer Science at UCSB. I am advised by Prof. Tao Yang and co-advised by Prof. Xifeng Yan. I previously worked with Prof. William Yang Wang, who also serves on my Ph.D. committee. I am affiliated with UCSB Search Systems Lab and UCSB NLP group. During my Ph.D., I did summer internships at Meta GenAI, AWS AI Labs, Apple MLR, and Microsoft Research. Before joining UCSB, I worked as a research scientist at NAVER and received my B.S. and M.S. from Seoul National University.

My research mainly focuses on improving the efficiency and reliability of language models through advanced algorithms for adaptive computation ( Length-Adaptive Transformer and Dynamic-TinyBERT), retrieval augmentation ( AcuRank, MacRAG, DensePhrases-UL-HN, and PKM-augmented PLMs), generation ( KnapSpec, Subword LM QAC, and SOM-DST), and data augmentation ( SSMix and VAT-D). Besides, I also did research on LLM reasoning/evaluation ( PPA-Plan, EM-MIA, ScholarBench, and Korean GEC) and multi-modal learning ( Textual KD SLU and ST-BERT). Recent interests include but are not limited to efficient LLM inference techniques (e.g., KV cache compression, speculative decoding, early-exit), retrievals (e.g., retrieval-augmented generation, multi-vector retrieval, reranking), and long-context modeling.

News:

  • [Apr 2026] 📄 PPA-Plan was accepted at ACL 2026.
  • [Feb 2026] 📄 KnapSpec was posted on arXiv.
  • [Jan 2026] 📄 EM-MIA was accepted at EACL 2026.
  • [Dec 2025] Attended NeurIPS 2025 in San Diego. 🌴
  • [Sep 2025] Finished a research internship at Meta GenAI.
  • [Sep 2025] 📄 AcuRank was accepted at NeurIPS 2025.
  • [Sep 2025] Passed Ph.D. qualifying exam and advanced to candidacy. ✨
  • [Aug 2025] 📄 ScholarBench was accepted at Findings of EMNLP 2025.
  • [May 2025] 📄 MacRAG was posted on arXiv.
Load more

Interests

  • Efficiency
  • Retrievals
  • Large Language Models
  • Long-context Modeling

Education

  • Ph.D. in Computer Science, Sep 2021 - Present

    University of California, Santa Barbara

  • M.S. in Electrical and Computer Engineering, Mar 2015 - Aug 2017

    Seoul National University

  • B.S. in Electrical and Computer Engineering / Mathematical Sciences (double major), Mar 2010 - Aug 2014

    Seoul National University

  • High School (early graduation), Mar 2008 - Feb 2010

    Seoul Science High School

Experience

 
 
 
 
 

Research Intern

Meta GenAI

Jun 2025 – Sep 2025 Menlo Park, CA, United States

Mentor(s):

Worked on:

  • LLM Post-training for Response Diversification
 
 
 
 
 

Applied Scientist Intern

AWS AI Labs

Jun 2024 – Sep 2024 Seattle, WA, United States

Mentors:

Worked on:

  • Data Contamination Detection for LLMs
 
 
 
 
 

Research Intern

Apple MLR

Jun 2023 – Sep 2023 Cupertino, CA, United States

Mentors:

Worked on:

  • Efficient Long-Context Modeling with Retrieval-Augmented Language Models
 
 
 
 
 

Research Intern

Microsoft Research

Jun 2022 – Sep 2022 Redmond, WA, United States (Remote)

Mentors:

Worked on:

  • Efficient Long-Document Summarization
 
 
 
 
 

Ph.D. Candidate

University of California, Santa Barbara

Sep 2021 – Present Santa Barbara, CA, United States

Working/Worked on:

  • Efficient Retrievals
  • Retrieval-augmented Generation
  • Efficient LLM Inference
  • Adaptive Computation for LLMs
  • Long-Context Modeling
  • Membership Inference Attacks for LLMs
  • Attribution of Language Models
 
 
 
 
 

Research Scientist

NAVER Clova & AI LAB

Sep 2017 – Aug 2021 Seongnam, Republic of Korea

Worked on:

  • Fundamental Research for LLMs

  • Language Representation by Clova (LaRva)

    • Pre-trained Language Models for Korean/Japanese
    • Korean Question Answering ( KorQuAD)
    • Knowledge Distillation of BERT
    • Memory-Augmented Language Models
    • Efficient Transformer Inference
    • Data Augmentation for NLP Models

  • Context Center AI ( CCAI)

    • Natural Language Understanding (Intent Classification & Slot Filling)
    • Automatic Speech Recognition Error Correction
    • End-to-End Spoken Language Understanding
    • Efficient Dialog State Tracking

  • Korean Grammatical Error Correction

  • Language Model based Query Auto-Completion

  • LINE Sticker Reply Recommendation

  • Community Question Answering based on Query Similarity

 
 
 
 
 

Data Scientist

Devsisters

May 2017 – Jul 2017 Seoul, Republic of Korea

Worked on:

  • User Action Modeling for Churn Prediction
  • Customer Service Automation
 
 
 
 
 

Master’s Student

Seoul National University

Mar 2015 – Aug 2017 Seoul, Republic of Korea

Worked on:

  • Language Model based Intrusion Detection System
  • RNA/Protein Secondary Structure Prediction
 
 
 
 
 

Research Intern

Seoul National University

Dec 2012 – Aug 2014 Seoul, Republic of Korea

Worked on:

  • Biological Sequence Classification
  • Parallel Programming for Biological Sequence Alignment

Publications

$*$ denotes eqaul contribution

KnapSpec: Self-Speculative Decoding via Adaptive Layer Selection as a Knapsack Problem

A self-speculative decoding method with training-free, knapsack-based adaptive layer selection that accounts for context-dependent attention overhead
Paper
KnapSpec: Self-Speculative Decoding via Adaptive Layer Selection as a Knapsack Problem

PPA-Plan: Proactive Pitfall Avoidance for Reliable Planning in Long-Context LLM Reasoning

A proactive planning strategy for long-context reasoning that focuses on preventing logical pitfalls and false assumptions before plan generation - ACL 2026
Paper
PPA-Plan: Proactive Pitfall Avoidance for Reliable Planning in Long-Context LLM Reasoning

Detecting Training Data of Large Language Models via Expectation Maximization

Novel membership inference attack framework for large language models via an expectation maximization algorithm - EACL 2026
Paper Code Poster Slides Video
Detecting Training Data of Large Language Models via Expectation Maximization

AcuRank: Uncertainty-Aware Adaptive Computation for Listwise Reranking

Listwise reranking with uncertainty-aware adaptive computation via Bayesian modeling of documents’ relevance with TrueSkill - NeurIPS 2025
Paper Code Poster Video
AcuRank: Uncertainty-Aware Adaptive Computation for Listwise Reranking

ScholarBench: A Bilingual Benchmark for Abstraction, Comprehension, and Reasoning Evaluation in Academic Contexts

A bilingual benchmark for abstraction, comprehension, and reasoning evaluation in academic contexts - Findings of EMNLP 2025
Paper Code Dataset Video
ScholarBench: A Bilingual Benchmark for Abstraction, Comprehension, and Reasoning Evaluation in Academic Contexts

MacRAG: Compress, Slice, and Scale-up for Multi-Scale Adaptive Context RAG

Multi-scale adaptive context RAG for long-context large language models
Paper Code
MacRAG: Compress, Slice, and Scale-up for Multi-Scale Adaptive Context RAG

Towards Standardizing Korean Grammatical Error Correction: Datasets and Annotation

Collection of three datasets and developing a new automatic error-type annotation tool for Korean grammatical error correction - ACL 2023
Paper Code Video
Towards Standardizing Korean Grammatical Error Correction: Datasets and Annotation

Bridging the Training-Inference Gap for Dense Phrase Retrieval

Improving dense phrase retrieval with unified loss and hard negatives based on efficient validation using subcorpus - Findings of EMNLP 2022
Paper Poster Slides Video
Bridging the Training-Inference Gap for Dense Phrase Retrieval

Consistency Training with Virtual Adversarial Discrete Perturbation

Virtual adversarial training with discrete token perturbation for text classification - NAACL 2022
Paper Code
Consistency Training with Virtual Adversarial Discrete Perturbation

Dynamic-TinyBERT: Boost TinyBERT's Inference Efficiency by Dynamic Sequence Length

Dynamic sequence length reduction for a TinyBERT model - ENLSP Workshop @ NeurIPS 2021
Paper Code Model Poster
Dynamic-TinyBERT: Boost TinyBERT's Inference Efficiency by Dynamic Sequence Length

NASCUP: Nucleic Acid Sequence Classification by Universal Probability

Classification method for nucleotide sequences using compact context-tree models and universal probability from information theory - IEEE Access 2021
Paper Code Dataset
NASCUP: Nucleic Acid Sequence Classification by Universal Probability

SSMix: Saliency-based Span Mixup for Text Classification

Token-level mixup approach based on saliency information - Findings of ACL 2021
Paper Code
SSMix: Saliency-based Span Mixup for Text Classification

Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Search

Framework of training any transformer with length drop and using it for anytime prediction with a multi-objective evolutionary search - ACL 2021
Paper Code Slides Video
Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Search

Two-stage Textual Knowledge Distillation for End-to-End Spoken Language Understanding

Knowledge distillation from text BERT to speech model by matching sequence-level contextualized representations in pretraining and predicted logits in finetuning - ICASSP 2021
Paper Code Poster
Two-stage Textual Knowledge Distillation for End-to-End Spoken Language Understanding

ST-BERT: Cross-modal Language Model Pre-training For End-to-end Spoken Language Understanding

Speech-text cross-modal pretraining with cross-modal masked language modeling (CM-MLM) and cross-modal conditioned language modeling (CM-CLM) - ICASSP 2021
Paper Poster
ST-BERT: Cross-modal Language Model Pre-training For End-to-end Spoken Language Understanding

AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights

Projecting out the radial component to mitigate the decay of effective step sizes for scale-invariant weights when updating with momentum-based optimizers - ICLR 2021
Paper Code Slides Video
AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights

Large Product Key Memory for Pretrained Language Models

Improving accuracy and speed trade-off when finetuning pretrained language models by using large product key memory and mitigating a catastrophic drift with initialization and residual memory - Findings of EMNLP 2020
Paper Code Video
Large Product Key Memory for Pretrained Language Models

Efficient Dialogue State Tracking by Selectively Overwriting Memory

Decomposition of open-vocabulary dialog state tracking to state operation prediction and slot value generation, achieving excellent joint goal accuracy with very efficient computation - ACL 2020
Paper Code Video
Efficient Dialogue State Tracking by Selectively Overwriting Memory

Subword Language Model for Query Auto-Completion

Utilization of subword language model for faster query auto-completion with a retrace algorithm and a reranking method by approximate marginalization - EMNLP-IJCNLP 2019
Paper Code
Subword Language Model for Query Auto-Completion

Mimicry Resilient Program Behavior Modeling with LSTM based Branch Models

Anomaly detection robust to mimicry attacks with language modeling of branch sequences - S&P 2018 DLS Workshop
Paper
Mimicry Resilient Program Behavior Modeling with LSTM based Branch Models

Training IBM Watson using Automatically Generated Question-Answer Pairs

Examination on IBM Watson training with manually labeled and automatically generated question-answer pairs - HICSS 2017 (IBM Best Technology Paper Honorarium)
Paper
Training IBM Watson using Automatically Generated Question-Answer Pairs

LSTM-Based System-Call Language Modelling and Robust Ensemble Method for Designing Host-Based Intrusion Detection System

System-call language-modeling approach for designing anomaly-based host intrusion detection systems and novel ensemble method to enhance the precision
Paper
LSTM-Based System-Call Language Modelling and Robust Ensemble Method for Designing Host-Based Intrusion Detection System

Academic Activities

Organizing Committee

Program Committee/Reviewer

  • Conferences
    • ICLR 2021, 2022, 2023, 2024, 2025, 2026
    • NeurIPS 2016, 2021, 2022, 2023, 2024, 2025
    • ICML 2020, 2021, 2022, 2023
    • ACL 2020, 2023; EMNLP 2020, 2021, 2022, 2023; NAACL 2021; EACL 2023
    • ACL Rolling Review (ARR) 2021-
    • AAAI 2017, 2023, 2024; AISTATS 2025, 2026
    • COLM 2024, 2025, 2026
    • COLING 2020, 2022, 2024, 2025; LREC 2026
    • SIGIR 2026; CAIS 2026
  • Workshops
  • Journals
    • TMLR 2022-
    • NEJLT 2024

Volunteer

  • EACL 2026, EMNLP 2022, ICML 2022, NAACL 2022, ACL 2021, NAACL 2021, ICLR 2021

Institutional Service

Presentations

Teaching

  • Teaching Assistant, Machine Learning, UCSB, Spring 2026
  • Teaching Assistant, Parallel Computing, UCSB, Winter 2026
  • Teaching Assistant, Artificial Intelligence, UCSB, Spring 2025
  • Teaching Assistant, Generative AI, UCSB, Spring 2025
  • Teaching Assistant, Problem Solving with Computers I, UCSB, Winter 2025
  • Teaching Assistant, Problem Solving with Computers II, UCSB, Fall 2024
  • Teaching Assistant, Machine Learning, UCSB, Spring 2024
  • Teaching Assistant, Machine Learning, UCSB, Winter 2024
  • Teaching Assistant, Problem Solving with Computers I, UCSB, Fall 2023
  • Teaching Assistant, Machine Learning, UCSB, Winter 2023
  • Judges of UCSB’s 10th Annual Hackathon, 2023
  • Project Mentor, NAVER CLOVA AI RUSH, 2021
  • Project Mentor, NAVER AI RUSH, 2020
  • Teaching Assistant, Machine Learning, Seoul National University, Spring 2016
  • Tutor, Programming Methodology, Seoul National University, Spring 2014
  • Problem Setter, Korean Olympiad in Informatics (KOI), 2010 – 2014
  • Student Coach, Training Camp for International Olympiad in Informatics (IOI), 2010 – 2014

Research Mentor

  • Hun Tae Kim, M.S. Student at UCSB, Nov 2025 – Present
  • Alan Wang, Westlake High School, Jun 2022 – Jul 2022 (now Undergrad at Carnegie Mellon University)
  • Sandra Ravishankar, Mountain View High School, Jun 2022 – Jul 2022 (now Undergrad at Duke University)
  • Soyoung Yoon, Undergrad at KAIST, Jul 2020 – Jan 2021 (now Ph.D. student at SNU)
  • Jungsoo Park, M.S. Student at Korea University, Jul 2020 – Jan 2021 (now Ph.D. student at Georgia Tech)
  • Sungbin Kim, M.S. Student at Inha University, Feb 2020 – Feb 2021 (now at LG Uplus)
  • Tae-Hwan Jung, Undergrad at Kyung Hee University, Dec 2019 – Jun 2020
  • Bumju Kwak, Undergrad at Seoul National University, Apr 2019 – Aug 2019 (now at Karrot)
  • Kyungwoo Song, Ph.D. Student at KAIST, Oct 2018 – Dec 2018 (now Assistant Professor at Yonsei University)

Awards

KSEA-SC South-Western Regional Conference 2026

  • 2nd Place, Student Research Poster Competition

SustaiNLP 2021 Workshop

  • Best Paper Award

ACM International Collegiate Programming Contest (ACM-ICPC)

  • Asia Daejeon Regional: Gold Prize, 2013
  • Asia Daejeon Regional: Special Prize, 2012
  • Asia Daejeon Regional: Special Prize, 2011

Korean Collegiate Mathematical Competition

  • Mathematics Major Division: Silver Prize, 2013
  • Mathematics Major Division: Bronze Prize, 2012
  • Non-Mathematics Major Division: Gold Prize, 2011

Korea Olympiad in Informatics (KOI)

  • Gold Prize, 2008

Korean Mathematical Olympiad (KMO)

  • Silver Prize, 2008

Scholarship

Korean Computer Scientists and Engineers Association in America (KOCSEA)

  • Travel Support for KOCSEA Tech Symposium, 2025

Korean-American Scientists and Engineers Association (KSEA)

  • Travel Support for UKC FIRE Symposium, 2025

University of California, Santa Barbara

  • GSA Conference Travel Grant, 2025
  • GSA Conference Travel Grant, 2022
  • Academic Excellence Fellowship, 2021

Seoul National University

  • Graduate Student Scholarship, 2015 – 2016
  • Partial Scholarship, 2011 – 2014

Korean Foundation for Advanced Study (KFAS)

  • Undergraduate Scholarship, 2011 – 2013

Korea Student Aid Foundation (KOSAF)

  • National Science and Engineering Scholarship, 2012