About Me

Note: My main page has been moved to zijun-yao.github.io.

I am a Ph.D. candidate in the Knowledge Engineering Group (KEG), Department of Computer Science and Technology, Tsinghua University (THU, Sep 2023 – Jun 2026, expected), advised by Prof. Juanzi Li. Before that, I finished my education as a master student in THU-KEG, also advised by Prof. Juanzi Li from Sep 2020 to Jun 2023. I received my bachelor’s degree in Computer Science and Technology from Beijing University of Posts and Telecommunications (BUPT, Sep 2016 – Jun 2020).

My research sits at the intersection of large language models (LLMs), knowledge engineering, and reasoning. I have worked as a research intern at Zhipu.AI on building and post-training LLMs, where I work closely with Dr. Xin Lv, and I was a visiting scholar at the NExT++ Research Centre, National University of Singapore (Mar 2025 – Sep 2025) hosted by Prof. Chua Tat-Seng. Earlier, I spent time in THU-KEG on graph learning, supervised by Prof. Jie Tang. I also worked with Prof. Bin Wu on evolutionary game theory on graph.

You can find my publications on Google Scholar.

Detailed Research Interests

I am dedicated to exploring how to (1) establish the science of large language models (Science of LLMs); and (2) enable LLMs with knowledge and reasoning skills to solve scientific tasks (LLMs for Science).

1) Science of LLMs
I aim to build a systematic science of LLMs that connects internal model mechanics to observable behaviors.

• Microscopic perspective: Probe how hidden states, neurons, and sparse features arise and correspond to specific model behaviors; develop tools (e.g., sparse autoencoders) to interpret and steer model internals.
• Macroscopic perspective: Study how architectural and training choices (data, objectives, RL, distillation) shape emergent capabilities such as implicit reasoning and robustness.

2) LLMs for Science
I explore how LLMs can assist and automate the scientific workflow.

• Knowledge-intensive QA and retrieval: Combine parametric and retrieved knowledge for fact-seeking, multi-hop reasoning, and conflict resolution.
• Agentic research assistants: Build multi-agent systems and evaluation protocols for end-to-end scientific inquiry (problem formulation → evidence gathering → reasoning → reporting).
• Learning from real-world feedback: Use verifiable signals and reward models to help LLMs improve their research skills and factual reliability over time.

Selected publications

1. SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generation
   Zijun Yao*, Weijian Qi*, Liangming Pan, Shulin Cao, Linmei Hu, Weichuan Liu, Lei Hou, Juanzi Li
   ACL, 2025. Oral Presentation (2.9% in Submission)

2. VisKoP: Visual Knowledge oriented Programming for Interactive Knowledge Base Question Answering
   Zijun Yao*, Yuanyong Chen*, Xin Lv, Shulin Cao, Amy Xin, Jifan Yu, Hailong Jin, Jianjun Xu, Peng Zhang, Lei Hou, Juanzi Li
   Demo of ACL, 2023. Best Demo Award

3. KoRC: Knowledge oriented Reading Comprehension Benchmark for Deep Text Understanding
   Zijun Yao*, Yantao Liu*, Xin Lv, Shulin Cao, Jifan Yu, Lei Hou, Juanzi Li
   Findings of ACL, 2023.

4. Untangle the KNOT: Interweaving Conflicting Knowledge and Reasoning Skills in Large Language Models
   Yantao Liu*, Zijun Yao*, Xin Lv, Yuchen Fan, Shulin Cao, Jifan Yu, Lei Hou, Juanzi Li
   LREC-COLING, 2024. [arXiv]

5. RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style
   Yantao Liu, Zijun Yao, Rui Min, Yixin Cao, Lei Hou, Juanzi Li
   ICLR, 2025. Oral Presentation (1.2% in Submission)

6. Transferable and Efficient Non-Factual Content Detection via Probe Training with Offline Consistency Checking
   Xiaokang Zhang*, Zijun Yao*, Jing Zhang, Kaifeng Yun, Jifan Yu, Juanzi Li, Jie Tang
   ACL, 2024. [arXiv]

7. LinguaLens: Towards Interpreting Linguistic Mechanisms of Large Language Models via Sparse Auto-Encoder
   Yi Jing, Zijun Yao, Lingxu Ran, Hongzhu Guo, Xiaozhi Wang, Lei Hou, Juanzi Li
   EMNLP, 2025.

8. How does Transformer Learn Implicit Reasoning?
   Jiaran Ye*, Zijun Yao*, Zhidian Huang, Liangming Pan, Jinxin Liu, Yushi Bai, Amy Xin, Liu Weichuan, Xiaoyin Che, Lei Hou, Juanzi Li
   NeurIPS, 2025. Spotlight (3.5% in Submission)

9. Interpretable and Low-Resource Entity Matching via Decoupling Feature Learning from Decision Making
   Zijun Yao, Chengjiang Li, Tiansi Dong, Xin Lv, Jifan Yu, Lei Hou, Juanzi Li, Yichi Zhang, Zelin Dai
   ACL-IJCNLP, 2021. [arXiv]

^{* Equal contribution}