FEATURED PROJECT
TinyLLaVA Factory
A modular framework for building small-scale multimodal models, with interchangeable model components and reproducible training recipes.
About me
My research focuses on long-context modeling and multimodal large language models.
I am a Ph.D. student in the School of Artificial Intelligence at Beihang University. I am interested in making language and vision-language models more effective and efficient through better data, training methods, and evaluation. My work spans long-context data synthesis, context extension, and small-scale multimodal models.
Long-context modelingMultimodal learningEfficient training
Open Source
View on GitHubPublications 8
Google ScholarPapers and preprints, grouped by year. My name is highlighted.
2026
arXiv2026
LongForge: Measuring and Improving Long-Context Instruction Data via Evidence-Assertion Reasoning GraphsNew preprint
arXiv:2610.23167 · 2026
Measures and improves long-context instruction data using evidence-assertion reasoning graphs.
@article{chen2026longforge, title = {{LongForge: Measuring and Improving Long-Context Instruction Data via Evidence-Assertion Reasoning Graphs}}, author = {Ziyang Chen and Junlong Jia and Xing Wu and Tinghao Yu and Songlin Hu}, journal = {arXiv preprint arXiv:2610.23167}, year = {2026}, eprint = {2610.23167}, archivePrefix = {arXiv}, url = {https://arxiv.org/abs/2610.23167} }COLM2026
PolicyLong: Towards On-Policy Context Extension
COLM 2026
Refreshes long-context training data using the evolving model's uncertainty to create an on-policy curriculum.
@inproceedings{jia2026policylong, title = {{PolicyLong: Towards On-Policy Context Extension}}, author = {Junlong Jia and Jiang Zhou and Ziyang Chen and Xing Wu and Chaochen Gao and TingHao Yu and Feng Zhang and Songlin Hu}, booktitle = {Conference on Language Modeling}, year = {2026}, url = {https://arxiv.org/abs/2604.07809} }ICLR2026
EntropyLong: Effective Long-Context Training via Predictive Uncertainty
ICLR 2026
Constructs long-context training data by using predictive uncertainty to verify useful long-range dependencies.
@inproceedings{jia2026entropylong, title = {{EntropyLong: Effective Long-Context Training via Predictive Uncertainty}}, author = {Junlong Jia and Ziyang Chen and Xing Wu and Chaochen Gao and Zijia Lin and Songlin Hu and Binghui Guo}, booktitle = {International Conference on Learning Representations}, year = {2026}, url = {https://proceedings.iclr.cc/paper_files/paper/2026/hash/07e278a120830b10aae20cc600a8c07b-Abstract-Conference.html} }AAAI2026
LiteLong: Resource-Efficient Long-Context Data Synthesis for LLMs
AAAI 2026
Synthesizes long-context data through structured topic organization, multi-agent debate, and lightweight document retrieval.
@inproceedings{jia2026litelong, title = {{LiteLong: Resource-Efficient Long-Context Data Synthesis for LLMs}}, author = {Junlong Jia and Xing Wu and Chaochen Gao and Ziyang Chen and Zijia Lin and Zhongzhi Li and Weinong Wang and Haotian Xu and Donghui Jin and Debing Zhang and Binghui Guo}, booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence}, year = {2026}, volume = {40}, pages = {31274--31282}, doi = {10.1609/aaai.v40i37.40390}, url = {https://ojs.aaai.org/index.php/AAAI/article/view/40390} }arXiv2026
LongBench Pro: A More Realistic and Comprehensive Bilingual Long-Context Evaluation Benchmark
arXiv:2601.02872 · 2026
A bilingual benchmark of 1,500 natural long-context samples spanning 11 primary tasks and 25 secondary tasks.
@article{chen2026longbenchpro, title = {{LongBench Pro: A More Realistic and Comprehensive Bilingual Long-Context Evaluation Benchmark}}, author = {Ziyang Chen and Xing Wu and Junlong Jia and Chaochen Gao and Qi Fu and Debing Zhang and Songlin Hu}, journal = {arXiv preprint arXiv:2601.02872}, year = {2026}, eprint = {2601.02872}, archivePrefix = {arXiv}, url = {https://arxiv.org/abs/2601.02872} }2024
ICML2024
On the Nonlinearity of Layer Normalization
ICML 2024
Studies the nonlinearity and representation capacity of layer normalization, including group-based layer normalization.
@inproceedings{ni2024layernormalization, title = {{On the Nonlinearity of Layer Normalization}}, author = {Yunhao Ni and Yuxin Guo and Junlong Jia and Lei Huang}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, year = {2024}, volume = {235}, pages = {37957--37998}, url = {https://proceedings.mlr.press/v235/ni24b.html} }arXiv2024
TinyLLaVA Factory: A Modularized Codebase for Small-scale Large Multimodal Models
arXiv:2405.11788 · 2024
An open-source modular codebase for training and extending small-scale large multimodal models.
@article{jia2024tinyllavafactory, title = {{TinyLLaVA Factory: A Modularized Codebase for Small-scale Large Multimodal Models}}, author = {Junlong Jia and Ying Hu and Xi Weng and Yiming Shi and Miao Li and Xingjian Zhang and Baichuan Zhou and Ziyu Liu and Jie Luo and Lei Huang and Ji Wu}, journal = {arXiv preprint arXiv:2405.11788}, year = {2024}, eprint = {2405.11788}, archivePrefix = {arXiv}, url = {https://arxiv.org/abs/2405.11788} }arXiv2024
TinyLLaVA: A Framework of Small-scale Large Multimodal Models
arXiv:2402.14289 · 2024
A framework for systematically studying model components, data quality, and training recipes for small multimodal models.
@article{zhou2024tinyllava, title = {{TinyLLaVA: A Framework of Small-scale Large Multimodal Models}}, author = {Baichuan Zhou and Ying Hu and Xi Weng and Junlong Jia and Jie Luo and Xien Liu and Ji Wu and Lei Huang}, journal = {arXiv preprint arXiv:2402.14289}, year = {2024}, eprint = {2402.14289}, archivePrefix = {arXiv}, url = {https://arxiv.org/abs/2402.14289} }No publications match your search.