2024 → 2025
The origins of SWE-bench-Live
In 2024, Linghao Zhang, Yu Kang and Chaoyun Zhang came up with the initial idea of scaling the SWE dataset creation with an automatic GitHub issue&PR scraping pipeline.
If you refer to the SWE task creation pipeline of SWE-bench-Live, or SWE-bench-Live/SWE-bench-Live (Python only tasks), please cite
@article{zhang2025swebenchgoeslive,
title={SWE-bench Goes Live!},
author={Linghao Zhang and Shilin He and Chaoyun Zhang and Yu Kang and Bowen Li and Chengxing Xie and Junhao Wang and Maoquan Wang and Yufan Huang and Shengyu Fu and Elsie Nallipogu and Qingwei Lin and Yingnong Dang and Saravan Rajmohan and Dongmei Zhang},
journal={Advances in Neural Information Processing Systems},
volume={38},
year={2025}
}
2025 → 2026
RepoLaunch across languages and platforms
In 2025, Kenan Li, Rongzhi Li, Qirui Jin and Liao Zhu further extended the automatic dataset creation idea and proposed the ultimate RepoLaunch agent. RepoLaunch agent builds and tests code repositories across multiple programming languages and operating systems, which directly enables the automatic SWE dataset creation for multiple languages and platforms.
If you refer to the automated build and test tool RepoLaunch, SWE benchmarking/training/RFT/RL environment build, SWE-bench-Live/Multi-Language or SWE-bench-Live/Windows, please cite
@article{li2026repolaunch,
title={RepoLaunch: Automating Build and Management of Code Repositories across Languages and Platforms},
author={Kenan Li and Rongzhi Li and Linghao Zhang and Qirui Jin and Liao Zhu and Xiaosong Huang and Geng Zhang and Yikai Zhang and Shilin He and Chengxing Xie and Xin Zhang and Zijian Jin and Bowen Li and Chaoyun Zhang and Yu Kang and Yufan Huang and Elsie Nallipogu and Saravan Rajmohan and Qingwei Lin and Dongmei Zhang},
journal={arXiv preprint arXiv:2603.05026},
year={2026}
}
2026 → 2027
RepoLaunch creates a new SWE benchmark: OS-bench
In 2026, Qirui Jin, Kenan Li and Liao Zhu used RepoLaunch to create executable environments and test evaluators for a new SWE benchmark: OS-bench, which evaluates coding agents on cross-platform code migration tasks from Linux to Windows and from Windows to Linux, in both Linux and Windows environments.
If you refer to the benchmark OS-bench, please cite
@article{jin2026bench,
title={OS-Bench: Evaluating SWE Agents on Cross-OS Code Migration},
author={Jin, Qirui and Li, Kenan and Zhu, Liao and Tung, Lingching and Zhang, Yikai and Wu, Yijia and Jin, Zijian and Gao, Pengfei and Kang, Yu and Lin, Qingwei and Lee, Wenke},
journal={arXiv preprint arXiv:2610.19368},
year={2026}
}