 Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang.
JAX: composable transformations of Python+NumPy programs, 2018.
URL http://github.com/google/jax.
Brandon et al. (2023)
William Brandon, Aniruddha Nrusimha, Kevin Qian, Zachary Ankner, Tian Jin, Zhiye Song, and Jonathan Ragan-Kelley.
Striped attention: Faster ring attention for causal transformers.
arXiv preprint arXiv:2311.09431, 2023.
Brooks et al. (2024)
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh.
Video generation models as world simulators.
2024.
URL https://openai.com/research/video-generation-models-as-world-simulators.
Brown et al. (2020)
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.
Language models are few-shot learners.
Advances in neural information processing systems, 33:1877–1901, 2020.
Byeon et al. (2022)
Minwoo Byeon, Beomhee Park, Haecheon Kim, Sungjun Lee, Woonhyuk Baek, and Saehoon Kim.
Coyo-700m: Image-text pair dataset.
https://github.com/kakaobrain/coyo-dataset, 2022.
Chen et al. (2023a)
Lin Chen, Jisong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin.
Sharegpt4v: Improving large multi-modal models with better captions.
arXiv preprint arXiv:2311.12793, 2023a.
Chen et al. (2023b)
Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian.
Extending context window of large language models via positional interpolation.
arXiv preprint arXiv:2306.15595, 2023b.
Chiang et al. (2023)
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing.
Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023.
URL https://lmsys.org/blog/2023-03-30-vicuna/.
Child et al. (2019)
Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever.
Generating long sequences with sparse transformers.
arXiv preprint arXiv:1904.10509, 2019.
Dao et al. (2022)
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré.
Flashattention: Fast and memory-efficient exact attention with io-awareness.
Advances in Neural Information Processing Systems, 35:16344–16359, 2022.
Ding et al. (2023)
Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou.
Enhancing chat language models by scaling high-quality instructional conversations.
arXiv preprint arXiv:2305.14233, 2023.
Esser et al. (2021)
Patrick Esser, Robin Rombach, and Bjorn Ommer.
Taming transformers for high-resolution image synthesis.
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021.
Facebook (2023)
Facebook.
Fully Sharded Data Parallel: faster AI training with fewer GPUs — engineering.fb.com.
https://engineering.fb.com/2021/07/15/open-source/fsdp/, 2023.
[Accessed 16-May-2023].
Fu et al. (2024)
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al.
Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis.
arXiv preprint arXiv:2405.21075, 2024.
Gafni et al. (2022)
Oran Gafni, Adam Polyak, Oron Ashual, Shelly Sheynin, Devi Parikh, and Yaniv Taigman.
Make-a-scene: Scene-based text-to-image generation with human priors.
In European Conference on Computer Vision, pages 89–106. Springer, 2022.
Gao et al. (2020)
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al.
The pile: An 800gb dataset of diverse text for language modeling.
arXiv preprint arXiv:2101.00027, 2020.
Geng and Liu (2023)
Xinyang Geng and Hao Liu.
Openllama: An open reproduction of llama.
URL: https://github. com/openlm-research/open_ll