Demystifying Instruction Mixing for Fine-tuning Large Language Models

Summary

Instruction tuning significantly enhances the performance of large language models (LLMs) across various tasks. However, the procedure to optimizing the mixing of instruction datasets for LLM fine-tuning is still poorly understood. This study categorizes instructions into three primary types: NLP downstream tasks, coding, and general chat. We explore the effects of instruction tuning on different combinations of datasets on LLM performance, and find that certain instruction types are more advantageous for specific applications but can negatively impact other areas. This work provides insights into instruction mixtures, laying the foundations for future research.

Plain-language summary

Instruction tuning significantly enhances the performance of large language models (LLMs) across various tasks. However, the procedure to optimizing the mixing of instruction datasets for LLM fine-tuning is still poorly understood. This study categorizes instructions into three primary types: NLP downstream tasks, coding, and general chat. We explore the effects of instruction tuning on different combinations of datasets on LLM performance, and find that certain instruction types are more advantageous for specific applications but can negatively impact other areas. This work provides insights into instruction mixtures, laying the foundations for future research.

Research problem

However, the procedure to optimizing the mixing of instruction datasets for LLM fine-tuning is still poorly understood. However, a key unresolved issue is determining how to com- bine various instruction datasets to optimize overall LLM performance. However, combining all instruction types does not uniformly improve performance across all tasks. • Instructions reformulated from NLP down- stream tasks (such as P3) can negatively im- pact the model’s conversational abilities.

Methodology

For instance, using general-purpose, chat-like in- structions can improve the performance of LLMs as chat assistants (Chiang et al., 2023; Ouyang et al., 2022; Taori et al., 2023; Ding et al., 2023), while training LLMs on instructions based off NLP tasks improves their performance on NLP bench- marks (Sanh et al., 2022; Chung et al., 2022; Muen- nighoff et al., 2023b), and incorporating coding instructions enhances LLM code generation (Fu and Khot, 2022; Gunasekar et al., 2023). For P3, the statistics come from the original dataset, while for Alpaca, we use a dependency parsing approach to extract the root verb of each instruction. The core of our investigation revolves around un- derstanding the influence of instruction dataset dis- tributions on model performance in these differ- ent areas. We fine-tune models across all eight potential combinations of these in- struction datasets, and carry out detailed evaluation of model performance in terms of NLP downstream tasks, coding proficiency, and chat capabilities. However, combining all instruction types does not uniformly improve performance across all tasks. • Instructions reformulated from NLP down- stream tasks (such as P3) can negatively im- pact the model’s conversational abilities.

Main findings

Instruction tuning significantly enhances the performance of large language models (LLMs) across various tasks. We explore the effects of instruction tuning on different combinations of datasets on LLM performance, and find that certain instruction types are more advantageous for specific applications but can negatively impact other areas. Demystifying Instruction Mixing for Fine-tuning Large Language Models Renxi Wang1,2 Haonan Li1,2 Minghao Wu3 Yuxia Wang1,2 Xudong Han1,2 Chiyu Zhang4 Timothy Baldwin1,2,5 1Mohamed bin Zayed University of Artificial Intelligence 2LibrAI 3Monash University 4 University of British Columbia 5 The University of Melbourne {renxi.wang,haonan.li,yuxia.wang,xudong.han,timothy.baldwin}@mbzuai.ac.ae minghao.wu@monash.edu chiyuzh@mail.ubc.ca Abstract Instruction tuning significantly enhances the performance of large language models (LLMs) across various tasks. We explore the effects of instruction tuning on different combinations of datasets on LLM performance, and find that certain instruction types are more advantageous for specific applications but can negatively im- pact other areas. For instance, using general-purpose, chat-like in- structions can improve the performance of LLMs as chat assistants (Chiang et al., 2023; Ouyang et al., 2022; Taori et al., 2023; Ding et al., 2023), while training LLMs on instructions based off NLP tasks improves their performance on NLP bench- marks (Sanh et al., 2022; Chung et al., 2022; Muen- nighoff et al., 2023b), and incorporating coding instructions enhances LLM code generation (Fu and Khot, 2022; Gunasekar et al., 2023).

Key contributions

Limitations

• However, combining all instruction types does not uniformly improve performance across all tasks. • Instructions reformulated from NLP down- stream tasks (such as P3) can negatively im- pact the model’s conversational abilities.
• Since the vanilla model cannot follow instructions, we exclude it from this table.

Future work

• This work provides insights into instruction mixtures, laying the foundations for future research.
• This work provides insights into instruction mixtures, laying the founda- tions for future research.1 1 Introduction Instruction tuning has been shown to have surpris- ing efficacy for aligning large language models (LLMs) with human instructions (Chung et al., 2022; Li et al., 2023; Wu et al., 2023; Xu et al., 2023; Touvron et al., 2023; Muennighoff et al., 2023a; Gunasekar et al., 2023).
• Limitations Our work is subject to several limitations that should be addressed in future research. (1) We only use LLaMA-2 7B and 13B models in our ex- periments.
• We leave the exploration of the impact of more instructions and mixing ratios to future work.
• We acknowledge these limitations and propose that future work should focus on addressing them to help the community better understand the impact of instruction mixture on LLMs.

Models and methods

Datasets

Code repositories

Keywords