MIG: Automatic Data Selection for Instruction Tuning by Maximizing Information Gain in Semantic Space

Yicheng Chen, Yining Li, Kai Hu, Zerun Ma, Haochen Ye, Kai Chen

Project | Code

• We present an information-based instruction-tuning datasets measurement that comprehensively evaluate data quality and diversity. Accordingly, we introduce MIG, an efficient data selection algorithm that iteratively samples data points that maximizes the information gain on the label graph.

Auto Cherry-Picker: Learning from High-quality Generative Data Driven by Language

Yicheng Chen, Xiangtai Li, Yining Li, Yanhong Zeng, Jianzong Wu, Xiangyu Zhao, Kai Chen

Project | Code

• We present Auto Cherry-Picker, an innovative framework designed to synthesize training samples for both perception and multi-modal reasoning tasks from a simple object list in natural language. It employs a nowly designed metric, CLIS, to ensure the quality of the synthetic data.