A team of researchers from Carnegie Mellon University and Tsinghua University have introduced a new system called Prompt2Model that can take a natural language prompt as input and automatically generate a task-specific AI model ready for deployment. The system was presented in a paper at this year’s Conference on Empirical Methods in Natural Language Processing (EMNLP).
Prompt2Model aims to combine the ease and flexibility of prompting large language models like GPT-3 with the efficiency and reliability of specialized AI models. Users provide a prompt that describes the desired task, along with a few examples, similar to how GPT-3 is prompted. Prompt2Model then goes through several automatic steps:
- Parsing the prompt
- Retrieving existing relevant datasets
- Using GPT-3.5 to generate synthetic training data
- Identifying a suitable pretrained model
- Finetuning the model on the retrieved and generated data
In their experiments, the authors tested Prompt2Model on question answering, code generation, and temporal expression normalization tasks. They found that for two out of the three tasks, Prompt2Model produced models that significantly outperformed GPT-3.5 despite being up to 700 times smaller. The final models achieved over 20% higher accuracy than GPT-3.5 on the same few-shot prompts.
The researchers highlight Prompt2Model’s modular design as an advantage. Each component like dataset retrieval, model training, and evaluation can be customized or replaced. This enables the system to be used as a testbed for exploring new techniques in areas like model distillation and synthetic data generation.
Prompt2Model demonstrates a promising new capability in automating the machine learning pipeline end-to-end from natural language descriptions. If the performance improvements hold up across more tasks, it could become a valuable tool for quickly prototyping and building custom AI models without manual data annotation or architecture design. The ability to produce compact yet accurate models directly from prompts could make AI much more accessible.
However, work remains to handle low-resource languages better and to ensure the quality of generated datasets. As with any automated machine learning system, there is also the ethical consideration that it lowers the barrier for creating potentially harmful AI applications. But on the whole, Prompt2Model represents an exciting step towards customizable, deployable AI from intuitive natural language instructions.