NVIDIA has unveiled its latest AI model family, Nemotron-4 340B, which includes variants such as Nemotron-4-340B-Base, Nemotron-4-340B-Instruct, and Nemotron-4-340B-Reward. Released under the NVIDIA Open Model License Agreement, these models are freely accessible, allowing for distribution, modification, and utilization in both research and commercial applications. Designed to operate on a single DGX H100 with 8 GPUs in FP8 precision, these models provide a competitive edge in various AI benchmarks.
A standout feature of the Nemotron-4 340B series is its reliance on synthetic data—over 98% of the data used for model alignment is synthetically generated. This not only demonstrates the model’s capability in producing high-quality synthetic data but also supports the development of smaller language models through such datasets. NVIDIA is also sharing the synthetic data generation pipeline used during the model alignment, further fostering open research and development within the AI community.
This move by NVIDIA could potentially revolutionize how synthetic data is used and generated in AI, providing valuable tools for researchers and developers aiming to push the boundaries of machine learning and generative AI.