Meta AI Research introduces Shepard, a tool designed to evaluate and improve the output of Language Learning Models (LLMs) like ChatGPT. Shepard provides specific feedback on issues such as factuality, logical errors, coherence, and alignment. The tool was trained using data from community QA websites and human-annotated data, and outperformed baseline models in evaluations. However, questions remain about its efficiency and the reliability of LLM outputs.
Read more at Medium…
Meta AI Research introduces Shepard, a tool designed to evaluate and improve the output of Language Learning Models (LLMs) like ChatGPT. Shepard provides specific feedback on issues such as factuality, logical errors, coherence, and alignment. The tool was trained using data from community QA websites and human-annotated data, and outperformed baseline models in evaluations. However, questions remain about its efficiency and the reliability of LLM outputs.