Evaluating LLMs: how and why?

As the age of AI is well and truly upon us, the importance of evaluating these new tools has never been more prevalent. But just how can this be achieved, how does evaluating an LLM work, and why is it so important?

The potential problems with LLMs

LLMs, whilst fantastic tools at the cutting edge of current technologies, do come with their own unique set of problems, namely in reliability and trustworthiness. As Agarwall put it plainly, “LLM applications can be very hard to get right.”

These models, which you can learn more about here, are trained on vast datasets and can sometimes generate responses that can be inaccurate or biased, reflecting the limitations and biases present in the data they were trained on. Furthermore, their capacity to produce content that is indistinguishable from human-generated text raises concerns about misuse, such as the creation of misleading information. The difficulty in attributing sources to the information these models provide also complicates efforts to verify the accuracy of their outputs. Consequently, while these models offer remarkable capabilities, their potential for propagating inaccuracies or biases necessitates careful consideration and handling to ensure they are used ethically and responsibly.

Agarwall highlighted two cases in which an LLM went a little off the rails, these being Air Canada’s AI chatbot support and Google’s initial Bard demo. In the case of Air Canada, its support AI chatbot began lying to a customer concerning the company's bereavement policy, ‘hallucinating’ an answer that did not align with actual airline policy. Meanwhile, Google was embarrassed during the first demo of its AI chatbot, Bard, when it made a factual error about the James Webb Space Telescope, claiming it took the very first pictures of an exoplanet, which it did not. Whilst these are simple examples with little consequence, they do signal the potential for error which could come to bear in more significant cases.

Addressing concerns

Detecting faults with an LLM is more challenging than it may seem, given the fact that performance errors can occur for hundreds of reasons. Agarwall adds: “Poor training data, data drift, issues with fine-tuning, and adversarial attacks a just a few of the things that could go wrong,” and this is only the starting point – once a problem is detected, it must then be diagnosed.

“Diagnosing a problem requires meticulously identifying the problem at the source, and given hundreds of possible root causes, it requires hundreds of metrics to pinpoint that root cause. This is why RagaAI LLM Hub’s ability for testing adds significant value to a developer’s workflow, saving crucial time by eliminating ad-hoc analysis and accelerating LLM development by up to three times,” says Agarwall.

The process of evaluating and comparing LLMs

“Evaluating and comparing LLM applications requires a very holistic approach, from prompt quality and context DB, to LLM selection and response quality,” explains Agarwall. This is something which RagaAI targets, holistic evaluation with over 100 tests across the spectrum.

“Within prompting, RagaAI iterates and identifies prompt templates that work best and establishes guardrails to prevent adversarial attacks.

“Additionally, context management for RAGs is important, helping users find the right balance between LLM performance and cost or latency at scale.

“Response generation is another key target, using metrics to detect hallucinations in the LLM responses and defining guardrails to prevent biases, PII leakage, and the like,” expanded Agarwall.

Covering these key topics is at the core of evaluating and comparing LLMs.

RagaAI’s solution

RagaAI is working towards addressing this problem by focusing its efforts within three key dimensions:

• Comprehensive Testing - One platform to detect all issues (100+) that can happen with an LLM application - whether it is the data, the model or the operational environment (hardware, integrations etc.).
• Multi-Modality – The ability to evaluate AI applications that work with all data modalities simultaneously like images, text, code and tabular data. This is crucial to build system-level reliability and trustworthiness in critical applications like healthcare, finance and automotive.
• Actionable Recommendations - A comprehensive test suite helps RagaAI not just identify issues, but also track their root cause and provide scientific recommendations.

The solution is delivered in an open-source package, something Agarwall notes as “our effort in the direction to make this available to everyone so that the developer community can build on top of the best available solution. The company goal is to provide the best technology to make LLMs trustworthy and reliable.”

Delivering on this, RagaAI believes that its offering can enable firms to build and deploy performant, safe, and trustworthy LLM applications whilst simultaneously accelerating AI development by almost three times and reducing the development infrastructure costs by half through automated issue detection and actionable recommendations.