Certifying Large Language Models with LLMCert

Speaker: Isha Chaudhary

Abstract: Large Language Models (LLMs) are increasingly deployed in critical systems, e.g., healthcare and finance and can produce incorrect and biased responses. These can cause huge social and economic losses to the deploying agencies and their clients. Conventional studies are, however, insufficient to thoroughly evaluate LLMs, as they cannot scale to a large number of possible inputs and provide no formal guarantees. Therefore, we develop and present the first family of LLM certification frameworks, LLMCert, consisting of certifiers providing formal probabilistic guarantees for desirable properties such as correct LLM reasoning and fairness on prohibitively large distributions of prompts. Our certificates are quantitative — they consist of provably high-confidence, tight bounds on the probability of desirable LLM responses for random prompts sampled from a distribution. We design and certify novel specifications for bias and knowledge comprehension in individual certifiers – LLMCert-B (https://certifyllm.com/) and LLMCert-C (https://arxiv.org/abs/2402.15929), respectively. We illustrate bias certification for distributions of prompts created by applying varying prefixes drawn from a prefix distribution to a given set of prompts. We consider prefix distributions for random token sequences, mixtures of manual jailbreaks, and jailbreaks in the LLM’s embedding space to certify bias. We obtain non-trivial certified bounds on the probability of unbiased responses of SOTA LLMs, exposing their vulnerabilities over distributions of prompts generated from computationally inexpensive prefix distributions.

For knowledge comprehension certification, we design and use novel distributions of knowledge comprehension prompts with natural noise, using knowledge graphs. We certify SOTA LLMs over specifications arising in precision medicine and general question-answering. We show previously undiscovered vulnerabilities of SOTA LLMs owing to natural noise in prompts. We also establish the first performance hierarchies with formal guarantees among SOTA LLMs, pertaining to question-answering in precision medicine.

Bio: Isha Chaudhary is a third-year Ph.D. candidate at the Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, advised by Prof. Gagandeep Singh. Her research interest is broadly in trustworthy foundation models and neural networks for computer systems. She graduated from a B.Tech. in Electrical Engineering from IIT Delhi in 2022. For details about her work, please check out: https://ishachaudhary.web.illinois.edu/.