AI hallucination detection was always challenging before the launch of Wisecube Pythia. Pythia eradicates the challenges of misdiagnosis and inappropriate treatment plans with the help of Wisecube’s comprehensive knowledge graph and semantic AI. Real-time hallucination detection also reduces the need for manual inspection, speeds up the process, and saves costs associated with AI errors and their liabilities. 

Pythia easily integrates with the LangChain ecosystem so that developers can take its full advantage while developing trustworthy AI systems. The straightforward syntax of the Wisecube Python SDK makes the LangChain integration easy and allows the development of specialized healthcare LLMs. 

Let’s explore the step-by-step approach to integrating Pythia in the LangChain ecosystem using the Wisecube Python SDK. 

Getting an API Key

You need a unique API key to authenticate Wisecube Pythia. Fill out the API key request form with your email address and the purpose of the API request to get your unique API key. 

Installing Wisecube Python SDK

Before you start using the Wisecube Python SDK, install it on your machine or cloud-based IDE – if you’re using one. Copy the following command in your Python console and run the code to install Wisecube:

pip install wisecube

Installing Language Processing Libraries from LangChain

Using LangChain requires installing its libraries to build Natural Language Processing (NLP) pipelines. Follow the steps below to install relevant LangChain libraries in your project:

pip install langchain_core
pip install langchain_openai
pip install langchain_community

Installing Wikidata as a Knowledge Base

We’ll need a knowledge base to serve as a reference for our LLM responses. In this tutorial, we’ll install Wikidata as our knowledge base. Wikidata is a valuable resource for information retrieval, but it is worth noting that you can use any data as your knowledge base. 

Copy and run the following code to install Wikidata API and Mediawiki library:

pip inst all wikibase-rest-api-client
pip install mediawikiapi

Authenticating API Keys

The API key we requested in step one needs to be stored in our project so we can interact with Pythia. The os and getpass modules will help us save the API key securely by prompting the user to enter it securely in the console. Lastly, the OpenAI API key is stored in the environment variable named “OPENAI_API_KEY.”

The following code imports os and getpass, prompts you to enter the Wisecube API key and OpenAI API key, and saves them to relevant variables:

import os
from getpass import getpass
  
API_KEY = getpass("Wisecube API Key:")
OPENAI_API_KEY = getpass("Open API Key:")
os.environ["OPENAI_API_KEY"] = OPENAI_API_KEY

Defining a Function to Use Pythia for LLM Hallucination Detection

Once your API key is authenticated, you can use Pythia to detect hallucinations against references. To do so, we’ll import LangChain libraries to build an NLP pipeline and create a wrapper function to call Pythia. Below is the breakdown of the wrapper function in the following code snippet:

1. Create an instance of WikidataQueryRun to query the Wikidata knowledge base.  

2. Query Wikidata with the input question using wikidata.run(question)

3. Use regular expression to extract description and subclass fields from Wikidata response.

4. Create a Wisecube client using WisecubeClient(API_KEY).client.

5. Ask Pythia about hallucinations in Wikidata response and return the result.

Copy and run the following code to implement the get_pubmed_and_sdk_response function in your project:

import re
from langchain_core.runnables import RunnablePassthrough
from langchain_community.tools.pubmed.tool import PubmedQueryRun
from langchain_core.prompts import PromptTemplate
from langchain_openai import ChatOpenAI
from langchain_community.tools.wikidata.tool import WikidataAPIWrapper, 
WikidataQueryRun
from wisecube_sdk.client import WisecubeClient
  
def get_pubmed_and_sdk_response(question):    

    wikidata = WikidataQueryRun(api_wrapper=WikidataAPIWrapper())    
    wik = wikidata.run(question)   
    print("Wikidata Response:", wik)   
      
    #extract fields from a string to call ask_pythia    
    description_pattern = r"Description: (.+)"    
    subclass_of_pattern = r"subclass of: (.+)"    
    reference = re.search(description_pattern, wik).group(1)    
    response = re.search(subclass_of_pattern, wik).group(1)    
    

    qa_client = WisecubeClient(API_KEY).client    
    response_from_sdk = qa_client.ask_pythia(reference, response, question)    
    print("SDK Response:", response_from_sdk)   
      
    return {        
            'result': wik,        
            'response_from_sdk': response_from_sdk    
    }

Defining a LangChain Pipeline

Finally, we’ll create a LangChain pipeline to answer the user question and detect hallucinations. To do so, we create RunnablePassthrough, PromptTemplate instance, and ChatOpenAI instance to specify the output format and represent the integration of the OpenAI model in our LangChain pipeline. 

Next, we create a LangChain chain of events to get our question answered against the Wikidata knowledge base and detect hallucinations using Pythia.

The following code builds and runs the LangChain pipeline and returns the final answer the ChatOpenAI model generates based on Wikidata response and Wisecube hallucination detection:

retriever = RunnablePassthrough()
prompt = PromptTemplate.from_template(
  """Answer the question based on the following responses: {result} and {response_from_sdk}""")

llm = ChatOpenAI()
  
chain = (  
         retriever    
         | (lambda question: get_pubmed_and_sdk_response(question))    
         | prompt    
         | llm
)

question = "diabetes"
result = chain.invoke(question)

print("Final Result:", result)

The final output for our query is in the screenshot below, where SDK Response categorizes claims into relevant classes, including entailment, contradiction, neutral, and missing facts.  

Benefits of Integrating Pythia with LangChain

Pythia’s ability to integrate into existing workflows allows immediate hallucination detection in your workflows. LangChain integration ensures developers take full advantage of LangChain’s boundless benefits in NLP development while detecting hallucinations and taking corrective actions in real-time. Integrating Pythia with LangChain offers the following benefits:

Extensive Knowledge Graph

Pythia uses Wisecube’s foundational knowledge graph, which consists of 35 different external data sources, 1M+ publications, 260M extracted facts from text, and 5B+ total facts. This allows in-depth claim verification and reliable hallucination detection.

Advanced Hallucination Detection

Pythia uses knowledge triplets to capture finer details and understand the context behind the claims. The ability to trace connections between facts allows improvement of LLMs over time.

Real-time Hallucination Detection

Pythia monitors LLM responses in real-time. Continuous monitoring allows immediate comparison of responses with the LLM knowledge base, resulting in on-time corrective measures.

Privacy Protection 

Pythia is designed to protect user data so developers can build robust systems without worrying about data loss. Privacy protection makes Pythia a trusted hallucination detection tool.

Enhanced User Trust 

Pythia's advanced hallucination detection, real-time analysis, on-time corrective measures, and privacy protection results improved LLM performance. Reliable LLMs enhance user trust and company reputation.

Request your API key today and discover the hidden potential of your LLMs using Pythia for real-time hallucination detection.