Developer Quickstart

Getting started with Pythia

Pythia is a real-time hallucination detection library for LLM developers to detect hallucinations in LLM responses against verified references. The Wisecube Python SDK offers a straightforward syntax to integrate Pythia into Python Generative AI applications for continuous monitoring and analysis.

Getting started with Pythia involves four simple steps:

1. Request Wisecube API key

2. Install Wisecube Python SDK

3. Authenticate your API key

4. Send your API request

If you need further assistance or have any questions, please join our Subreddit.

1. Request Wisecube API Key

Submit the API request form with your active email address and the purpose of the API request. You’ll receive an API key in your inbox.

2. Install Wisecube Python SDK

Install the latest Wisecube SDK in your Python environment. Run the following command in your Python console to install the SDK:

pip install wisecube==1.0.0

You can run pip list | grep wisecube to list the Python libraries you have installed in your environment to verify the Wisecube installation.

3. Authenticate API Key

Import WisecubeClient from wisecube and pass your API key to it to authenticate the Wisecube API key:

from wisecube_sdk.client import WisecubeClient
API_KEY = "YOUR_API_KEY" #replace with your API key
client = WisecubeClient(API_KEY).client

4. Send API Request to Pythia

You are ready to detect hallucinations with Pythia now. All it requires is to pass the reference, response, and question in the ask_pythia call.

Copy and run the following code to see Pythia in action

reference = ["2-D ECHOCARDIOGRAM,Multiple views of the heart and great vessels reveal normal intracardiac and great vessel relationships.  Cardiac function is normal.  There is no significant chamber enlargement or hypertrophy.  There is no pericardial effusion or vegetations seen.  Doppler interrogation, including color flow imaging, reveals systemic venous return to the right atrium with normal tricuspid inflow.  Pulmonary outflow is normal at the valve.  Pulmonary venous return is to the left atrium.  The interatrial septum is intact.  Mitral inflow and ascending aorta flow are normal.  The aortic valve is trileaflet.  The coronary arteries appear to be normal in their origins.  The aortic arch is left-sided and patent with normal descending aorta pulsatility."]
response = "65 year old female presented with pT3 pN1a moderately differentiated adenocarcinoma of rectum on 9/15/2021."
question = "What was theTNM stage of the cancer?"

client.ask_pythia(reference,response,question)

The output of the above code should look like the screenshot below:

Pythia categorizes claims into relevant classes including:

• Entailment: Accurate outputs.

• Contradiction: Hallucinated claims.

• Neutral: Neither accurate nor hallucinated outputs. 

• Missing facts: Claims present in reference but disregarded by LLMs.

Finally, it provides an overall performance with the percentage contribution of each class in the metrics dictionary.

Next Steps

Now that you understand how Pythia works, you are set to explore its full potential:

• Develop a detailed understanding of Pythia’s functionality, and read the Wisecube methodology for hallucination detection.

• For an in-depth understanding of Pythia API, explore the API documentation.

• Learn to integrate Pythia within LLM workflows, and check out example use cases and tutorials.

• Have questions or need further assistance? Head over to the Pythia Subreddit.