Overview
Amazon Web Services (AWS) offers a fully managed service for developing generative AI models called AWS Bedrock. AWS Bedrock provides foundational generative AI models and customization tools to develop personalized generative AI systems.
Pythia easily integrates with AWS Bedrock to detect real-time hallucinations in LLM outputs and highlight areas of improvement in LLMs.
Integrating Pythia with AWS Bedrock
1. Get the API key
Submit the API key request form to get your Wisecube API key.
2. Install Wisecube SDK
pip install wisecube
3. Authenticate Wisecube API Key
Replace YOUR_API_KEY with the Wisecube API key you requested in step one to authenticate your API key.
API_KEY = "YOUR_API_KEY"
client = WisecubeClient(API_KEY).client
4. Install LangChain and AWS Libraries
Install the necessary libraries to interact with LangChain and AWS. faiss-cpu is a CPU-only version of the FAISS database, which we’ll use to store our data.
pip install boto3
pip install awscli
pip install langchainpip install faiss-cpu
5. Create an In-memory Vector Store to Store References
Create an in-memory vector store to store data. Similarity search results from this vector store will serve as references in the ask_pythia call.
from langchain.embeddings import BedrockEmbeddings
from langchain.indexes import VectorstoreIndexCreator
from langchain.vectorstores import FAISS
from langchain.text_splitter import CharacterTextSplitter
from langchain.document_loaders.csv_loader import CSVLoader
embeddings = BedrockEmbeddings() #create a Titan Embeddings client
loader = CSVLoader(file_path="diabetes.csv")
documents = loader.load()
index_creator = VectorstoreIndexCreator(
vectorstore_cls=FAISS,
embedding=embeddings,
text_splitter=CharacterTextSplitter(chunk_size=300, chunk_overlap=0),
)
def get_index(): #returns an in-memory vector store to be used in the application
index_from_loader = index_creator.from_loaders([loader])
return index_from_loader
def get_similarity_search_results(index, question):
results = index.vectorstore.similarity_search_with_score(question)
flattened_results = [{"content":res[0].page_content, "score":res[1]} for res in results] #flatten results for easier display and handling
return flattened_results
6. Define a Function to Interact Pythia with Bedrock LLM
Define a function that creates an LLM using a Bedrock foundational model and interacts with Pythia to detect hallucinations in LLM response. Extract only the string portion from the response and reference since ask_pythia expects text inputs only.
import boto3
import json
from wisecube_sdk.client import WisecubeClient
from wisecube_sdk.model_formats import OutputFormat, WisecubeModel
bedrock = boto3.client(service_name='bedrock-runtime', region_name='us-east-1')
modelId = 'amazon.titan-text-express-v1'
accept = 'application/json'
contentType = 'application/json'
def bedrock_and_sdk_response(question) :
body = {
"inputText": question,
"textGenerationConfig": {
"maxTokenCount": 4096,
"stopSequences": ["User:"],
"temperature": 0,
"topP": 1
}
}
body=json.dumps(body)
response = bedrock.invoke_model(body=body, modelId=modelId, accept=accept, contentType=contentType)
response_body=json.loads(response.get("body").read()) repsonse_text=response_body['results'][0]['outputText']
index = get_index()
reference = get_similarity_search_results(index, question)[0]["content"]
response_from_pythia = client.ask_pythia(reference,response_text,question)
return response_body, response_from_pythia
7. Detect Hallucinations with Pythia
Define a question variable to generate an LLM response and detect hallucinations using Pythia.
question ="What are the symptoms of type 2 diabetes?"
bedrock_and_sdk_response(question)