[Rag, OpenAI] RAG 기반 피터래빗이야기 챗봇 구축하기 (LangChain + OpenAI + Streamlit)

 피터래빗이야기 PDF 파일을 기반으로, GPT와 LangChain을 활용해 RAG 기반 대화형 챗봇을 만들어보았습니다.

 

📁 1. PDF 파일 준비

먼저 프로젝트 루트 디렉토리 아래에 `pdf` 폴더를 만들고, 여기에 오만과 편견 PDF 파일을 넣어줍니다:

project/
 └── pdf/
     └── 파일명.pdf

Kids-Stories-Peter-Rabbit.pdf

1.08MB

🧠 2. GPT 처리 모듈: `gpt_proc.py`

# gpt_proc.py

from langchain.prompts import PromptTemplate
from langchain_openai import ChatOpenAI
from langchain_community.document_loaders import PyPDFLoader
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain.text_splitter import RecursiveCharacterTextSplitter, TextSplitter
from langchain.chains import RetrievalQA, ConversationalRetrievalChain

mylist =[]

def create_vector():
    print("create vector")
    pdfloader = PyPDFLoader( 'pdf/Kids-Stories-Peter-Rabbit.pdf')
    document = pdfloader.load()

    txt_split = RecursiveCharacterTextSplitter( chunk_size=500, chunk_overlap=20 )
    docs = txt_split.split_documents( document )

    embedding = OpenAIEmbeddings( model='text-embedding-ada-002')
    vdb = FAISS.from_documents( docs, embedding )
    return vdb

def convesation_response( question, hist):
    result = qa_chain.invoke( {'question':question, 'chat_history':hist} )
    return  result['answer']

gpt = ChatOpenAI( model='gpt-3.5-turbo', temperature=0)
vdb = create_vector()
retriever=  vdb.as_retriever( )
qa_chain = ConversationalRetrievalChain.from_llm(llm=gpt, 
                                                     retriever=retriever )

💬 3. Streamlit 챗봇 UI: `chatbot.py`

# chatbot.py

import streamlit as st
from streamlit_chat import message
from gpt_proc import convesation_response, mylist



if "history" not in st.session_state:
    st.session_state['history'] =[]

response_container = st.container()

with st.form("rag chat", clear_on_submit=True):
    user_input = st.text_input('User')
    submit = st.form_submit_button('send')
    if submit:
       response = convesation_response(user_input, 
                                       st.session_state['history'] )
       st.session_state['history'].append((user_input,response) )
       mylist.append( user_input )
       print( mylist)

       with response_container:
           for u, r in st.session_state['history']:
               message( u, avatar_style="fun-emoji" ,is_user=True)
               message( r, avatar_style="bottts")
        #    st.write( response)

▶️ 4. 실행 방법

터미널에서 아래 명령어로 실행할 수 있습니다:

streamlit run chatbot.py