[ad_1]
Seamless Integration of Python, Pandas, and LLM
These days, it’s straightforward to make use of completely different massive language fashions (LLMs) through the online interface or the general public API. However can we seamlessly combine LLM into the info evaluation course of and use the mannequin instantly from Python or Jupyter Pocket book? Certainly, we will, and on this article, I’ll present three other ways to do it. As common, all parts used within the article can be found free of charge.
Let’s get into it!
1. Pandas AI
The primary Python library I’m going to check is Pandas AI. It permits us to ask questions on our Pandas dataframe in pure language. As a toy instance, I created a small dataframe with all EU nations and their populations:
import pandas as pddf = pd.DataFrame({
"Nation": ['Austria', 'Belgium', 'Bulgaria', 'Croatia', 'Cyprus', 'Czech Republic', 'Denmark', 'Estonia', 'Finland',
'France', 'Germany', 'Greece', 'Hungary', 'Iceland', 'Ireland', 'Italy', 'Latvia', 'Liechtenstein', 'Lithuania',
'Luxembourg', 'Malta', 'Monaco', 'Montenegro', 'Netherlands', 'Norway', 'Poland', 'Portugal', 'Romania', 'Serbia',
'Slovakia', 'Slovenia', 'Spain', 'Sweden', 'Switzerland'],
"Inhabitants": [8_205000, 10_403000, 7_148785, 4_491000, 1_102677, 10_476000, 5_484000, 1_291170, 5_244000,
64_768389, 82_369000, 11_000000, 9_930000, 308910, 4_622917, 58_145000, 2_217969, 35000, 3_565000,
497538, 403000, 32965, 666730, 16_645000, 4_907000, 38_500000, 10_676000, 21_959278, 7_344847,
5_455000, 2_007000, 46_505963, 9_045000, 7_581000]
})
df.to_csv('knowledge.csv', index=False)
Earlier than utilizing Pandas AI, let’s create the LLM occasion:
from pandasai.llm.local_llm import LocalLLM
from pandasai.llm import OpenAI# Native LLM
pandas_llm = LocalLLM(api_base="http://localhost:8000/v1")
OR
# OpenAI
pandas_llm = OpenAI(api_token="...")
Right here, now we have two decisions. These readers who’ve an OpenAI API key can use the OpenAI class. It would present higher and sooner outcomes, however this API is clearly not free. An alternative choice is to make use of a LocalLLM occasion, which makes use of the OpenAI-compatible server…
[ad_2]
