LangChain LLM Examples¶

LangChain is a framework that provides, among others, a set of Python APIs to ease the use of Large Language Models.
This notebook shows a few basic examples to get started, see https://python.langchain.com/en/latest/

OpenAI example¶

OpenAI provides a state of the art LLM, via an API. The model inference is perfomed on OpenAI platform. You need to register to get an API key. There is a limited amount of compute that can be done for free, see also pricing: https://openai.com/pricing

In [2]:

from langchain.llms import OpenAI
import os

from getpass import getpass
OPENAI_API_KEY = getpass()

# Get an OPenAI API Key from https://platform.openai.com/
os.environ['OPENAI_API_KEY'] = OPENAI_API_KEY

# This will use OpenAI models, the default model currently is text-davinci-003
llm=OpenAI(temperature=0)

········

In [10]:

llm.json

Out[10]:

<bound method BaseModel.json of OpenAI(cache=None, verbose=False, callbacks=None, callback_manager=None, client=<class 'openai.api_resources.completion.Completion'>, model_name='text-davinci-003', temperature=0.0, max_tokens=256, top_p=1, frequency_penalty=0, presence_penalty=0, n=1, best_of=1, model_kwargs={}, openai_api_key=None, openai_api_base=None, openai_organization=None, openai_proxy=None, batch_size=20, request_timeout=None, logit_bias={}, max_retries=6, streaming=False, allowed_special=set(), disallowed_special='all')>

In [21]:

from langchain import PromptTemplate, LLMChain

template = """Question: {question}

Answer: Let's think step by step."""
prompt = PromptTemplate(template=template, input_variables=["question"])
llm_chain = LLMChain(prompt=prompt, llm=llm)

question = "How to cook rice?"

print(llm_chain.run(question))

huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
    - Avoid using `tokenizers` before the fork if possible
    - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)


1. Measure out the desired amount of rice. For every cup of rice, you will need two cups of water.

2. Rinse the rice in a fine mesh strainer until the water runs clear.

3. Place the rinsed rice in a pot and add the measured water.

4. Bring the water to a boil over high heat.

5. Once boiling, reduce the heat to low and cover the pot with a lid.

6. Simmer the rice for 15-20 minutes, or until all the water has been absorbed.

7. Remove the pot from the heat and let the rice sit, covered, for 10 minutes.

8. Fluff the rice with a fork and serve.

Open access models, download and run on local GPU¶

There is a variety of models available for open access. However they currently do not perfom as well as the commercial ones and require machine with powerful GPUs to run the more advanced model.
Here we just provide some basic example of small models that can fit on a T4.
Note download the models from the web and uploading the weights into the GPU can take a few minutes.

In [1]:

# Wrap a model downloaded from HuggingFace with the LangChain API

import torch
from transformers import pipeline
from langchain.llms import HuggingFacePipeline

pipe = pipeline(model="databricks/dolly-v2-3b", torch_dtype=torch.bfloat16, trust_remote_code=True, device=0)

llm = HuggingFacePipeline(pipeline=pipe)

In [2]:

from langchain import PromptTemplate, LLMChain

template = """Question: {question}

Answer: Let's think step by step."""
prompt = PromptTemplate(template=template, input_variables=["question"])
llm_chain = LLMChain(prompt=prompt, llm=llm)

question = "How to cook rice?"

print(llm_chain.run(question))

a small saucepan or microwave safe bowl and add enough water to cover the rice by 1-2 cm. Bring the pot or bowl to a boil, reduce heat and let cook for 15-20 minutes until the water is mostly absorbed.

Step 3. Fluff with a fork and enjoy warm

It's ready when you pour it out of the pot or bowl and it's been cooled on a dish.

In [ ]: