{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "bc2a41df",
   "metadata": {},
   "source": [
    "# LangChain LLM Examples\n",
    "LangChain is a framework that provides, among others, a set of Python APIs to ease the use of Large Language Models.  \n",
    "This notebook shows a few basic examples to get started, see https://python.langchain.com/en/latest/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b46a56a1",
   "metadata": {},
   "source": [
    "# OpenAI example\n",
    "OpenAI provides a state of the art LLM, via an API. The model inference is perfomed on OpenAI platform. You need to register to get an API key. There is a limited amount of compute that can be done for free, see also pricing: https://openai.com/pricing "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "a41beffc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "········\n"
     ]
    }
   ],
   "source": [
    "from langchain.llms import OpenAI\n",
    "import os\n",
    "\n",
    "from getpass import getpass\n",
    "OPENAI_API_KEY = getpass()\n",
    "\n",
    "# Get an OpenAI API Key from https://platform.openai.com/\n",
    "os.environ['OPENAI_API_KEY'] = OPENAI_API_KEY\n",
    "\n",
    "# This will use OpenAI models, the specified model, see also https://openai.com/pricing\n",
    "model = \"gpt-3.5-turbo-instruct\"\n",
    "llm=OpenAI(model=model)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "425d96cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8d405691",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " \n",
      "\n",
      "Step 1: Measure the rice\n",
      "- The standard ratio for cooking rice is 1:2, which means for every 1 cup of rice, you will need 2 cups of water.\n",
      "- Measure the desired amount of rice and place it in a bowl.\n",
      "\n",
      "Step 2: Rinse the rice\n",
      "- Rinse the rice in a fine-mesh strainer under cold running water.\n",
      "- This will help remove any excess starch and impurities from the rice.\n",
      "\n",
      "Step 3: Add water\n",
      "- In a separate pot, add the measured amount of water.\n",
      "- Bring the water to a boil over high heat.\n",
      "\n",
      "Step 4: Add the rice\n",
      "- Once the water is boiling, add the rinsed rice to the pot.\n",
      "- Stir the rice briefly to ensure it is evenly distributed in the pot.\n",
      "\n",
      "Step 5: Reduce heat and cover\n",
      "- Reduce the heat to low and cover the pot with a lid.\n",
      "- Let the rice simmer for about 18-20 minutes.\n",
      "\n",
      "Step 6: Check the rice\n",
      "- After 18-20 minutes, check the rice to see if it is done.\n",
      "- The rice should be tender and all the water should be absorbed.\n",
      "\n",
      "Step 7: Fluff the rice\n",
      "- Once the rice is\n"
     ]
    }
   ],
   "source": [
    "from langchain import PromptTemplate, LLMChain\n",
    "\n",
    "template = \"\"\"Question: {question}\n",
    "\n",
    "Answer: Let's think step by step.\"\"\"\n",
    "prompt = PromptTemplate(template=template, input_variables=[\"question\"])\n",
    "llm_chain = LLMChain(prompt=prompt, llm=llm)\n",
    "\n",
    "question = \"How to cook rice?\"\n",
    "\n",
    "print(llm_chain.run(question))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c53573b",
   "metadata": {},
   "source": [
    "# Open access models, download and run on local GPU\n",
    "There is a variety of models available for open access. However they currently do not perfom as well as the commercial ones and require machine with powerful GPUs to run the more advanced model.  \n",
    "Here we just provide some basic example of small models that can fit on a T4.  \n",
    "Note download the models from the web and uploading the weights into the GPU can take a few minutes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c8ea6d0b",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-05-16 15:10:50.119570: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n",
      "2024-05-16 15:10:50.199324: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
      "To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from transformers import pipeline\n",
    "from langchain.llms import HuggingFacePipeline\n",
    "from langchain import PromptTemplate, LLMChain\n",
    "\n",
    "# model = \"EleutherAI/pythia-2.8b\" \n",
    "model = \"databricks/dolly-v2-3b\" # Or try another model\n",
    "\n",
    "pipe = pipeline(\n",
    "    model=model, \n",
    "    torch_dtype=torch.bfloat16, \n",
    "    trust_remote_code=True, \n",
    "    device=0, \n",
    "    max_length=200,\n",
    ")\n",
    "\n",
    "llm = HuggingFacePipeline(pipeline=pipe)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "5139e6f8",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Both `max_new_tokens` (=256) and `max_length`(=200) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
      "Prompt after formatting:\n",
      "\u001b[32;1m\u001b[1;3mQuestion: How to cook rice?\n",
      "\n",
      "Answer: Let's think step by step.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "ell.\n",
      "3. Bring the water to a boil.\n",
      "4. Once the water boils, reduce heat to low.\n",
      "5. Simmer for 10-15 minutes.\n",
      "6. Open the rice valve and let the rice steam.\n",
      "7. When the rice is cooked, fluff the rice with a fork.\n"
     ]
    }
   ],
   "source": [
    "template = \"\"\"Question: {question}\n",
    "\n",
    "Answer: Let's think step by step.\"\"\"\n",
    "\n",
    "prompt = PromptTemplate(template=template, input_variables=[\"question\"])\n",
    "\n",
    "llm_chain = LLMChain(prompt=prompt, llm=llm, verbose=True, memory=None)\n",
    "\n",
    "question = \"How to cook rice?\"\n",
    "print(llm_chain.run(question))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b082fe6",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}