{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://swan.web.cern.ch/sites/swan.web.cern.ch/files/pictures/logo_swan_letters.png\" alt=\"SWAN\" style=\"float: left; width: 20%; margin-right: 15%; margin-left: 15%; margin-bottom: 2.0em;\">\n",
    "<img src=\"https://spark.apache.org/images/spark-logo-trademark.png\" alt=\"EP-SFT\" style=\"float: left; width: 25%; margin-right: 15%; margin-bottom: 2.0em;\">\n",
    "<p style=\"clear: both;\">\n",
    "<div style=\"text-align:center\"><h1>Integration of SWAN with Spark clusters</h1></div>\n",
    "<hr style=\"border-top-width: 4px; border-top-color: #34609b;\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The current setup allows to execute PySpark operations on CERN Hadoop and Spark clusters. \n",
    "\n",
    "This notebook illustrates the use of __Spark in SWAN to analyze the monitoring data available on HDFS (analytix)__ and plots a heatmap of loadAvg across machines in a particular service."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Connect to the cluster (analytix)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To connect to a cluster, click on the star button on the top and follow the instructions\n",
    "* The star button only appears if you have selected a SPARK cluster in the configuration\n",
    "* The star button is active after the notebook kernel is ready"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import necessary Spark and Python dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyspark.sql.functions import from_unixtime, when, col\n",
    "from pyspark.sql.types import *\n",
    "from pyspark.sql.functions import from_json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Select the data\n",
    "*This reads monitoring data stored in Hadoop*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the dataframe from the parquet files containing monitoring data\n",
    "df = spark.read.parquet(\"hdfs://analytix/project/monitoring/collectd/load/2022/10/14/\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check the data structure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root\n",
      " |-- dstype: string (nullable = true)\n",
      " |-- host: string (nullable = true)\n",
      " |-- interval: double (nullable = true)\n",
      " |-- plugin: string (nullable = true)\n",
      " |-- plugin_instance: string (nullable = true)\n",
      " |-- time: long (nullable = true)\n",
      " |-- type: string (nullable = true)\n",
      " |-- type_instance: string (nullable = true)\n",
      " |-- env: string (nullable = true)\n",
      " |-- region: string (nullable = true)\n",
      " |-- dc: string (nullable = true)\n",
      " |-- value: double (nullable = true)\n",
      " |-- value_instance: string (nullable = true)\n",
      " |-- _id: string (nullable = true)\n",
      " |-- availability_zone: string (nullable = true)\n",
      " |-- landb_service_name: string (nullable = true)\n",
      " |-- event_timestamp: long (nullable = true)\n",
      " |-- submitter_environment: string (nullable = true)\n",
      " |-- submitter_hostgroup: string (nullable = true)\n",
      " |-- timestamp: long (nullable = true)\n",
      " |-- toplevel_hostgroup: string (nullable = true)\n",
      " |-- version: string (nullable = true)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "df.printSchema()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a temporary table view"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.createOrReplaceTempView(\"loadAvg\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Do the heavylifting in spark and collect aggregated view to panda DF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Extract the data running a query using Spark. \n",
    "# Fetch the results into a Pandas DataFrame for later plotting\n",
    "df_loadAvg_pandas = spark.sql(\"\"\"SELECT host,\n",
    "                                      avg(value) as avg,\n",
    "                                      hour(from_unixtime(timestamp / 1000, 'yyyy-MM-dd HH:mm:ss')) as hr\n",
    "                               FROM loadAvg\n",
    "                               WHERE submitter_hostgroup like 'swan/node/production'\n",
    "                               AND dayofmonth(from_unixtime(timestamp / 1000, 'yyyy-MM-dd HH:mm:ss')) = 14\n",
    "                               GROUP BY hour(from_unixtime(timestamp / 1000, 'yyyy-MM-dd HH:mm:ss')), host\"\"\").toPandas()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize load heatmap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Heatmap of loadAvg for cluster on 2022/10/14')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAApMAAAH1CAYAAACqU3UnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABEkElEQVR4nO3dd5xkVZnw8d8zQxqQjCiSBhFUTIiIiqgYUBARwQRGTCyuruLuurKmRX03GFdXXXFMIKsCStRFQVHAXRclSM4gyBAlDkHS8Lx/nFtOUVPdXV1dp7q7+vftz/101b23nnPrVt2qp86559zITCRJkqR+zJvuDZAkSdLsZTIpSZKkvplMSpIkqW8mk5IkSeqbyaQkSZL6ZjIpSZKkvplMas6IiL0j4vcRcWdEZER8cYL1D2zW23EoG9iDiFjYbNPB070tUxUR74uICyPiz81z2n+6t2k8zTaePN3bIUkzjcnkJDRfJuMOzBkRVzXrLRzSZrXK3bEp98BhljtbRMRzgO8BqwNfAz4B/GxaN6qSiFg1Im5v3g/fn+7t6SYi9gK+BNwLfJHyepw2nds000TEPs1ruM90b8t0iojVIuKNEfH9iLg4Iu5ufhCeERF/FxErjfPYrSLiiIi4KSLujYhLIuITEbGgy7pbNz8g/zciro+I+yPi2oj4QURs02X9iIidI+LLEXF2RNzWVsYXI+JREzyvFZvHHNc27x0R8fWI+G1E3NO8/v+vh330iog4OSLuiIi7mse/tYfH/XdE3BoRKzT3d4qIz0fESc38jIj/mShOR8yPtb4rI+Ilk3msZq8VpnsDpCHZFQjgLZn5m+nemMpeD6wJJLBnRKybmbdM8zZ1ekXrf2ZeN61bopnuecB/AbcCvwKOAdYBdgM+R3mPvzgz721/UEQ8C/glsCLwI+Aa4EXAx4EXN4+5r+0hBwHPAs4EjgLuArYG9gJeExGvy8yj29ZfGfgpcD9wKvALYH5TxvuBvSLieZl52RjP64XAWkB7zM9Tjt3bgOuAzSfaORHxXuDLwC2U/XQ/8Brg4Ih4Smb+/RiPWx14MXB4Zj7YzH4PsDvlR97lwNoTld8RcxvgY5R994jJPFazmzWTmise0/yfC4nLvsBDlC/alYEJayimwWMATCTVgxuANwEbZOZrMvOAzNwX2BI4C9iekgT9RUTMB74DrAq8JjPfkJkfoiSLRwLPBT7QUc73gC0yc9vMfE9mfigzX9aUvQLwjY5a0KXAR4FHZ+ZOmfn3mfkB4OnA14FHAV8Y53nt0cT4cdu8vYCFmbkO0EuN5ELKcX4r0NruDwBPBa4A/q5plenm5ZTPh/Zk9tPAkymJ4G4Tld+xLasAhwJndMTUXJCZTj1OlJqenGCdq5r1FnZZ9gTgYMov5PuAG4HvA4/vsu6WwL9RDsw/NetfDSwCNupY9+DWtnWZdmzW2ae5vw+wE/Bryq/HP1E+dNdq1ns68BPKL+O7gOPGeC7PoDRTnkP5ILsXuIzyy3rtLuu3l78r8Bvg7qacH1E+xCfzWswD9gNOb7bz7ub2u4F5XcrtNi33vDrKOLB9H3YsezGlmbz13C9tXq81p7qvmsesTvkiWtysfzHwt8Bjm206eIzHPblZfiKwbvO+uahjnec06xw1znO/qHnsOm3zVm72yZXNsj9QvvBWbuKd3MPr1tqny01T2L8nNzFWotQ6XdJsX9d9NMZx+W3KsXsfcBPl+Hh3l+P/5I55B4/1XgJ2bJYd2DH/sZTj+HLgz81zPI9SM7Zux3Ma931LSXL+mnKKwBLgHuD3wHtpOw6adRe23juUz5fDm+f6EF3e412ez8rAAcC5TTlLmv30ui7rtpe1EDgMuLl5Lc+g1EhP6vN3jG16Q1POjzvmv6iZf0qXx7SOoauA6LGcS5vHPKPH9R/TrH/nGMuD8sP2V+PE2KeJ8f/GWeeTzTqf6LLs7c2yQ8Z47OGUz80FYyxvvYb/0+Nz/vfmfbFl23HxkkG8zk4zf7KZe0giYmdK08mKlF+ilwMbAXsCu0bECzPzrLaH7ElJln5FSbzuB54EvBPYLSK2zcxrm3WPaf6/FTiF8kXUclXHpryS0sT4E8qX1/aUD63NIuIA4CTKF8S3gKdQfp1u3jSXPNQW512UX9ansKx5ZxtKwrNLRDwrM+/ssiv2BHah/HI9mdKM9GrghRGxfWZe0uUx3RxK+SK5Bvgm5YNrD+A/gR2ANzbrnU05H+9VwNMoSd3tzbLW/0mJiL+inHd5N/BDyhfyjsCHKK/NczOzPfak9lVErEx5HZ5JSUC/R2kO+xjwggk2b9/m/8GZeUtE/ITSDPi8zPw1QGb+X0RcAryiWxN4RGxHSbCOzMxbm3lBqdHZlZIIf4XyXt6H8r7s1cnN/32ATSmvzcP0sX9bjqTss59SjombJtqYiNi1KWNlSvL6A8q+fhrwD812DExEbED50bMGcHyzzasAmwFvpuzXWyhfxrdTmhyPpbyPW25vYrU+S15GSaC/T0nWXkhp9nxWE7PT5sBvKQnS94AFlMRwvO1eCTiB8v67GPgqTa0fcHhEbJ2ZH+7y0E2B31F+gBxKaZ5+PXBsRLwkM381Xrk9eKD5/2DH/Bc1/5c7Lzozr4yISylJz2MpNXj9ljOW+ydY/znABsC/9hhvLGM+T8px0L7OXzSfMbsAP8vMP09xG4iIF1Ka9j+QmZeWjwvNKdOdzc6miWU1AweOM93O8rUHa1Nq4G4GtuqI+SRKzdpZHfM3BFbusg0vpTSNfK1j/o50qQVpW75Ps/xB4AVt8+cBP2+W3Qq8seNx32qW7d4xf1Ngfpdy3tGs/6Exyk86aiUoH0IJnNTj67B3s/5ZwCPa5q9GqfVI4A0djzm483XpoZwD6aiZbJ73fZQv3yd0rP+fzfqLprivPtzMP5KH17Ju1rxGXWsmKUnJrc17cEEzb7dm/UM71v3HZv57u8T5arNst7Z5b27mnQqs1DZ/LUpy0VPNZNvjTqZLLX+f+/fkZv65wHqT2Ib1gDsoX/wv6LK8swVgyjWTwN80897fZf3VaKsloq02f4L355fb31+UHyvLHbcsq2lK4F963U8d75fjgRXa5q/PstaY7cco6586Yr2sFWsy2zDGdv20ifVXHfN/2Mx/9RiP+0mzfJceynhWs+5iuhzHYzzmQ81jfjDG8s82yzceJ0br9R+vZvJPzTrrjrH8rmb5qh3zd23mv2mc2K3XcNyaSco5nldTfixHx3FhzeQcmaZ9A2bT1Pbh2Mu0sO1xrWTpPWPE/fdm+VY9bse5wJUd83akt2Ty0C7L3tIsO7XLshd0+0IYZ9uC8gX9yzHKXy5hpHz5Xd4s37SHMlrJ70u7LHtxs6yz/NaH28JenkfzmANZPpn8CGN8GVN+NCyhNF0u90NgEvvqMsoPhs3H2aaDx3kdv942bwXKOWd/pq1JnVIrvhQ4vSPGSpSasRt5eNLwiyb287uU+0YGl0xOev+yLJncvdfym8f9XfO4L/W4/iCTyX17KK91zOzTZdk8yo/T69tfp7bla1Gar49om7ewiXdDL+/PLu/Jh+hI8JtlrR9F3+5S1lV0/yF1NXDzZLahS4z3NmX8HlixY9mJjJPMUGpkE9h7gjLWZlkT93LN+WM85pksOw1guWO4WefyzmNvnNd/vGTy/mad5d4DzfJrm+UbdMz/ZvPYtcaJ3XoNJ0omv0tJWh/b5bgwmZwjk83cfcjMMevwI+IqSu1Ku9YJ0E8bY+ieLZv/TwQubOIE5Ut6H0qT29qUpKvlfvpzRpd5rU4QZ3ZZ1mpK36h9ZtPE9leUE8a3ovw6be/QteEY5Z/SOSMzlzbDT2xOOWfz6rE2vrEN5Yvt5DHiL23i1NAaIuSXnQsy87aI+D3wfEoz8TkwuX3V9LB8HHBNZnZrfjsZ+Kcxtu1dzf/vtG3TgxHxPUqT+puB/2jmL46Ik4CdImKrzLywechulKbIf89lPTyh7M+HKKdcdJrU0CETmPT+bfO7SZb17Ob/T8dda7COA/4F+GpEvIzSdPy/wIWZ5Vu4R1tSzom9DPjoGM2Kf6Z8pnQ6Jx/ei3lcbe/JazPz4i6rtF6rbsfc2Zm5tMv8a1j2uThpEbEnZUipGyi1jw+M/4jlQzT/x9znEbEa5fXaAvhMZh7Rw3ZtSTn1YEVgr27HcEQ8lfJZ95FJbnM/lnueTeekV1J+GN0+peDldXgzpaLkyqnE0uxmMjkc6zb/3zXuWg8fSuELwP6UmocTKEld69yWfVg+Ye3VHV3mPdjDshU75h9OOQ/wSso5XTdQmiehbPfKY5R/4xjzb2j+rznG8nZrArdm5nIJdZM83UxpfquhtX3Xj7G8NX+ttnmT2Vet+BPtp4eJiCdSzhW9ODM7x2v8DiWZfBdNMtk4mNIZ662UZjlY1vP7kI4YrX3e7Rywsba1H/3s35au+2YcrRjXjrfSIGXm1c05qQcCO1POIQa4JiI+l5n/MeaDH671mbIFY/+4gO7Ds0x2P03lNbl9jMc8SJ+jiUTEqygdem4CXjhGEtP6LBvr82SNjvU6y1gN+G/KMfWFLD3BJ9quLSjnuK9DSSSPG2PVPZr/R00Uswd3UE7XWJPSotCp9Tzbz4ndAXjkVMuPiHUovdZ/yYDPLdbsYzI5HK0PrKdl5rkTrRwR6wPvA86nnId0Z8fyvQe/ib2LiG0pH4i/AF7eXisQEfMoHRfGMtZAvo9u/nf9cO9wB7BORKzYWSPRDL67HhN0KJiC1vY9Grigy/IN2tfrY1+14k+0nzq1Ot48IcYeWP/JTSenVu3i0ZT99KaI+DDlS3AXSs1VZ63fEso+X6FLQjnu4MyTNKn9226SNXuwLNHZkNKbuh+tTmndPkvX6vaAzLwIeH3zXn0a8BJK8/eXIuLuzPxWD+W2nv/RmbnnuGt22YRJrt/+mnQz5msyaBHxWkpHoxuAF+XYYzi2OvJtOcbyLZr/l3YpY3VKIvk8So1kL4nkEymd5tYFXpuZx46z+h6UH33dankn6xLK592WwP91bNMGlPNwF2fmPR3lJ+WH7VRs0pT9IuChMWrHf97M/0BmfnGK5WkGc5zJ4WjVFD2vx/UfS3ltTuySSG7ULO/Uakqa32XZoD2u+X9cl+al7Si9Q8fygs4ZTbPLDs3d3/dQ/u8p++f5XZY9n7IPzuqybBBa27dj54KIWIvSO/1eytA6MMl91bzelwMbRkS3AYu7lbsypanpIcoQN9/qMp3QrP6X2vEsvTiPoAxj8hLKaRUrsHytJCzb59t3WbZDl3n9muz+nYrWcbnLFGLc1vzfuMuybcd7YGY+mJlnZuanKZ3KoIw60DLeMX0xJRl+dnMaRTXNe/IKyntyiy6rvLD5X+uYAyAi3kDpbX8dpcPUWIkkLGt637lLnMdSkq+rKa0F7cvWpJxv+Tzgn3tMJJ9COf1kHUqT+5hJWkRsRvkBcfREcXs05vNk2fu685SRVwGnZeZYNc29uoXunzXfopx+AeUUkm9RKkY0yqb7pM3ZNMHy4+F1Wecqlu+Asy7lS+cmYLsuj5nHwzt5PLqJ8Vse3kvzESzrvZgdMbZq5o81ptg+jH0y/46M0XmHtvHi2uY9u5l3ZMe661POu0zgqjHKT8buzf3LbtveZZtaY8v9jrZeipShSn7XLOvslX5w5+vSQzkHsnwHnIWU81VvBx7Xsf6Xm/W/McV91erN/SN66M3dtj/G7B1LGbeyNR7nmm3zn9s89nuUZOABYP0uj38r3Xtzr8lge3NPav+OF6uHbWjvzd2tY1Evvblf38z/fsf8pwB3snwHnO2AR3Up6zXNuoe3zXt5M2+5MQSb5a0xBr9Gl7ECKTWGW7XdX9j53pnEvmq9J4/h4Z9J61HGG01gh17Lmuxr1rz/llKSv017WH8+5fzzBF7ZNn8ey3p6H9DxmLUpwzYl8PEet2trSkeoe4CX9bB+q9PXM3tYdx8m7oCzGeXH1S0sP4JIq1Pjc9rmP6OZ98Eeym+9hj2NM9nx2IOxA86cmmzmHoIs4/29hvJr9LSm48MFlJqkTSgnoq9LGdqFzLwhIg6jdNg4OyJOpHxp70T54Dib8iHW7hLKuV97RcT9wB9Z1nt7og4tk3U6pdPAnhHxG0oHjEdRfglfwvhXmfkxcHREHE35sHsa5UvzVsrgyxPKzO9HxO7A64ALIuIYynN9FeXD9YjM/N7kn1ZPZV8VEftThs85KyKOoAzP8QLK63gxy84/hP721eeb5/LqpowTKK//6ynJ3Cs71m81cX9znO2+MyJ+SPmCelOz/WTm/0bE5cBracZAzcxu4zN+l/J+3Bk4P8r1hFdstvEM4PEsa/LtWx/7dypl3dzUdv0I+FVE/JQyUsIalCuIbEx5P43nWEotzN5Nq8FvKcf07s2y13Ws/wbgPRFxCuX9fxulM8ZulPNov9i27v9RkpT9m/PTWuemfjkz7wA+RTl+9qOMv/lLymfA+pRm3OdSOnlcyNR9jvKe3R04JyKOp/x4e21T3mcyc5Adsf6iGcPw25RE8FfA27o0qd6ebc2oWTr1vY1SK/ejiPgR5TPxxZQa4/+ljKLR7qhm2RXAvDE6Sx6TmWc327U2pWl7neb/c6L71Wa+mMs6uuxBaXY+fYzn+k6W1fS3WjV2a95bUJrH/63tef4hIj5IORf6jIg4nGWXU9wI+Hxmtjd/t87XPHqM8negjGUMy8633SIiDm4rc59uj9UcN93Z7Gya6LNmsm3ZQsqgxJdRksIllC/HQ4FXday7KvDPlC+ceym9H79KSTpP7rYdlCEpTqLUtjxEW60aA6yZbOavQxn376pm+66g9FJdtZl3Vcf6fymfMmj6/1FqyW6njKe45SRfi3mU5PMMyhfuPZSavvfQceWPZv2Dx3pdxinjwPZ92LHspZTmsNsoScDlwGfoMtTGZPdV85g1KJ2wrmXZFXD+jo4r4FCShqQkGitO8Hy2b9Y9u2P+R1lWa9x1XL5mvVUotWF/aJ7zVc17dMPmscdMYt92fQ/3uX/HjdXDtjyJkixfS/kivpEyKsC+Het1rX2lJJ2HU34Q/ZnyA2JPug8N9CxKTeI5betfTukk9eQusXemHCut8QIf9h6m9NZ9M+W4v7XZ/mspP1o+TNs4hkyhZrLt9f8wpcnyz5Sa1/+hy/A6E5U1mdeM8a9i1ZqWO4aax25FqYm8uXkfXUoZKL9bTe5VPZSzT5fnONG0sFn/UZTa1f8Y57kePEGs5d5/zeN2a96zd7LsamBv7bLeBcC5U9nXPb5mredhzeQcmVoDjEpVRcQ+lC/Mt2XmwdO7NRqkiNiJkvj9W2b+43RvjzQTRcS+lN7PL8qpX/mnn/K3pLSGfDIz/2nY5Wu02QFHUk8i4jFd5q1LuWY2DK5TgTSK9qCc23jqNJYPHqeqwHMmJfXqCxHxNMrA5X+inJO1C6UZ/+uZOdlBw6U5IzOnMmrAIMr/NPDp6dwGjS6TSUm9Oopy3tdulDEU76Wcg/Vtxun8I0kabZ4zKUmSpL55zqQkSZL6NmObuf/6qAurVZlustZYl42empvu6rzAyWCstaDOy7TjputUiQtw39IpDznY1boLVqoSd968rpcCG4juVxmbupvvvm/ilfpwya13VYkLcNd9SydeqQ8P1nm7cfGNd9cJDMyv9J57zsI1Jl6pD+9/92erxAVgrUFekXOZXd+ya5W4N9xS732x744Lq8TdePVVq8RdsEK9i6799ffqXFTprI+/qN4Hfo8WPP29VXKcP//+K9Py3KyZlCRJUt9mbM2kJEnSSIrRqsszmZQkSRqmWuc/TZPRSo0lSZI0VNZMSpIkDdOINXOP1rORJEnSUFkzKUmSNEwjds6kyaQkSdIw2cwtSZIkFdZMSpIkDdOINXNbMylJkqS+WTMpSZI0TJ4zKUmSJBXWTEqSJA3TiJ0zaTIpSZI0TDZzS5IkSYU1k5IkScM0Ys3c1kxKkiSpb9ZMSpIkDdOInTNpMilJkjRMNnNLkiRJhTWTkiRJwzRizdyj9WwkSZI0VNVrJiNie2Bhe1mZ+d3a5UqSJM1II1YzWTWZjIhDgc2Bs4GlzewETCYlSdLcNG+0OuDUrpncFtgqM7NyOZIkSZpAROwMfAmYD3wzM/+tY/kHgTc2d1cAngg8MjNvHStm7XrW84FHVy5DkiRp9oh5daaJio2YD3wV2AXYCtg7IrZqXyczP5uZW2fm1sA/AqeMl0hCpZrJiPgxpTl7deDCiPgdcF/bhr6yRrmSJEka03bA5Zl5JUBEHAbsDlw4xvp7Az+YKGitZu7PVYorSZI0u1UatDwi9gX2bZu1KDMXtd3fELim7f5i4FljxFoV2Bl470TlVkkmM/OUZkM2A67PzHub+wuAR9UoU5IkaVao1Ju7SRwXjbNKtyx2rH4tuwH/O1ETN9Q/Z/KHwENt95c28yRJkjRci4GN2+5vBFw3xrp70UMTN9RPJlfIzPtbd5rbK1UuU5IkaeaKqDNN7HRgi4jYLCJWoiSMxy2/ebEm8ALg2F6C1k4m/xQRf+lsExG7AzdXLlOSJEkdMvNByjmQJwAXAUdk5gURsV9E7Ne26h7AiZl5dy9xa48zuR/wvYj4SnN/MfCWymVKkiTNXNN4BZzMPB44vmPeQR33DwYO7jVm1WQyM68Anh0RjwAiM++sWZ4kSZKGq/q1uQEy867W7YjYJjPPGka5kiRJM06loYGmy3TUs757GsqUJEmaGabpCji1DL3kzHzXsMuUJElSHdWbuSNiQ2DT9rIy89Ta5UqSJM1II9bMXTWZjIhPA6+nXPNxaTM7AZNJSZKkEVC7ZvJVwOMz877K5UiSJM0O03h+Yw21k8krgRUBk0lJkiSwmXuS7gHOjoiTaEsoM/N9lcuVJEnSENROJo+jyzUfJUmS5iybuXsTEfOBN2fmS2qVIUmSpOlVLZnMzKURcU9ErJmZd9QqR5IkaVaxZnJS7gXOi4ifA3e3ZnrOpCRJmrPsgDMp/91MkiRJGkFVk8nMPCQiFgCbZOYlNcuSJEmaFUasmbvqs4mI3YCzgZ8197eOCHt3S5IkjYjaqfGBwHbA7QCZeTawWeUyJUmSZq6IOtM0qZ1MPtilJ3dWLlOSJElDUrsDzvkR8QZgfkRsAbwP+E3lMiVJkmYuz5mclL8BnkS5lOL3gTuA/SuXKUmSNHONWDN37d7c9wAfaSZJkiSNmNq9uX8eEWu13V87Ik6oWaYkSdJMFhFVpulSu5l7vcy8vXUnM28D1q9cpiRJkoakdgechyJik8z8I0BEbIq9uSVJ0hw2nbWINdROJj8C/E9EnNLcfz6wb+UyJUmSZq7RyiWrd8D5WURsAzybsus+kJk31yxTkiRJw1O7ZpImefxJ7XIkSZJmg1Fr5h76qJkRcdawy5QkSVId1WsmO2XmNsMuU5IkaaYYtZrJoSeTkiRJc9moJZNVmrkj4taI+GZEvDhGbY9JkiTpL2qdM/kn4Gzgk8DiiPhSRDy7UlmSJEmzhlfA6c3dmfmVzHwu8BzgWuA/I+LKiPiXSmVKkiRpyGolk39JjzPzj5n5mabjzS7AfZXKlCRJmvmi0jRNanXA+VW3mZl5CfCJSmVKkiRpyKokk5n5tzXiSpIkzXaj1je56qDlEbFRRBwdEX+KiBsj4siI2KhmmZIkSTOZHXAm5zvAccAGwIbAj5t5XUXEvhFxRkScceGJR1TeNEmSJE1V7WTykZn5ncx8sJkOBh451sqZuSgzt83Mbbd66esqb5okSdLwWTM5OTdHxJsiYn4zvQm4pXKZkiRJGpLal1N8O/AV4N+BBH4DvK1ymZIkSTPWqHXAqZ1MbpyZr2yfERHPBf5YuVxJkqSZabRyyerN3F/ucZ4kSZJmoSo1kxHxHGB74JER0T7m5BrA/BplSpIkzQY2c/dmJeARTfzV2+YvAV5TqUxJkiQNWa0r4JwCnBIRB2fm1RGxepmdd9UoT5IkabawZnJyVo+I3wPrAETEzcBbM/P8yuVKkiTNSKOWTNbugLMI+NvM3DQzNwX+rpknSZKkEVC7ZnK1zPxV605mnhwRq1UuU5IkaeYarYrJ6snklRHxMeDQ5v6bgD9ULlOSJElDUruZ++2Ua3Ef1Uzr4RVwJEnSHDZq1+auVjMZEfOBH2bmS2qVIUmSpOlVLZnMzKURcU9ErJmZd9QqR5IkaTYZtd7ctc+ZvBc4LyJ+DtzdmpmZ76tcriRJ0oxkMjk5/91MkiRJGkFVk8nMPCQiVgKeACRwSWbeX7NMSZKkmcyayUmIiJcDXweuoIyqtFlE/FVm/rRmuZIkSRqO2s3cXwBemJmXA0TE5pRmb5NJSZI0N41WxWT1ZPKmViLZuBK4qXKZkiRJM5bN3JNzQUQcDxxBOWfytcDpEbEnQGYeVbl8SZIkVVQ7mVwFuBF4QXP/T8A6wG6U5NJkUpIkzSnWTE5CZnrpREmSpBFWu2byLyLirMzcZljlSZIkzUTWTPZvtPacJElSP0YsI5o3xLK8Eo4kSdKIGVrNZGZ+dFhlSZIkzVSj1sxdtWYyIvaMiMsi4o6IWBIRd0bEkpplSpIkaXhq10x+BtgtMy+qXI4kSdKsYM3k5NxoIilJkjQzRMTOEXFJRFweEQeMsc6OEXF2RFwQEadMFLN2zeQZEXE4cAxwX2umV76RJElz1XTVTEbEfOCrwE7AYspVCY/LzAvb1lkL+E9g58z8Y0SsP1Hc2snkGsA9wEvb5nnlG0mSNGdNYzP3dsDlmXllsx2HAbsDF7at8wbgqMz8I0Bm3jRRUK+AI0mSNDdsCFzTdn8x8KyOdbYEVoyIk4HVgS9l5nfHC1o1mYyIVYB3AE+iXKcbgMx8e81yJUmSZqxKFZMRsS+wb9usRZm5aIKSs+P+CsAzgBcDC4D/i4jTMvPSscqt3cx9KHAx8DLgk8AbATvkSJIkDViTOC4aZ5XFwMZt9zcCruuyzs2ZeTdwd0ScCjwNGDOZrN2b+3GZ+THg7sw8BNgVeErlMiVJkmasiKgy9eB0YIuI2CwiVgL2Ao7rWOdY4HkRsUJErEppBh+3IrB2zeQDzf/bI+LJwA3AwsplSpIkzVjT1QEnMx+MiPcCJwDzgW9n5gURsV+z/KDMvCgifgacCzwEfDMzzx8vbu1kclFErA18jJL5PqK5LUmSpCHLzOOB4zvmHdRx/7PAZ3uNWbs39zebm6cAj61ZliRJ0mwwYhfAqd6b+wrgNODXwKntg2JKkiRp9qvdzL0V5cTN5wGfi4gnAOdk5h6Vy5UkSZqRRu3a3LWTyaWUTjhLKSdx3ghMOJK6JEnSqBqxXLJ6MrkEOA/4AvCNzLylcnmSJEkaotrJ5N7ADsBfA++MiN9Qzp08qXK5kiRJM5LN3JOQmccCxzbnSu4C7A/8A+XyPJIkSZrlavfmPhLYGric0qP7zcDvapYpSZI0k41YxWT1yyn+Fnh6Zr6McnHx/YEnVi5TkiRJQ1I7mXxTZi6JiB2AnYBDgIMmeIwkSdLImjcvqkzT9nwqx1/a/N8VOKg5h3KlymVKkiTNWBF1pulSO5m8NiK+DrwOOD4iVh5CmZIkSRqS2kMDvQ7YGfhcZt4eERsAH6xcpiRJ0ozl0ECTkJn3AEe13b8euL5mmZIkSRqe2jWTkiRJajNiFZMmk5IkScM0as3cdoaRJElS36yZlCRJGiJrJiVJkqSGNZOSJElDNGIVkyaTkiRJw2QztyRJktSwZlKSJGmIRqxi0ppJSZIk9c+aSUmSpCHynElJkiSpYc2kJEnSEI1YxaTJpCRJ0jDZzC1JkiQ1rJmUJEkaohGrmLRmUpIkSf2zZlKSJGmIRu2cSZNJSZKkIRqxXHLmJpPbbfyIarGftN6aVeJ+8X/+UCXu/HkLqsRdmlklLsAzNlm7Stwb77i3Stz7HnyoSlyAx6xd5/WbV+nT6JGrrVIlLsDFtyypEnfB/PlV4q44v94nflY6/p64zhpV4h79X/9UJS7A+qutXCXuvHl1Xr+VV6h3htg1d9xTJe79S+t8xm24Vp3PN4Cvv+kZ1WJrsGZsMilJkjSKRq2Z2w44kiRJ6ps1k5IkSUM0YhWTJpOSJEnDZDO3JEmS1LBmUpIkaYhGrGLSmklJkiT1z5pJSZKkIfKcSUmSJKlhzaQkSdIQjVrNpMmkJEnSEI1YLmkztyRJkvpnzaQkSdIQjVoztzWTkiRJ6ps1k5IkSUM0YhWTJpOSJEnDZDO3JEmS1LBmUpIkaYhGrGLSmklJkiT1z5pJSZKkIZo3YlWTJpOSJElDNGK5pM3ckiRJ6p81k5IkSUPk0ECSJElSw5pJSZKkIZo3WhWTJpOSJEnDZDO3JEmS1LBmUpIkaYhGrGLSmklJkiT1z5pJSZKkIQpGq2rSmklJkiT1rWrNZESsDLwaWNheVmZ+sma5kiRJM5VDA03OscAdwJnAfZXLkiRJmvFGbWig2snkRpm5c+UyJEmSNE1qnzP5m4h4SuUyJEmSZo2IOtN0qZJMRsR5EXEusANwVkRcEhHnts2XJEnSkEXEzk1ednlEHNBl+Y4RcUdEnN1MH58oZq1m7ldUiitJkjSrzZumasSImA98FdgJWAycHhHHZeaFHav+OjN7zuWq1Exm5tWZeTWwAXBr2/1bgUfXKFOSJGk2mMZm7u2AyzPzysy8HzgM2H2qz6f2OZNfA+5qu393M0+SJEnDtSFwTdv9xc28Ts+JiHMi4qcR8aSJgtbuzR2Zma07mflQRHjVHUmSNGfVGhooIvYF9m2btSgzF7Wv0uVh2XH/LGDTzLwrIl4OHANsMV65tWsmr4yI90XEis30fuDKymVKkiTNOZm5KDO3bZsWdayyGNi47f5GwHUdMZZk5l3N7eOBFSNivfHKrZ1M7gdsD1xLeQLP4uEZsyRJ0pwyjedMng5sERGbRcRKwF7AcQ/ftnh0NFWnEbEdJVe8ZbygVZucM/MmyoZKkiSJ6evNnZkPRsR7gROA+cC3M/OCiNivWX4Q8Brg3RHxIPBnYK/2Uxa7Gfr5ixHxisz8ybDLlSRJmuuapuvjO+Yd1Hb7K8BXJhOzdjN3N8+chjIlSZJmhKg0TZehJ5OZ+U/DLlOSJEl1VG/mjojtgYXtZWXmd2uXK0mSNBPVGhpoulRNJiPiUGBz4GxgaTM7AZNJSZKkEVC7ZnJbYKuJegFJkiTNFfNGq2KyejJ5PuVa3NdXLkeSJGlWsJl7ctYDLoyI3wH3tWZm5isrlytJkqQhqJ1MHlg5viRJ0qwyYhWT9ZLJiJgHfDUzn1yrDEmSJE2vaslkZj4UEedExCaZ+cda5UiSJM0mnjM5ORsAFzTnTN7dmuk5k5Ikaa6yN/fkfKJyfEmSJE2jqslkZp4SEZsCW2TmLyJiVWB+zTIlSZJmslFr5q56be6IeBfwI+DrzawNgWNqlilJkqThqZpMAu8BngssAcjMy4D1K5cpSZI0Y0WlabrUPmfyvsy8v1WdGxErUK7NLUmSNCfNs5l7Uk6JiA8DCyJiJ+CHwI8rlylJkqQhqZ1MHgD8CTgP+CvgeOCjlcuUJEmasSLqTNOlp2buiHhuZv7vRPO6WAB8OzO/0TxmfjPvnn42VpIkSTNLrzWTX+5xXqeTKMljywLgFz2WKUmSNHIioso0XcatmYyI5wDbA4+MiL9tW7QGvY0XuUpm3tW6k5l3NWNNSpIkaQRM1My9EvCIZr3V2+YvAV7TQ/y7I2KbzDwLICKeAfy5nw2VJEkaBSPWmXv8ZDIzT6H0yD44M68GiIh5wCMyc0kP8fcHfhgR1zX3NwBeP4XtlSRJmtVGbWigXseZ/NeI2A9YCpwJrBkRX8jMz473oMw8PSKeADyeMp7mxZn5wJS2WJIkSTNGrx1wtmpqIl9FGd5nE+DNvTwwMx/IzPMz87zMfCAiHt3fpkqSJM1+ozY0UK/J5IoRsSIlmTy2qV3s90o23+rzcZIkSZphem3m/jpwFXAOcGpEbEpzve3Jysxd+3mcJEnSKJjOYXxq6KlmMjP/IzM3zMyXZ3E18MLJFBQR6/Swzr4RcUZEnHHy0d+fTHhJkqRZYV6labr0VHZErBkRX2glehHxeWC1cdb/aNvtrSLiUuDMiLgqIp411uMyc1FmbpuZ2+64xxsm8zwkSZI0DXpNZL8N3Am8rpmWAN8ZZ/09225/Fnh/Zm7WPPbf+9hOSZKkkTCnroDTZvPMfHXb/U9ExNk9PvYxmflTgMz8XUQsmOgBkiRJmh16TSb/HBE7ZOb/AETEcxn/SjaPjYjjKGNLbhQRq2bmPc2yFfvfXEmSpNlt3mj1v+k5mXw3cEhErNncvw146zjr795xfx5ARDwK+NqktlCSJGmEzNVk8iLgM8DmwFrAHZQxJ8/ttnJzGcZu828EvjrZjZQkSdLM1GsyeSxwO3AWcO1EK0fECsA7KAnnhpQBzq9r4nzLSypKkqS5atTGmew1mdwoM3eeRNxDKcnnJ4DFrRiUpvH/Al4/iViSJEmaoXpNJn8TEU/JzPN6XH+bzHx8x7zFwGnNmJOSJElz0pw6ZzIizqM0Ua8AvC0irgTuo/TSzsx86hgPvS0iXgscmZkPNbHmAa+ldN6RJEnSCJioZvIVfcbdC/g08J8RcRsl+VwT+FWzTJIkaU4asVMmx08mm2twT1pmXkVzXmRErAtEZt7cTyxJkqRRMm/Esslez5mctGZMyp1penNHxHXACZl5e60yJUmSNFy9Xpt7UiLiLZRhhHYEVgVWA14InNkskyRJmpPmVZqmS62ayY8Az+ishYyItYHfAt+tVK4kSZKGqFYyGZRe4J0eapZJkiTNSSN2ymS1ZPKfgbMi4kTgmmbeJsBOwKcqlSlJkjTjjVoHnCpN7Jl5CLAtcAplXMr7gZOBbTPz4BplSpIkafiq9ebOzNuAw2rFlyRJmo1GrGKybuefiNgzIi6LiDsiYklE3BkRS2qWKUmSpOGpVjPZ+AywW2ZeVLkcSZKkWWFOXZt7AG40kZQkSVpm1Drg1E4mz4iIw4FjKB1xAMjMoyqXK0mSpCGonUyuAdwDvLRtXgImk5IkaU4asYrJuslkZr6tZnxJkiRNr6rJZESsArwDeBKwSmt+Zr69ZrmSJEkz1ah1wKl9XfBDgUcDL6MMYL4RcGflMiVJkjQktZPJx2Xmx4C7m6vi7Ao8pXKZkiRJM1ZU+psutTvgPND8vz0ingzcACysXKYkSdKMNWrN3LWTyUURsTbwMeA44BHNbUmSJI2A2r25v9ncPAV4bM2yJEmSZgNrJichIq4ATgN+DZyamRfWLE+SJEnDVbuZeyvgWcDzgM9FxBOAczJzj8rlSpIkzUgxYqOW104ml1I64SwFHgJuBG6qXKYkSdKMZTP35CwBzgO+AHwjM2+pXJ4kSZKGqHYyuTewA/DXwDsj4jeUcydPqlyuJEnSjDRirdzVe3MfCxzbnCu5C7A/8A/AgprlSpIkaThq9+Y+EtgauJzSo/vNwO9qlilJkjSTzRuxqsnal1P8LfD0zHwZEJSaySdWLlOSJGnGmhd1pml7PpXjvykzl0TEDsBOwCHAQZXLlCRJ0pDUTiaXNv93BQ5qzqFcqXKZkiRJM1ZEnam3smPniLgkIi6PiAPGWe+ZEbE0Il4zUczayeS1EfF14HXA8RGx8hDKlCRJUoeImA98ldIpeitg74jYaoz1Pg2c0Evc2ond65oN2TkzbwfWAT5YuUxJkqQZax5RZerBdsDlmXllZt4PHAbs3mW9vwGOpMcLzdQeGuge4Ki2+9cD19csU5IkSV1tCFzTdn8x5bLXfxERGwJ7AC8CntlLUJucJUmShqjWOZMRsW9EnNE27dtZdJfNyY77XwQ+lJlLu6zbVe0r4EiSJKlNrWF8MnMRsGicVRYDG7fd3wi4rmOdbYHDovToWQ94eUQ8mJnHjBXUZFKSJGluOB3YIiI2A64F9gLe0L5CZm7Wuh0RBwM/GS+RBJNJSZKkoZquK+Bk5oMR8V5K5+j5wLcz84KI2K9Z3tdY4CaTkiRJc0RmHg8c3zGvaxKZmfv0EtNkUpIkaYhG7NLcJpOSJEnDNF3N3LU4NJAkSZL6Zs2kJEnSEI1YxaQ1k5IkSeqfNZOSJElDNGo1eSaTkiRJQxQj1s49asmxJEmShsiaSUmSpCEarXpJayYlSZI0BdZMSpIkDZGDlkuSJEkNayYlSZKGaLTqJU0mJUmShmrEWrlt5pYkSVL/rJmUJEkaIgctlyRJkhrWTEqSJA3RqNXkmUxKkiQNkc3ckiRJUsOaSUmSpCEarXpJayYlSZI0BdZMSpIkDdGonTM5Y5PJHR+7frXYDy7NKnG/sPuTqsSdN6/Om27BivOrxAW49rY/V4n7x9vvqRL3BVs+skpcgLvufbBK3PXXWLlK3D/eXGcfA6w4r05jyE8uublK3DMuuqlKXICXPWPDKnFPuqrOvthps/WqxAW48o67q8Rdd5WVqsT9zeLbqsQFOPWSW6rE/eTLnlAl7rW31/msB7jvwYeqxZ5uo9YsPGrPR5IkSUM0Y2smJUmSRtGoNXNbMylJkqS+WTMpSZI0RKNVL2nNpCRJkqbAmklJkqQhGrFTJk0mJUmShmneiDV028wtSZKkvlkzKUmSNESj1sxtzaQkSZL6Zs2kJEnSEMWInTNpMilJkjRENnNLkiRJDWsmJUmShsihgSRJkqSGNZOSJElDNGrnTJpMSpIkDdGoJZM2c0uSJKlv1kxKkiQN0aiNM2nNpCRJkvpmzaQkSdIQzRutikmTSUmSpGGymVuSJElqWDMpSZI0RA4NJEmSJDWsmZQkSRoiz5mUJEmSGtZMSpIkDZFDA0mSJKlvNnNLkiRJDWsmJUmShmjUhgaqmkxGxMrAq4GF7WVl5idrlitJkqThqF0zeSxwB3AmcF/lsiRJkma8EauYrJ5MbpSZO1cuQ5IkadaYN2Lt3LU74PwmIp5SuQxJkiRNkyo1kxFxHpBN/LdFxJWUZu4AMjOfWqNcSZKkmW606iXrNXO/olJcSZIkzSBVksnMvBogIp4NXJCZdzb3Vwe2Aq6uUa4kSdKMN2JVk7XPmfwacFfb/bubeZIkSXNSVPqbLrWTycjMbN3JzIdwoHRJkqSRUTuZvDIi3hcRKzbT+4ErK5cpSZI0Y0XUmaZL7WRyP2B74FpgMfAsYN/KZUqSJGlIqjY5Z+ZNwF41y5AkSZpNRqz/TfWayeVEhMMGSZIkjYihJ5PAM6ehTEmSpJkhKk3TZOjJZGb+07DLlCRJmimmc2igiNg5Ii6JiMsj4oAuy3ePiHMj4uyIOCMidpgoZvVheiJie2Bhe1mZ+d3a5UqSJGmZiJgPfBXYidIx+vSIOC4zL2xb7STguMzMiHgqcATwhPHiVk0mI+JQYHPgbGBpMzsBk0lJkjQnTeMwPtsBl2fmlWU74jBgd+AvyWRmtl9sZjVK3jau2jWT2wJbtQ9cLkmSpGmxIXBN2/3WsI0PExF7AP8KrA/sOlHQ2udMng88unIZkiRJs0at/jcRsW9znmNr6hzbu1ud6HIVfpl5dGY+AXgV8KmJnk/tmsn1gAsj4nfAfa2ZmfnKyuVKkiTNTJWauTNzEbBonFUWAxu33d8IuG6ceKdGxOYRsV5m3jzWerWTyQMrx5ckSVJvTge2iIjNKFcn3At4Q/sKEfE44IqmA842wErALeMFrZZMRsQ84KuZ+eRaZUiSJM02vQ7jM2iZ+WBEvBc4AZgPfDszL4iI/ZrlBwGvBt4SEQ8AfwZeP1Hfl2rJZGY+FBHnRMQmmfnHWuVIkiSpN5l5PHB8x7yD2m5/Gvj0ZGLWbubeALigOWfy7tZMz5mUJElz1TQODVRF7WTyE5XjS5IkzSojlkvWTSYz85SI2BTYIjN/ERGrUtroJUmSNAKqjjMZEe8CfgR8vZm1IXBMzTIlSZJmtFoDTU6T2oOWvwd4LrAEIDMvo4ymLkmSpBFQ+5zJ+zLz/mjONI2IFejhGo+SJEmjarqGBqqlds3kKRHxYWBBROwE/BD4ceUyJUmSNCS1k8kDgD8B5wF/RRnX6KOVy5QkSZqxIupM06V2M/cCyujq3wCIiPnNvHsqlytJkjQjjVYjd/2ayZMoyWPLAuAXlcuUJEnSkNSumVwlM+9q3cnMu5qxJiVJkuamEauarF0zeXdEbNO6ExHPoFw0XJIkSSOgds3k/sAPI+K65v4GwOsrlylJkjRjjdrQQLUvp3h6RDwBeDylUvfizHygZpmSJEkz2XT2vK6hds0kTfJ4fut+RDw6M2+oXa4kSZLqq33OZDffmoYyJUmSZoQRuzT38JPJzNx12GVKkiSpjqEkkxHxuIh4dURsNcF6+0bEGRFxxvcPsQJTkiSNoBGrmqxyzmRE/Ap4bWbeHBFvBj4GnAocGBGLMvPL3R6XmYuARQBX3XJv1tg2SZKk6WRv7t48MjNvbm6/D3hOZt7SDFh+GtA1mZQkSdLsUiuZfCAiNszMa4G7gLub+fcB8yuVKUmSNOM5NFBvPgCcGBFHAhcAv4yInwHPA75TqUxJkiQNWZVkMjNPjojtgTcAqwNnUmol/yYzL65RpiRJ0mwwYhWT9QYtz8w7gK/Vii9JkqTpV6s39wrAO4BXARsCCVwHHAt8y0sqSpKkOWvEqiZr1UweCtwOfAJY3MzbCHgr8F/A6yuVK0mSNKM5NFBvtsnMx3fMWwycFhGXVipTkiRJQ1brCji3RcRrI+Iv8SNiXkS8HritUpmSJEkzXkSdabrUSib3Al4D3BgRl0bEZcANwJ7NMkmSJI2AWkMDXUVzXmRErAtE2xVxJEmS5qzROmOy4tBAEbEmsDNNb+6IuA44ITNvr1WmJEnSjDdi2WSVZu6IeAtwFrAjsCqwGvBC4MxmmSRJkkZArZrJjwDP6KyFjIi1gd8C361UriRJ0ow2akMD1eqAE5SByjs9xMhV7kqSJM1dtWom/xk4KyJOBK5p5m0C7AR8qlKZkiRJM950DuNTQ5Waycw8BNgWOAW4D7gfOBnYNjMPrlGmJEnSbBCVpulSrTd3Zt4GHFYrviRJkqZfrXMmAYiIPSPisoi4IyKWRMSdEbGkZpmSJEkz2ohVTVarmWx8BtgtMy+qXI4kSZKmQe1k8kYTSUmSpGVGbWig2snkGRFxOHAMpSMOAJl5VOVyJUmSNAS1k8k1gHuAl7bNS8BkUpIkzUmjNjRQ1WQyM99WM74kSdJsM2K5ZN1kMiJWAd4BPAlYpTU/M99es1xJkiQNR9WhgYBDgUcDL6MMYL4RcGflMiVJkmasiDrTdKmdTD4uMz8G3N1cFWdX4CmVy5QkSdKQ1O6A80Dz//aIeDJwA7CwcpmSJEkz2GidNVk7mVwUEWsDHwOOAx7R3JYkSZqT7M09CZn5zebmKcBja5YlSZKk4avdm/sK4DTg18CpmXlhzfIkSZJmuhGrmKzeAWcr4OvAusDnIuLKiDi6cpmSJEkaktrnTC6ldMJZCjwE3AjcVLlMSZKkGctzJidnCXAe8AXgG5l5S+XyJEmSZrQYsYbu2s3cewOnAn8NHBYRn4iIF1cuU5IkSUNSuzf3scCxEfEEYBdgf+AfgAU1y5UkSZqxRqtism7NZEQc2fTo/hKwGvBmYO2aZUqSJGl4ajdz/xZ4ema+jJKH7w88sXKZkiRJM1ZUmqZL7WTyTZm5JCJ2AHYCDgEOqlymJEmShqR2Mrm0+b8rcFBzDuVKlcuUJEmasSLqTNOl9tBA10bE14GXAJ+OiJWpn8BKkiTNWA4NNDmvA04Ads7M24F1gA9WLlOSJElDUntooHuAo9ruXw9cX7NMSZKkGW20KiZtcpYkSVL/ap8zKUmSpDYjVjFpMilJkjRM09nzugabuSVJktQ3ayYlSZKGyKGBJEmSpIY1k5IkSUPkOZOSJEmalSJi54i4JCIuj4gDuix/Y0Sc20y/iYinTRTTZFKSJGkOiIj5wFeBXYCtgL0jYquO1f4AvCAznwp8Clg0UVybuSVJkoZoGpu5twMuz8wry3bEYcDuwIWtFTLzN23rnwZsNFFQayYlSZJGQETsGxFntE37dqyyIXBN2/3FzbyxvAP46UTlWjMpSZI0RLWGBsrMRYzfLN2t4Oy6YsQLKcnkDhOVazIpSZI0NywGNm67vxFwXedKEfFU4JvALpl5y0RBTSYlSZKGaBrPmTwd2CIiNgOuBfYC3tC+QkRsAhwFvDkzL+0lqMmkJEnSEE1XLpmZD0bEe4ETgPnAtzPzgojYr1l+EPBxYF3gP6NkvQ9m5rbjxTWZlCRJmiMy83jg+I55B7XdfifwzsnENJmUJEkaJq+AI0mSJBXWTEqSJA1RraGBpovJpCRJ0hBNY2/uKmzmliRJUt+smZQkSRqiEauYtGZSkiRJ/bNmUpIkaZhGrGrSZFKSJGmIRq03t83ckiRJ6ps1k5IkSUPk0ECSJElSS2bO+gnYd7bFnm1xZ+M2uy/cF+6L0Yo7G7fZfTG794VTb9Oo1EzuOwtjz7a4NWPPtrg1Y8+2uDVjz7a4NWMbt37s2Ra3ZuzZFrd2bE1gVJJJSZIkTQOTSUmSJPVtVJLJRbMw9myLWzP2bItbM/Zsi1sz9myLWzO2cevHnm1xa8aebXFrx9YEojlxVZIkSZq0UamZlCRJ0jSY9clkROwcEZdExOURccAA4347Im6KiPMHFbOJu3FE/CoiLoqICyLi/QOKu0pE/C4izmnifmIQcdviz4+I30fETwYc96qIOC8izo6IMwYYd62I+FFEXNzs6+cMIObjm+1sTUsiYv8BbC4R8YHmdTs/In4QEasMIm4T+/1N3Aumsr3djomIWCcifh4RlzX/1x5g7Nc22/xQRGw7wLifbd4X50bE0RGx1oDifqqJeXZEnBgRjxnUNrct+/uIyIhYb0DbfGBEXNv2nn75oLY3Iv6m+Wy+ICI+M9m442zz4W3be1VEnD2guFtHxGmtz6KI2G5AcZ8WEf/XfM79OCLW6CNu1++NqR5/48QdxLE3VuwpHX/jxB3I8ac+TffYRFOZgPnAFcBjgZWAc4CtBhT7+cA2wPkD3uYNgG2a26sDlw5imymXjX9Ec3tF4LfAswe43X8LfB/4yYD3x1XAehXeG4cA72xurwSsNeD484EbgE0HEGtD4A/Agub+EcA+A9rOJwPnA6tSrnj1C2CLPmMtd0wAnwEOaG4fAHx6gLGfCDweOBnYdoBxXwqs0Nz+dD/bPEbcNdpuvw84aFDb3MzfGDgBuLqfY2aMbT4Q+Pspvse6xX1h815bubm//iD3RdvyzwMfH9A2nwjs0tx+OXDygOKeDryguf124FN9xO36vTHV42+cuIM49saKPaXjb5y4Azn+nPqbZnvN5HbA5Zl5ZWbeDxwG7D6IwJl5KnDrIGJ1xL0+M89qbt8JXERJJqYaNzPzrubuis00kBNiI2IjYFfgm4OIV1vzy//5wLcAMvP+zLx9wMW8GLgiM68eULwVgAURsQIl8btuQHGfCJyWmfdk5oPAKcAe/QQa45jYnZK40/x/1aBiZ+ZFmXlJP/EmiHtisy8ATgM2GlDcJW13V6PP42+cz55/B/6hQtwpGSPuu4F/y8z7mnVuGmBsACIigNcBPxhQ3ARatYZr0scxOEbcxwOnNrd/Dry6j7hjfW9M6fgbK+6Ajr2xYk/p+Bsn7kCOP/VntieTGwLXtN1fzAASs2GJiIXA0ym1iIOIN79p8rkJ+HlmDiQu8EXKl9hDA4rXLoETI+LMiBjUoLOPBf4EfCdK0/w3I2K1AcVu2Ys+vsS6ycxrgc8BfwSuB+7IzBMHEZtSK/n8iFg3Ilal1LhsPKDYAI/KzOuhfMgD6w8w9jC8HfjpoIJFxD9HxDXAG4GPDzDuK4FrM/OcQcVs896mefDb/Z6m0MWWwPMi4rcRcUpEPHNAcds9D7gxMy8bULz9gc82r9/ngH8cUNzzgVc2t1/LFI+/ju+NgR1/g/4+6jH2lI6/zri1jj9NbLYnk90ulT4rfo1ExCOAI4H9O35R9S0zl2bm1pRfettFxJOnGjMiXgHclJlnTjXWGJ6bmdsAuwDviYjnDyDmCpSmpq9l5tOBuylNQAMREStRvhx+OKB4a1NqGDYDHgOsFhFvGkTszLyI0pT0c+BnlFNBHhz3QXNERHyEsi++N6iYmfmRzNy4ifneQcRsfgR8hDpfjl8DNge2pvyQ+fyA4q4ArA08G/ggcERTkzhIezOgH3SNdwMfaF6/D9C0bAzA2ymfbWdSmmXv7zdQje+NmnHHiz3V469b3BrHn3oz25PJxTz8V95GDK55sJqIWJFyEHwvM48adPymSfdkYOcBhHsu8MqIuIpyGsGLIuK/BhAXgMy8rvl/E3A05dSFqVoMLG6rmf0RJbkclF2AszLzxgHFewnwh8z8U2Y+ABwFbD+g2GTmtzJzm8x8PqUJblA1OQA3RsQGAM3/vpozhy0i3gq8AnhjZtb4Afp9+mjOHMPmlB8a5zTH4UbAWRHx6KkGzswbmx+hDwHfYDDHH5Rj8Kjm9JvfUVo1Jt1paCzN6SB7AocPKibwVsqxB+WH4kD2RWZenJkvzcxnUJLfK/qJM8b3xpSPv5rfR2PFnurx18M2D/L4Uw9mezJ5OrBFRGzW1BbtBRw3zds0rubX+beAizLzCwOM+8hWr7iIWEBJUC6eatzM/MfM3CgzF1L27y8zcyC1ZhGxWkSs3rpNOTF7yr3nM/MG4JqIeHwz68XAhVON22bQNSJ/BJ4dEas2748XU84DGoiIWL/5vwnlC3iQ234c5UuY5v+xA4xdRUTsDHwIeGVm3jPAuFu03X0lAzj+ADLzvMxcPzMXNsfhYkoHhBumGruViDT2YADHX+MY4EVNGVtSOsHdPKDY0Hy+ZebiAca8DnhBc/tFDOhHV9vxNw/4KHBQHzHG+t6Y0vFX6/tovNhTPf7GiVvl+FOPOnvkzLaJcg7YpZRfex8ZYNwfUJp9HqB8eL9jQHF3oDTFnwuc3UwvH0DcpwK/b+KeTx89HHsoY0cG2Jubcm7jOc10wYBfv62BM5r9cQyw9oDirgrcAqw54H37CcqH3/nAoTS9YAcU+9eUZPoc4MVTiLPcMQGsC5xE+eI9CVhngLH3aG7fB9wInDCguJdTzrVuHX+T7vU5Rtwjm9fvXODHlE4BA9kXHcuvor/e3N22+VDgvGabjwM2GFDclYD/avbHWcCLBrkvgIOB/Qb8Xt4BOLM5Tn4LPGNAcd9P+Y66FPg3mouFTDJu1++NqR5/48QdxLE3VuwpHX/jxB3I8efU3+QVcCRJktS32d7MLUmSpGlkMilJkqS+mUxKkiSpbyaTkiRJ6pvJpCRJkvpmMilp1omIhRExqDERJUlTYDIpaSQ1V0mRJFVmMilptpofEd+IiAsi4sSIWBARJ0fEv0TEKZTBoiVJlfnLXdJstQWwd2a+KyKOYNm1eNfKzBeM8zhJ0gBZMylptvpDZp7d3D4TWNjcPnxatkaS5iiTSUmz1X1tt5eyrKXl7mnYFkmas0wmJUmS1DeTSUmSJPUtMnO6t0GSJEmzlDWTkiRJ6pvJpCRJkvpmMilJkqS+mUxKkiSpbyaTkiRJ6pvJpCRJkvpmMilJkqS+mUxKkiSpb/8fiR3luZpqnYsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# heatmap of loadAvg\n",
    "plt.figure(figsize=(12, 8))\n",
    "ax = sns.heatmap(df_loadAvg_pandas.pivot(index='host', columns='hr', values='avg'), cmap=\"Blues\")\n",
    "ax.set_title(\"Heatmap of loadAvg for cluster on 2022/10/14\", fontsize=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a histogram of uptime for the monitored entities"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# create the dataframe\n",
    "df = spark.read.parquet(\"hdfs://analytix/project/monitoring/collectd/uptime/2022/10/14/\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create temporary view\n",
    "df.createOrReplaceTempView(\"uptime\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Extract the data running a query using Spark. \n",
    "# Fetch the results into a Pandas DataFrame \n",
    "df_uptime_pandas = spark.sql(\"\"\"SELECT  host, round(max(value)/60/60/24) as days\n",
    "                              FROM uptime\n",
    "                               WHERE dayofmonth(from_unixtime(timestamp / 1000, 'yyyy-MM-dd HH:mm:ss')) = 14\n",
    "                               AND hour(from_unixtime(timestamp / 1000, 'yyyy-MM-dd HH:mm:ss')) = 12\n",
    "                               GROUP BY host\"\"\").toPandas()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtMAAAH1CAYAAAA06XKHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAApD0lEQVR4nO3de5hlZ10n+u+PNBeBUIAwiE1CwjQyZsYobYGDisR7olOm4SASGA0Ok4gzcQYZL2GYI+h5zijihQcOo7aSSXS4jphIDVHQkYA6qEkQMRASUqGRrjAkEiiuA5Xwnj/2brNT2bu6anXt2rWqPp/nqaf2fte+/Pbbq5Nvr/1b76rWWgAAgM2716wLAACAvhKmAQCgI2EaAAA6EqYBAKAjYRoAADoSpgEAoCNhGtiRqqpV1VWzroONqap7V9XPVtUHq+oLwz+/Q7Ou63iq6jnDWp8z61qAfhKmgakZhpR1F7OvqiPDx522he972vA1L92q1+S4/kOSn0lyS5JfSvKzST4w04qSVNVZw33hJbOuBdid9s26AIAJvjrJ52ZdBBv2L5J8Jsl3tta+OOtiNuHyJH+R5KOzLgToJ2Ea2JFaazM/qsmmfGWSj/csSKe1tpJkZdZ1AP2lzQPYkcb1TFfVyVX1f1fVdVX1qar6dFUtVdUbqurrh495SZIPDZ9y/rFWk7V9sVV1r6p6XlVdXVWfqarPDm//aFWN/W9jVT27qt5dVZ+vqlur6neq6iur6qq17Syj7QVV9cSqektV3T7a0lJV31pVh6vq/cPP8/nhZ3txVd1vzPu/ZPj8s6rqvKq6tqo+V1W3VNWvVNV9h4/7tmFNn6qqTwzr/PJNzv9cVf18Vd1QVf9n+DpvrarvWPO4S4ef/fQkjx6Z6yMbeI8jkx43+lnXjLfhZ/vK4ee6dThv11bVs9bWluTtw7svXrMvnDV8zNie6WO1VdUDq+pXq+ojw/d5z7Fe8KraV1X/cdgn/n+G++JF63ze766qK6vq72vQV75UVS+rqgcfb66AncuRaaAXqqqS/GGSb0zyriS/leSOJKckOSvJnya5NslVSR6c5N8n+ZskV4y8zHtGbv9Okmcl+cjwtVqSpyb5L0m+Ocmz17z/Tyb5xSSfSHJZBkczvzPJn2f9I5tPSvLCJH+W5JIkD0ty7OjtTyf5J0n+V5K3JLlfkm9K8pIkZ1XVd7TW7hzzmj+W5JzhZ7sqyXcl+fEkD62q30/y+uHrHc5gvv7l8H3PWafO0c/64OHnOiPJ1UlePnz+M5K8rap+tLX2G8OHX5HkSJLnD++/fPj7kxt5r44eksGcfTLJf83gz/sZSV5TVftbay8bqS1Jzk/yjgzm6pgjG3ifeyf5oyQPTfL7Se6T5Lwkb6qq70ryb5J8Q5I/SPKFJN+f5JVVdVtr7Q2jL1RVP5NBH/ntSf5HkluTnJnkJ5J8T1U9qbX2qY18eGCHaa358ePHz1R+MgioLYNwOOnnk8PHnDbmuVeN3P+a4djlY97nXkkeMnL/tOFjL51Q13nD7e9O8sCR8QckuWa47Vkj449JsprktiSnjIxXktcd+5xr3uOskc//IxPqeEySGjP+/wyf9wNrxl8yHF9J8tUj4/dN8r4kdyb5eJKnrJmbPxo+7+s2+Of2G8PH/8ZofUkeO3zvL4z58zqS5Mgm94+Jzxn5rGdN2KfemOReI+OnZxBUv5jkMWP+HF4y4X2eM9z+nDG1tSSLSe47Mv7k4fjtGfxD48Fr/jy/mOSv17zWtw6f879GH7/m/X91O/9u+vHjZ+t+tHkA2+HF6/zMbfK1Pr92oLX2pdbaJzbxGv9q+Pvi1tpnRl7nsxkcLU6Sfz3y+Gdl8E3eK1trHxl5fEtycQYhdpL3tLuO4q6t++bha6z18uHv757wmq9orV0/8jpfSPKGDILzW1pr7xjZ9qUk/21492vXqTPJYIm7DI5kfybJC0fra619MMkrMjhC+0PHe60pujPJTw8/W5KktfahDGq7d5If3ML3ev5wfo+9z59m0Eb0kGENnxzZdnMGR/S/pqpOGnmNfzf8fcHo44fPuTSDb0zu9k0I0B/aPICpa63VpG3DntlHb+Bl3p9B6Divqh6dwdfuf5bkmrb5k94OJvlS7v61/zHvyCCsPX5k7NjtP1v74Nbah6vqIxkcDR/nryYVUVUPyKAd5alJvirJyRkc7T5m/4SnXjNm7Jbh72vHbFse/n7UpFpG/JMk90/y562128ds/5Mk/yl3n5/t9nfD8LzWVRn8A22ravtka21pzPgtGRwJnzTXJyX5itw170/K4JuN76+q7x/znPskeXhVfXlr7eMnXjawnYRpoBdaa3dW1bdlsJbx05O8dLjp01V1WQZHUT8z8QXubi7J7eNCeGvtjqr6+yT/aM3jk+RjE17vY5kcpv/3uMHhEeA/SfLEJNdlcGT5tgxCVzIIhfed8JrjerTv2MC2e094vVHHPuukpeKOjT94A681LZP+HI7N9Wa/7ZhkUi/8Hck/rAQydlvuPtdfnsH/b198nPd7YAZtOkCPCNNAbwxbOX48yY9X1YEkT0nyI0kuyiDcbfTr/ZUMTta7d2ttdXRDVe3L4GS70ZPBjt1+RAa9yWs9Yr2yJ4yfm0GQvqy19pw1NTwyxw9e03IsIH7FhO2PXPO4E/GlDI7KjvPgdZ43ab6P1bzTlrpbyaC/+6GzLgTYenqmgV5qrd3UWnt1BoH6MxmE02OO9TCfdI8nDvx1Bv/9+5Yx275l+Lx3r3l8Mljl426GLSenbLzyf3Bg+PtNY7Y9pcPrbZUbMrhYztdV1UPGbP/W4e93j9m2WZ9I8ojhUfq15td53qk1/oqZZw1///XI2PH2he3wF0keUlX/dIY1AFMiTAO9UFWnTwgjD8mgHWL0xMRPZHBE+NQJL3fJ8PfPV9X9R97j/kl+YXj31SOPf20GX9//WFWdMvL4SvLz6RbUjgx/nzU6WFWPyV0tLNtu2PrymgxaDn5udFtV/eMMTqZbzWBpwRP1Vxl8Q/rDa97nORksETjJSUleWiPrgVfV6cPa7shdJ1wmd7VNTNoXtsOvDn//ZlV95dqNVfWAqvrn21wTsEW0eQB98bVJLq+qazPoMb4lycMzOCJ974wE0NbaZ6rqL5M8uapek+TGDI5Qvrm19t7W2mur6twM1iZ+X1VdkUH4PpTBiWVvbK29ZuT1lobrBP/nJH9TVW/IXetMPzSD9azP3OTnWUxyU5IXVNXXZHA09dQMLsv9lsw2/F2cwRJwF1XVEzK48MmxdaZPTnLRhBMAN+uVGQTpX6uqb89gze+vzWBt7P+RwVyM894M1ne+tqrelkGP9A9k0BryU2tOGrwhgxMBn1lVX0zydxn8Wf9Oa+3DW/AZjqu19j+r6uIM/uH1waq6MoMVQR6Ywcm3T8ng5Nazt6MeYGsJ00BfXJNBGHlKBqHjIRmcsHdtBkvF/cGax/9gBkcEz85gXelKcjSDIJbh2DsyWCbvR4Zj1yf55SS/tvbNW2s/X1VHk7wggwD46SRvTfJTSd6Wu/dYH1dr7bPDEyp/IYOj009OcnMGa0z/SgbhcCZaa7dX1bGLzTwtg8/8+QyOJL+stfa2LXqf9w+vqPifkyxkcFT5TzNY/eJpmRymP5HBBWh+MYM/iwdlsNrLL7XWXrvmPe6sqqdmMM/H/jFQGYTXbQnTwzpeWlV/nsHR82/O4B+BKxkE/cMZfPsB9FCNX+IUgI2oqgdlsLrEe1prT5p1Pbvd8NLl72itnTXrWgASPdMAG1JVD197otxw5Y9fzuAy4JfPpDAAZkqbB8DG/F9Jfq6q/jiD3t6HZrDyx1dlcDGZV86uNABmRZgG2Ji/zKDP9lsyuAhHMjiJ7P9N8tLW2j0ucw7A7qdnGgAAOtIzDQAAHfW6zeNhD3tYO+2002ZdBgAAu9y111779621h68d73WYPu2003LNNdfMugwAAHa5qhq7Nr02DwAA6EiYBgCAjoRpAADoSJgGAICOhGkAAOhImAYAgI6EaQAA6KiXYbqqFqrq8MrKyqxLAQBgD+tlmG6tLbbWLpybm5t1KQAA7GG9DNMAALATCNMAANCRMA0AAB0J0wAA0JEwDQAAHQnTAADQkTANAAAdCdMAANCRMA0AAB3tm3UBfXT+wkJWlpfHbpvbvz+XLS5uc0UAAMyCMN3ByvJyrphwKfNDE0I2AAC7jzYPAADoSJgGAICOhGkAAOhImAYAgI6EaQAA6EiYBgCAjoRpAADoSJgGAICOhGkAAOhImAYAgI6EaQAA6EiYBgCAjoRpAADoSJgGAICOhGkAAOhImAYAgI6EaQAA6EiYBgCAjvbNugDYjc5fWMjK8vLYbXP79+eyxcVtrggAmAZhGqZgZXk5V8zNjd12aELIBgD6Z8e0eVTVV1fVr1fV71bVj866HgAAOJ6phumquqSqbq2q69aMn11VN1TVTVV1cZK01q5vrT0vyTOSzE+zLgAA2ArTPjJ9aZKzRweq6qQkr0pyTpIzkpxXVWcMt31fkj9L8j+nXBcAAJywqfZMt9beWVWnrRl+YpKbWms3J0lVvT7JuUne31p7c5I3V9Vbkrx2mrWxeU6qAwC4u1mcgLg/yUdG7h9N8g1VdVaSpyW5b5IrJz25qi5McmGSnHrqqVMrknvqw0l1Aj8AsJ1mEaZrzFhrrV2V5KrjPbm1djjJ4SSZn59vW1oZvdeHwA8A7B6zWM3jaJJTRu4/KsktM6gDAABOyCzC9NVJHltVp1fVfZI8M8mbZ1AHAACckGkvjfe6JO9K8riqOlpVz22t3ZHkoiRvTXJ9kje21t43zToAAGAapr2ax3kTxq/MOicZAgBAH+yYKyBuRlUtVNXhlZWVWZcCAMAe1ssw3VpbbK1dODdh1QYAANgOvQzTAACwE8xinWl2sPUuenJkaSk5eHCbKwIA2LmEae5mvYuenLm6us3VAADsbNo8AACgo16Gaat5AACwE/QyTFvNAwCAnUDPdA+sd1Lg3P79uWxxcZsrAgAgEaZ7Yb2TAg9NCNnJ5BAugAMAbA1hehebFMLXC+AAAGxcL3umAQBgJxCmAQCgo162eVTVQpKFAwcOzLqUTXEiIQDA7tLLMN1aW0yyOD8/f8Gsa9mMricSAgCwM2nzAACAjnp5ZHonu3FpKYcOHhy77cjSUjJhGwAA/SNMb7F9q6sTWznOXF3d5moAAJgmbR4AANCRI9M7hPYQAID+EaZ3iK7tIUI4AMDsCNM9p0cbAGB2etkzXVULVXV4ZWVl1qUAALCH9TJMt9YWW2sXzk04IgsAANuhl2EaAAB2AmEaAAA6EqYBAKAjYRoAADoSpgEAoCNhGgAAOhKmAQCgo16GaRdtAQBgJ+hlmHbRFgAAdoJehmkAANgJhGkAAOhImAYAgI6EaQAA6EiYBgCAjoRpAADoSJgGAICOhGkAAOhImAYAgI6EaQAA6KiXYbqqFqrq8MrKyqxLAQBgD+tlmG6tLbbWLpybm5t1KQAA7GG9DNMAALAT7Jt1AbDX3Li0lEMHD47dNrd/fy5bXNzmigCAroRp2Gb7VldzxYQWpUPLy9tcDQBwIoTpPWi9I6NHlpaSCdsAALg7YXoPWu/I6Jmrq9tcDQBAfzkBEQAAOnJkGjo6f2EhKxN6nLXLAMDeIExDRyvLy9plAGCP0+YBAAAdOTINADOyXruYdeehH4Rp9gwXSwF2mvXaxaw7D/0gTLNnuFgKALDV9EwDAEBHwjQAAHTUyzBdVQtVdXhlZWXWpQAAsIf1Mky31hZbaxfOTeh/BQCA7dDLMA0AADuBMA0AAB0J0wAA0JEwDQAAHQnTAADQkTANAAAduZw4W+LGpaUcOnhw7La5/ftz2eLiNlcEADB9wjRbYt/qaq6YsO73oeXlba4GAGB7aPMAAICOhGkAAOhImAYAgI70TDNT5y8sZGVMT/V2n7ToBEoAoAthmplaWV4ee+Lidp+06ARKAKALbR4AANCRI9NwHJNaQI4sLSUTWkMAgL1BmIbjmNQCcubq6gyqAQB2EmGaqVvv5D5HdwGAPhOmmbr1Tu5zdBcA6DNhmh3JUnUAQB/0MkxX1UKShQMHDsy6FKbEUnUAQB/0cmm81tpia+3CuQlhCwAAtkMvwzQAAOwEwjQAAHQkTAMAQEfCNAAAdCRMAwBAR8I0AAB0JEwDAEBHwjQAAHQkTAMAQEfCNAAAdCRMAwBAR8I0AAB0JEwDAEBHwjQAAHQkTAMAQEf7Zl0AcJcbl5Zy6ODBsdvm9u/PZYuL21wRALAeYRp2kH2rq7libm7stkPLy9tcDQBwPNo8AACgI2EaAAA60uYBcALOX1jIyoQWHH3uJOvvI0eWlpIJ50kA/SBMA5yAleXlbe1zF977Z7195MzV1W2uBthqwjRAj2x3eAdgfXqmAQCgI2EaAAA6EqYBAKAjYRoAADoSpgEAoCNhGgAAOhKmAQCgI2EaAAA6EqYBAKCjHRWmq+pQVf1mVf1+VX3XrOsBAID1TD1MV9UlVXVrVV23Zvzsqrqhqm6qqouTpLV2RWvtgiTPSfID064NAABOxHYcmb40ydmjA1V1UpJXJTknyRlJzquqM0Ye8p+G2wEAYMeaephurb0zye1rhp+Y5KbW2s2ttS8meX2Sc2vgpUn+oLX27mnXBgAAJ2JWPdP7k3xk5P7R4diPJfmOJE+vqueNe2JVXVhV11TVNbfddtv0KwUAgAn2zeh9a8xYa629Iskr1ntia+1wksNJMj8/36ZQGwAAbMisjkwfTXLKyP1HJbllRrUAAEAnszoyfXWSx1bV6UmWkzwzybNmVAuwy5y/sJCV5eWx2+b2789li4vbXBEAu9XUw3RVvS7JWUkeVlVHk7y4tfbqqrooyVuTnJTkktba+6ZdC7A3rCwv54q5ubHbDk0I2QDQxdTDdGvtvAnjVya5ctrvDwAA07KjroC4UVW1UFWHV1ZWZl0KAAB72Kx6pk9Ia20xyeL8/PwFs64FODH6mwHos16GaWD30N8MQJ/1ss0DAAB2AmEaAAA6EqYBAKCjXvZMV9VCkoUDBw7MuhToLSf+AcCJ62WYtpoHnDgn/gHAidPmAQAAHQnTAADQkTANAAAdCdMAANCRMA0AAB31cjUPS+OxF924tJRDBw+O3WYpOwCYjV6GaUvjsRftW121lB0A7DDaPAAAoCNhGgAAOhKmAQCgI2EaAAA6EqYBAKCjXq7mAdzdpGXzLJkHANPVyzBtnWm4u0nL5lkyDwCmq5dtHq21xdbahXMT1twFAIDt0MswDQAAO4EwDQAAHQnTAADQkTANAAAd9XI1D/a2ScvAJcmRpaVkwjY4nvMXFrIyYQUUywwCMI4wTe9MWgYuSc5cXd3mathNVpaXJ+5blhkEYJwNtXlU1TdtZAwAAPaSjfZMv3KDYwAAsGes2+ZRVU9K8o1JHl5VLxjZ9KAkJ02zsPW4AiIAADvB8Y5M3yfJAzMI3SeP/HwqydOnW9pkroAIAMBOsO6R6dbaO5K8o6ouba19eJtqApgaq8EAsJU2uprHfavqcJLTRp/TWvu2aRQFMC1WgwFgK200TP/3JL+e5LeS3Dm9cgAAoD82GqbvaK392lQrAQCAntlomF6sqn+T5PIkXzg22Fq7fSpVAbvOpKsL6lMGoM82GqbPH/7+yZGxluQxW1sOsFtNurqgPmUA+mxDYbq1dvq0CwEAgL7ZUJiuqh8aN95a++2tLQcAAPpjo20eTxi5fb8k357k3UmEaYAtNqm/PNFjDrDTbLTN48dG71fVXJLfmUpFAHvcpP7yZOf0mK8X+Of2789li4vbXBHAbGz0yPRan0vy2K0sZDOqaiHJwoEDB2ZVAsAJ6fvR5/UC/6EJnwtgN9poz/RiBqt3JMlJSb46yRunVdTxtNYWkyzOz89fMKsaAE5EH44+A3B8Gz0y/Usjt+9I8uHW2tEp1AMAAL2x0Z7pd1TVI3LXiYgfnF5JwFa5cWkphya0C3RtJdArCwB32WibxzOSvCzJVUkqySur6idba787xdqAE7RvdXXLWwn0ygLAXTba5vGiJE9ord2aJFX18CR/nESYBgBgz9pomL7XsSA99PEk95pCPUCPTaOtBAB2so2G6T+sqrcmed3w/g8kuXI6JQF9NY22EgDYydYN01V1IMkjWms/WVVPS/LNGfRMvyvJa7ahPgAA2LGO16rx8iSfTpLW2u+11l7QWvvxDI5Kv3y6pQEAwM52vDB9WmvtvWsHW2vXJDltKhUBAEBPHC9M32+dbV+2lYUAAEDfHC9MX11V97hkd1U9N8m10ykJAAD64XireTw/yeVV9ezcFZ7nk9wnyVOnWBcAAOx464bp1trHknxjVX1rkn82HH5La+1Ppl4ZAADscBtaZ7q19vYkb59yLRtWVQtJFg4cODDrUoA9YtIFaVyMBmBv2+hFW3aU1tpiksX5+fl79HMDTMOkC9K4GA3A3tbLMA0ASXL+wkJWlpfHbpvbvz+XLS5uc0XAXiNMA9BbK8vLEy9hf2hCyAbYSsdbGg8AAJhAmAYAgI60eQDsEpNWHNE7DDA9wjTALjFpxRG9wwDTo80DAAA6EqYBAKAjbR7APUzqvU1c8Q8ARgnTwD1M6r1NXPEPAEYJ0wBsqfW+2dgpK4u4ciKwVYRpALbUet9s7JSVRVw5EdgqTkAEAICOHJkGmBIncgLsfsI0wJQ4kRNg9xOmAXa5PpwQCNBXwjTALteHEwIB+soJiAAA0JEwDQAAHQnTAADQUS97pqtqIcnCgQMHZl0KMCPrXcHOsnMAbJdehunW2mKSxfn5+QtmXQswG+tdwc6ycwBsF20eAADQkTANAAAdCdMAANCRMA0AAB0J0wAA0JEwDQAAHQnTAADQkTANAAAdCdMAANCRMA0AAB318nLiADAtNy4t5dDBg/cYn9u/P5ctLs6gImAnE6YBYMS+1dVcMTd3j/FDy8szqAbY6bR5AABAR45MA7ArTWrXSJIjS0vJhG1dXk8LCOxdwjQAu9Kkdo0kOXN1dUtfTwsI7F3aPAAAoCNhGgAAOtLmAbCH6QMGODHCNMAepg8Y4MRo8wAAgI6EaQAA6EibBwBjbfU6zQC7kTANwFhbvU4zwG6kzQMAADoSpgEAoCNtHsCOpWd37zh/YSErE5bis941sJMJ08COpWd371hZXrbeNdBL2jwAAKAjYRoAADoSpgEAoCM90wBsm0knlTqhFOirHROmq+oxSV6UZK619vRZ1wPA1pt0UqkTSoG+mmqbR1VdUlW3VtV1a8bPrqobquqmqro4SVprN7fWnjvNegAAYCtNu2f60iRnjw5U1UlJXpXknCRnJDmvqs6Ych0AALDlphqmW2vvTHL7muEnJrlpeCT6i0len+TcadYBAADTMIvVPPYn+cjI/aNJ9lfVl1fVryd5fFW9cNKTq+rCqrqmqq657bbbpl0rAABMNIsTEGvMWGutfTzJ84735Nba4SSHk2R+fr5tcW0AALBhszgyfTTJKSP3H5XklhnUAQAAJ2QWYfrqJI+tqtOr6j5JnpnkzTOoAwAATsi0l8Z7XZJ3JXlcVR2tque21u5IclGStya5PskbW2vvm2YdAAAwDVPtmW6tnTdh/MokV07zvQEAYNpm0eZxwqpqoaoOr6yszLoUAAD2sF6G6dbaYmvtwrkxl6QFAIDt0sswDQAAO4EwDQAAHQnTAADQkTANAAAdzeJy4iesqhaSLBw4cGDWpQBAblxayqGDB8duO7K0lEzYBvRfL8N0a20xyeL8/PwFs64FAPatruaKCStMnbm6us3VANtJmwcAAHQkTAMAQEfCNAAAdCRMAwBAR8I0AAB01MswXVULVXV4ZWVl1qUAALCH9TJMt9YWW2sXzk1YhggAALZDL8M0AADsBMI0AAB0JEwDAEBHwjQAAHQkTAMAQEfCNAAAdNTLMG2daQAAdoJehmnrTAMAsBP0MkwDAMBOIEwDAEBHwjQAAHQkTAMAQEfCNAAAdCRMAwBAR8I0AAB0JEwDAEBHvQzTroAIAMBO0Msw7QqIAADsBL0M0wAAsBMI0wAA0JEwDQAAHQnTAADQkTANAAAdCdMAANCRMA0AAB0J0wAA0JEwDQAAHQnTAADQ0b5ZF9BFVS0kWThw4MCsSwFgym5cWsqhgwfHbjuytJRM2AawHXoZpltri0kW5+fnL5h1LQBM177V1VwxNzd225mrq9tcDcDdafMAAICOhGkAAOhImAYAgI6EaQAA6EiYBgCAjoRpAADoSJgGAICOhGkAAOhImAYAgI6EaQAA6EiYBgCAjoRpAADoSJgGAICO9s26gC6qaiHJwoEDB2ZdCgBMxY1LSzl08ODYbXP79+eyxcVtrmhvOX9hISvLy/cYN/es1csw3VpbTLI4Pz9/waxrAYBp2Le6mivm5sZuOzQm5LG1VpaXx86/uWctbR4AANCRMA0AAB0J0wAA0JEwDQAAHQnTAADQkTANAAAdCdMAANCRMA0AAB0J0wAA0JEwDQAAHQnTAADQkTANAAAdCdMAANCRMA0AAB0J0wAA0JEwDQAAHQnTAADQkTANAAAdCdMAANCRMA0AAB3tm3UBXVTVQpKFAwcOzLoUANjVzl9YyMry8thtNy8v5zH794/dNrd/fy5bXJxmabuGOe63Xobp1tpiksX5+fkLZl0LAOxmK8vLuWJubuy2Mz/wgYnbDk0Ih9yTOe43bR4AANCRMA0AAB0J0wAA0JEwDQAAHQnTAADQkTANAAAdCdMAANCRMA0AAB0J0wAA0JEwDQAAHQnTAADQkTANAAAdCdMAANCRMA0AAB0J0wAA0JEwDQAAHQnTAADQkTANAAAdCdMAANCRMA0AAB0J0wAA0JEwDQAAHQnTAADQkTANAAAdCdMAANCRMA0AAB0J0wAA0JEwDQAAHQnTAADQkTANAAAdCdMAANDRvlkXcExVPSDJf0nyxSRXtdZeM+OSAABgXVM9Ml1Vl1TVrVV13Zrxs6vqhqq6qaouHg4/LcnvttYuSPJ906wLAAC2wrTbPC5NcvboQFWdlORVSc5JckaS86rqjCSPSvKR4cPunHJdAABwwqYapltr70xy+5rhJya5qbV2c2vti0len+TcJEczCNRTrwsAALbCLHqm9+euI9DJIER/Q5JXJPn/qup7kyxOenJVXZjkwiQ59dRTp1gmAPTP+QsLWVlevsf43P79uWxx4v9et1Ufatxqkz5zkhxZWkoOHtz0a964tJRDY5633jyuV8dunv9pmkWYrjFjrbX22SQ/fLwnt9YOJzmcJPPz822LawOAXltZXs4Vc3P3GD80IUDNQh9q3GqTPnOSnLm62uk1962ubnoe16tjN8//NM2ineJoklNG7j8qyS0zqAMAAE7ILML01UkeW1WnV9V9kjwzyZtnUAcAAJyQaS+N97ok70ryuKo6WlXPba3dkeSiJG9Ncn2SN7bW3jfNOgAAYBqm2jPdWjtvwviVSa6c5nsDAMC09XIJuqpaqKrDKysrsy4FAIA9rJdhurW22Fq7cG7C2agAALAdehmmAQBgJxCmAQCgI2EaAAA6EqYBAKCjXoZpq3kAALAT9DJMW80DAICdoJdhGgAAdgJhGgAAOhKmAQCgI2EaAAA6EqYBAKCjXoZpS+MBALATVGtt1jV0VlW3JfnwDN76YUn+fgbvu9uZ161nTqfDvG49czod5nXrmdPp6MO8Prq19vC1g70O07NSVde01uZnXcduY163njmdDvO69czpdJjXrWdOp6PP89rLNg8AANgJhGkAAOhImO7m8KwL2KXM69Yzp9NhXreeOZ0O87r1zOl09HZe9UwDAEBHjkwDAEBHwvQmVdXZVXVDVd1UVRfPup6+qKpTqurtVXV9Vb2vqv79cPwlVbVcVe8Z/nzPyHNeOJznG6rqu2dX/c5WVUeq6m+H83fNcOyhVfVHVfXB4e+HjDzevK6jqh43sj++p6o+VVXPt69uXlVdUlW3VtV1I2Ob3jer6uuH+/hNVfWKqqrt/iw7xYQ5fVlVfaCq3ltVl1fVg4fjp1XV50f22V8feY45HTFhXjf9d9683mXCnL5hZD6PVNV7huP93ldba342+JPkpCRLSR6T5D5J/ibJGbOuqw8/SR6Z5ODw9slJbkxyRpKXJPmJMY8/Yzi/901y+nDeT5r159iJP0mOJHnYmrFfTHLx8PbFSV5qXjvN7UlJ/neSR9tXO83ftyQ5mOS6kbFN75tJ/irJk5JUkj9Ics6sP9sOm9PvSrJvePulI3N62ujj1ryOOT3+vG7677x5XX9O12z/5SQ/M7zd633VkenNeWKSm1prN7fWvpjk9UnOnXFNvdBa+2hr7d3D259Ocn2S/es85dwkr2+tfaG19qEkN2Uw/2zMuUkuG96+LMmhkXHzunHfnmSptbbexaHM6QSttXcmuX3N8Kb2zap6ZJIHtdbe1Qb/Z/3tkefsOePmtLX2ttbaHcO7f5HkUeu9hjm9pwn76iT21Q1Yb06HR5efkeR1671GX+ZUmN6c/Uk+MnL/aNYPhIxRVacleXySvxwOXTT8evKSka98zfXGtSRvq6prq+rC4dgjWmsfTQb/kEnyj4bj5nVznpm7/8fevnriNrtv7h/eXjvOeP8qg6N3x5xeVX9dVe+oqicPx8zpxm3m77x53bgnJ/lYa+2DI2O93VeF6c0Z16djOZRNqKoHJnlTkue31j6V5NeS/OMkX5fkoxl87ZOY6834ptbawSTnJPm3VfUt6zzWvG5QVd0nyfcl+e/DIfvqdE2aR/O7QVX1oiR3JHnNcOijSU5trT0+yQuSvLaqHhRzulGb/TtvXjfuvNz9QEWv91VhenOOJjll5P6jktwyo1p6p6runUGQfk1r7feSpLX2sdbana21LyX5zdz19bi53qDW2i3D37cmuTyDOfzY8OuxY1+T3Tp8uHnduHOSvLu19rHEvrqFNrtvHs3d2xbM7xhVdX6Sf5Hk2cOvwzNsQ/j48Pa1GfT2flXM6YZ0+DtvXjegqvYleVqSNxwb6/u+KkxvztVJHltVpw+PWj0zyZtnXFMvDPujXp3k+tbar4yMP3LkYU9Ncuys3zcneWZV3beqTk/y2AxOQmBEVT2gqk4+djuDE5Guy2D+zh8+7Pwkvz+8bV437m5HTuyrW2ZT++awFeTTVfXPh/8d+aGR55DBKlNJfjrJ97XWPjcy/vCqOml4+zEZzOnN5nRjNvt33rxu2Hck+UBr7R/aN/q+r+6bdQF90lq7o6ouSvLWDM7yv6S19r4Zl9UX35TkB5P87bGlcJL8xyTnVdXXZfC1zZEkP5IkrbX3VdUbk7w/g68t/21r7c5trrkPHpHk8uFKQfuSvLa19odVdXWSN1bVc5P8XZLvT8zrRlXV/ZN8Z4b749Av2lc3p6pel+SsJA+rqqNJXpzkF7L5ffNHk1ya5Msy6Ace7QneUybM6QszWFnij4b/LfiL1trzMlhN4eeq6o4kdyZ5Xmvt2Alh5nTEhHk9q8PfefM6NG5OW2uvzj3PRUl6vq+6AiIAAHSkzQMAADoSpgEAoCNhGgAAOhKmAQCgI2EaAAA6EqYBdqGqeklV/cSs6wDY7YRpAADoSJgG2CWq6kVVdUNV/XGSxw3HLqiqq6vqb6rqTVV1/6o6uao+VFX3Hj7mQVV1pKruXVX/rqreX1XvrarXz/QDAfSAMA2wC1TV12dwZbHHJ3lakicMN/1ea+0JrbWvTXJ9kue21j6d5Kok3zt8zDOTvKm1tprk4iSPb62dmeR52/gRAHpJmAbYHZ6c5PLW2udaa59K8ubh+D+rqj+tqr9N8uwk/3Q4/ltJfnh4+4eT/Nfh7fcmeU1V/csMLpUMwDqEaYDdo40ZuzTJRa21r0nys0nulySttT9PclpVPSXJSa2164aP/94kr0ry9Umurap9U68aoMeEaYDd4Z1JnlpVX1ZVJydZGI6fnOSjw/7oZ695zm8neV2GR6Wr6l5JTmmtvT3JTyV5cJIHbkPtAL1VrY07kAFA31TVi5L8UJIPJzma5P1JPptBMP5wkr9NcnJr7TnDx39Fkg8leWRr7ZPDwP32JHNJKsl/a639wnZ/DoA+EaYB9qiqenqSc1trPzjrWgD6Si8cwB5UVa9Mck6S75l1LQB95sg0AAB05AREAADoSJgGAICOhGkAAOhImAYAgI6EaQAA6EiYBgCAjv5/Qj1lY+WrP7kAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# visualize with seaborn\n",
    "# histogram of uptime (time since last reboot)\n",
    "\n",
    "plt.figure(figsize=(12, 8))\n",
    "ax = sns.histplot(df_uptime_pandas['days'], kde=False, color='red', bins=range(0, 1800, 20))\n",
    "ax.set_title(\"Histogram of uptime\", fontsize=20)\n",
    "ax.set_yscale('log')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "spark.stop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  },
  "sparkconnect": {
   "bundled_options": [],
   "list_of_options": []
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}