Skip to content

AI Tools

This gallery contains examples of how to use some popular frameworks and libraries for "AI applications", including: semantic search, vector databases, large language models, image classification. The notebooks are intended to be run using GPU resources.
To use GPU resources in SWAN, you need to:

  • Open a ticket with the SWAN team to get access to GPU resources
  • Use SWAN from https://swan-k8s.cern.ch
  • Select a software stack with GPU
  • Note, to get the latest version of the tools used here select the 'bleeding edge' software stack

Open this Gallery in SWAN

Transformers library for text, image, and speech

This is to illustrate the use of the Transformers library from Hugging Face for LLM, Natural Language Processing (NLP), image, and speech tasks.

Large Language Models

These notebooks provide examples of how to use LLMs in notebook environments for tests and prototyping

Semantic search with Vector Databases and LLM

Semantic search allows to query a set of documents. This examples shows how to create vector embeddings, store them in a vector database, and perform semantic queries enhanced with LLM.
The vector database used in this example is Open Search with the k-NN plugin, available as a service for CERN users.