AI Tools
This gallery contains examples of how to use some popular frameworks and libraries for "AI applications",
including: semantic search, vector databases, large language models, image classification.
The notebooks are intended to be run using GPU resources.
To use GPU resources in SWAN, you need to:
- Access SWAN from you browser: https://swan.cern.ch
- Select a software stack with GPU
- Note, to get the latest version of the tools used here select the 'bleeding edge' software stack
Transformers library for text, image, and speech
This is to illustrate the use of the Transformers library from Hugging Face for LLM, Natural Language Processing (NLP), image, and speech tasks.
Transformers for text classification
Transformers for image classifier
Stable diffusion with transformers
Transformers for speech recognition
Large Language Models
These notebooks provide examples of how to use LLMs in notebook environments for tests and prototyping
Semantic search with Vector Databases and LLM
Semantic search allows to query a set of documents. This examples shows how to create
vector embeddings, store them in a vector database, and perform semantic queries enhanced with LLM.
The vector database used in this example is Open Search with the k-NN plugin, available as a service for CERN users.
