Explore the AI Singapore SEA-LION model with Chainlit and Ollama

Meet the Cast

• AI Singapore SEA-LION
  • Model: https://ollama.com/aisingapore/llama3-8b-cpt-sea-lionv2.1-instruct
• Chainlit
• Ollama

Note

This project is designed for local environments. Do not run it in production.

Prerequisites

• Docker
  • For the default model, set the memory limit to 6GB or more.
  • If a larger model is used, or if there are other active Docker containers in the environment, increase the memory limit further to take into account their memory requirements.

Getting Started

• Copy .env and update the values, if necessary:

  cp .env.example .env

• Start the services:

  docker compose up

• Pull the SEA-LION model with Ollama:

  docker compose exec ollama ollama pull aisingapore/llama3-8b-cpt-sea-lionv2.1-instruct

• Navigate to http://localhost:8000 to access the chatbot.

Default Model

• The default model is llama3-8b-cpt-sea-lionv2.1-instruct:q4_k_m.
• If you would like to test the other models, choose the model in https://ollama.com/aisingapore/llama3-8b-cpt-sea-lionv2.1-instruct.
  • Check that there is sufficient disk storage and memory. For example, llama3-8b-cpt-sea-lionv2.1-instruct:q8_0 requires 8.5GB of disk storage and 9.3GB of available memory in Docker.
  • Pull the model with Ollama.

    docker compose exec ollama ollama pull aisingapore/llama3-8b-cpt-sea-lionv2.1-instruct:q8_0

  • Update the model name in .env.

    LLM_MODEL=aisingapore/llama3-8b-cpt-sea-lionv2.1-instruct:q8_0
    

Customisations

• Please feel free to fork this repo and customise it.
• Upgrade the Chainlit version in the Dockerfile to support newer features.
• Examples:
  • OAuth
  • Data Persistence
  • Integrations with LangChain or other inference servers

Acknowledgements

• Kudos to the AI Singapore Team for their good work!