From 0bb756c619309dbc0bd324677e8007133f1b5e58 Mon Sep 17 00:00:00 2001 From: Josh Cowling Date: Tue, 20 Aug 2024 15:39:39 +0100 Subject: [PATCH] updated deployment scripts and readme --- .gitignore | 4 + .../prototype_ollama_example/README.md | 49 ++++++-- .../compose_files/full-rag-docker-compose.yml | 110 ++++++++++++++++++ .../infra-rag-docker-compose.yml | 96 +++++++++++++++ ...iner_and_ollama.sh => dsdl_deploy_full.sh} | 2 +- .../dsdl_deploy_infra.sh | 1 + .../dsdl_stop_environment.sh | 3 +- 7 files changed, 252 insertions(+), 13 deletions(-) create mode 100644 beta_content/passive_deployment_prototypes/prototype_ollama_example/compose_files/full-rag-docker-compose.yml create mode 100644 beta_content/passive_deployment_prototypes/prototype_ollama_example/compose_files/infra-rag-docker-compose.yml rename beta_content/passive_deployment_prototypes/prototype_ollama_example/{dsdl_deploy_dev_container_and_ollama.sh => dsdl_deploy_full.sh} (94%) mode change 100644 => 100755 create mode 100755 beta_content/passive_deployment_prototypes/prototype_ollama_example/dsdl_deploy_infra.sh diff --git a/.gitignore b/.gitignore index b93fc1d..70a3e36 100644 --- a/.gitignore +++ b/.gitignore @@ -117,3 +117,7 @@ testing/config.yaml certificates/*.key certificates/*.pem +# prototype deployment files +beta_content/passive_deployment_prototypes/prototype_ollama_example/compose_files/volumes/ +beta_content/passive_deployment_prototypes/prototype_ollama_example/volumes/ +beta_content/passive_deployment_prototypes/prototype_ollama_example/conf_files/containers.conf diff --git a/beta_content/passive_deployment_prototypes/prototype_ollama_example/README.md b/beta_content/passive_deployment_prototypes/prototype_ollama_example/README.md index 0f51329..72718a2 100644 --- a/beta_content/passive_deployment_prototypes/prototype_ollama_example/README.md +++ b/beta_content/passive_deployment_prototypes/prototype_ollama_example/README.md @@ -1,18 +1,45 @@ -Usage steps to deploy on a docker host: -- Install docker https://docs.docker.com/engine/install/ (Docker desktop not required) -- Install docker-compose https://docs.docker.com/compose/install/ -- Read dsdl_deploy_static_dev_container.sh to ensure you are happy with what we're configuring. -- Run dsdl_deploy_static_dev_container.sh as root or with sufficient privs -- Stop Splunk on the search head -- Copy the contents of ./conf_files/ from the docker host into splunk/etc/apps/mltk-container/local/ on the serch head -- Start Splunk on the search head -- Check the dev container appears with details in the Containers page in the DSDL UI -- Run a simple example to validate that fit/apply works +Scripts in this prototype deployement deploy a DSDL dev container, ollama container and milvus. +You may choose to deploye one, several or all depending on your needs and which example you are tryig to run. + +## Case 1: I am deploying supporting infrastructure (ollama/milvus) into a docker environment which will be used along-side an existing DSDL dev container: + +1. Clone/copy this repo onto the docker host connected to DSDL. +2. Ensure Docker and Docker Compose are installed. + - https://docs.docker.com/engine/install/ + - https://docs.docker.com/compose/install/ +3. Read the infra-only compose file: [infra-rag-docker-compose.yml]() and ensure you understand what you are deploying, you may need to adjust settings or remove the sections related to GPU resources for ollama (if you do not have an nvidia GPU enabled environment) +4. Run ```./dsdl_deploy_infra.sh``` to deploy the supporting infrastructure into your docker environment. + +5. Adjust dev containter network if nessesary: + + - For an existing DSDL container to comminicate with ollama and milvus they must be configured to be deployed in the same network, and this network must not be the default network. By default the compose files in this prototype deploy containers to a network named ```dsenv-network```. + + If you have previously deployed a dsdl container and are not using compose to deploy it here you may need to run: + ``` + docker network connect dsenv-network + ``` + to move the container to the correct network. + +## Case 2: I am deploying a statically deployed DSDL dev container and all supporting infra (ollama/milvus) + +Static deployment of the DSDL dev container is useful in environments where DSDL cannot dynamically deploy containers via the docker management API, perhaps because of security or communication restrictions. + +Deploy a full static environment to a docker host: +1. Install docker https://docs.docker.com/engine/install/ (Docker desktop not required) +2. Install docker-compose https://docs.docker.com/compose/install/ +3. Read dsdl_deploy_full.sh to ensure you are happy with what we're configuring. +4. Run dsdl_deploy_full.sh as root or with sufficient privs +5. Deploy the configuration created by this script in ```./conf_files/*```. + - Stop Splunk on the search head. + - Copy the contents of ```./conf_files/``` from the docker host into ```splunk/etc/apps/mltk-container/local/``` on the serch head. + - Start Splunk on the search head. + - Check the dev container appears with details in the Containers page in the DSDL UI. + - Run a simple example to validate that fit/apply works. Note: - This template uses the default container configuration which uses an insecure certificate, this may or may not be fine for your test environment but should not be used in production. - This template only deploys a dev container, future guidance will detail how to configure dev and prod containers using dynamic templates and deploy them statically in docker and in K8s using HELM. -- If you stop or restart the environment you will need to update the config on the splunk search head +- If you stop or restart the docker containers in the environment you will need to update the config on the splunk search head. Routing requirements: - Splunk search head must be able to access port 5000 or appropriately mapped ingress on the docker host for API communication diff --git a/beta_content/passive_deployment_prototypes/prototype_ollama_example/compose_files/full-rag-docker-compose.yml b/beta_content/passive_deployment_prototypes/prototype_ollama_example/compose_files/full-rag-docker-compose.yml new file mode 100644 index 0000000..d9ffbf1 --- /dev/null +++ b/beta_content/passive_deployment_prototypes/prototype_ollama_example/compose_files/full-rag-docker-compose.yml @@ -0,0 +1,110 @@ +# DSDL Data Science Environment: Docker Compose Configuation +# This notebooks deploys a "static" DSDL environment in docker with no dynamic control needed from the DSDL app +# It contains a DSDL dev container, as well as optional Milvus supporting services +version: '1.0' + +# ------------------------------------------------------------------------------------------------------------------------ # + +# Define all services: Ollama, DSDL Dev + +services: +# DSDL Dev Container + dsdl-dev: + extends: + file: dsdl-base.yml + service: dsdl-base + container_name: dsdl-dev + image: splunk/mltk-container-ubi-llm-rag:5.2.0 + ports: + - 8888:8888 # Jupyter + - 6000:6000 + - 4040:4040 + - 6006:6006 # Tensorboard + environment: + - MODE_DEV_PROD=DEV + + ollama: + container_name: ollama + image: ollama/ollama + ports: + - "11434:11434" + volumes: + - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/ollama:/root/.ollama +# # remove this section to disable GPU binding + deploy: + resources: + reservations: + devices: + - driver: nvidia + capabilities: ["gpu"] + count: all + + etcd: + container_name: milvus-etcd + image: quay.io/coreos/etcd:v3.5.5 + environment: + - ETCD_AUTO_COMPACTION_MODE=revision + - ETCD_AUTO_COMPACTION_RETENTION=1000 + - ETCD_QUOTA_BACKEND_BYTES=4294967296 + - ETCD_SNAPSHOT_COUNT=50000 + volumes: + - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/etcd:/etcd + command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd + healthcheck: + test: ["CMD", "etcdctl", "endpoint", "health"] + interval: 30s + timeout: 20s + retries: 3 + + minio: + container_name: milvus-minio + image: minio/minio:RELEASE.2023-03-20T20-16-18Z + environment: + MINIO_ACCESS_KEY: minioadmin + MINIO_SECRET_KEY: minioadmin + ports: + - "9001:9001" + - "9000:9000" + volumes: + - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/minio:/minio_data + command: minio server /minio_data --console-address ":9001" + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"] + interval: 30s + timeout: 20s + retries: 3 + + standalone: + container_name: milvus-standalone + image: milvusdb/milvus:v2.4.1 + command: ["milvus", "run", "standalone"] + security_opt: + - seccomp:unconfined + environment: + ETCD_ENDPOINTS: etcd:2379 + MINIO_ADDRESS: minio:9000 + volumes: + - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/milvus:/var/lib/milvus + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:9091/healthz"] + interval: 30s + start_period: 90s + timeout: 20s + retries: 3 + ports: + - "19530:19530" + - "9091:9091" + depends_on: + - "etcd" + - "minio" + +# ------------------------------------------------------------------------------------------------------------------------ # +volumes: + mltk-container-data: + mltk-container-app: + mltk-container-notebooks: + +# ------------------------------------------------------------------------------------------------------------------------ # +networks: + default: + name: dsenv-network diff --git a/beta_content/passive_deployment_prototypes/prototype_ollama_example/compose_files/infra-rag-docker-compose.yml b/beta_content/passive_deployment_prototypes/prototype_ollama_example/compose_files/infra-rag-docker-compose.yml new file mode 100644 index 0000000..b9df919 --- /dev/null +++ b/beta_content/passive_deployment_prototypes/prototype_ollama_example/compose_files/infra-rag-docker-compose.yml @@ -0,0 +1,96 @@ +# DSDL Data Science Environment: Docker Compose Configuation +# This notebooks deploys a "static" DSDL environment in docker with no dynamic control needed from the DSDL app +# It contains a DSDL dev container, as well as optional Milvus supporting services +version: '1.0' + +# ------------------------------------------------------------------------------------------------------------------------ # + +# Define all services: Ollama, DSDL Dev + +services: + + ollama: + container_name: ollama + image: ollama/ollama + ports: + - "11434:11434" + volumes: + - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/ollama:/root/.ollama + # remove this section to disable GPU binding + deploy: + resources: + reservations: + devices: + - driver: nvidia + capabilities: ["gpu"] + count: all + + etcd: + container_name: milvus-etcd + image: quay.io/coreos/etcd:v3.5.5 + environment: + - ETCD_AUTO_COMPACTION_MODE=revision + - ETCD_AUTO_COMPACTION_RETENTION=1000 + - ETCD_QUOTA_BACKEND_BYTES=4294967296 + - ETCD_SNAPSHOT_COUNT=50000 + volumes: + - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/etcd:/etcd + command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd + healthcheck: + test: ["CMD", "etcdctl", "endpoint", "health"] + interval: 30s + timeout: 20s + retries: 3 + + minio: + container_name: milvus-minio + image: minio/minio:RELEASE.2023-03-20T20-16-18Z + environment: + MINIO_ACCESS_KEY: minioadmin + MINIO_SECRET_KEY: minioadmin + ports: + - "9001:9001" + - "9000:9000" + volumes: + - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/minio:/minio_data + command: minio server /minio_data --console-address ":9001" + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"] + interval: 30s + timeout: 20s + retries: 3 + + standalone: + container_name: milvus-standalone + image: milvusdb/milvus:v2.4.1 + command: ["milvus", "run", "standalone"] + security_opt: + - seccomp:unconfined + environment: + ETCD_ENDPOINTS: etcd:2379 + MINIO_ADDRESS: minio:9000 + volumes: + - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/milvus:/var/lib/milvus + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:9091/healthz"] + interval: 30s + start_period: 90s + timeout: 20s + retries: 3 + ports: + - "19530:19530" + - "9091:9091" + depends_on: + - "etcd" + - "minio" + +# ------------------------------------------------------------------------------------------------------------------------ # +volumes: + mltk-container-data: + mltk-container-app: + mltk-container-notebooks: + +# ------------------------------------------------------------------------------------------------------------------------ # +networks: + default: + name: dsenv-network diff --git a/beta_content/passive_deployment_prototypes/prototype_ollama_example/dsdl_deploy_dev_container_and_ollama.sh b/beta_content/passive_deployment_prototypes/prototype_ollama_example/dsdl_deploy_full.sh old mode 100644 new mode 100755 similarity index 94% rename from beta_content/passive_deployment_prototypes/prototype_ollama_example/dsdl_deploy_dev_container_and_ollama.sh rename to beta_content/passive_deployment_prototypes/prototype_ollama_example/dsdl_deploy_full.sh index f8a85e4..e43e62d --- a/beta_content/passive_deployment_prototypes/prototype_ollama_example/dsdl_deploy_dev_container_and_ollama.sh +++ b/beta_content/passive_deployment_prototypes/prototype_ollama_example/dsdl_deploy_full.sh @@ -10,7 +10,7 @@ if [ -z "$external_address" ]; then read -p "External Address - Enter the address of your container/environment/ingress relative to your users: " external_address fi -docker compose -f ./compose_files/ollama-docker-compose.yml up --detach +docker compose -f ./compose_files/full-rag-docker-compose.yml up --detach # Timeout timeout=10 diff --git a/beta_content/passive_deployment_prototypes/prototype_ollama_example/dsdl_deploy_infra.sh b/beta_content/passive_deployment_prototypes/prototype_ollama_example/dsdl_deploy_infra.sh new file mode 100755 index 0000000..d89531b --- /dev/null +++ b/beta_content/passive_deployment_prototypes/prototype_ollama_example/dsdl_deploy_infra.sh @@ -0,0 +1 @@ +docker compose -f ./compose_files/infra-rag-docker-compose.yml up --detach \ No newline at end of file diff --git a/beta_content/passive_deployment_prototypes/prototype_ollama_example/dsdl_stop_environment.sh b/beta_content/passive_deployment_prototypes/prototype_ollama_example/dsdl_stop_environment.sh index 47e1121..e02d4c2 100755 --- a/beta_content/passive_deployment_prototypes/prototype_ollama_example/dsdl_stop_environment.sh +++ b/beta_content/passive_deployment_prototypes/prototype_ollama_example/dsdl_stop_environment.sh @@ -1 +1,2 @@ -docker compose -f ./compose_files/minimal-docker-compose.yml down \ No newline at end of file +docker compose -f ./compose_files/full-rag-docker-compose.yml down +docker compose -f ./compose_files/infra-rag-docker-compose.yml down \ No newline at end of file