Migration Guide (docker) - Kairntech Documentation

The following lines will guide you trough the migration from the version manta to the version naja.

These notes are based on a docker-compose deployment.

ℹ️ Note:

The main differences reside in new environment files, instead of a single .env file.

The new naja version also comes with Elastic Search authentication (in addition to MongoDB authentication).

1. Backup

sudo su
cp -rf /opt/sherpa /opt/sherpa_bck/

2. Create MongoDB dump (migration 4.0.6 → 7.0.14)

If MongoDB with authentication:

MONGODUMP=mongodump.gz
docker exec -i sherpa-mongodb rm "${MONGODUMP}"
docker exec -i sherpa-mongodb mongodump -uadmin -p**** --gzip --quiet --archive="${MONGODUMP}"
DATENOW=$( date +%Y%m%d )
docker cp sherpa-mongodb:"${MONGODUMP}" mongodump."${DATENOW}".gz

If MongoDB without authentication:

MONGODUMP=mongodump.gz
docker exec -i sherpa-mongodb rm "${MONGODUMP}"
docker exec -i sherpa-mongodb mongodump --gzip --quiet --archive="${MONGODUMP}"
DATENOW=$( date +%Y%m%d )
docker cp sherpa-mongodb:"${MONGODUMP}" mongodump."${DATENOW}".gz

3. Stop Sherpa

cd /opt/sherpa
docker compose -f docker-compose.yml down

4. Remove volumes

Logs:

for VOL in $(docker volume ls|grep sherpa|grep log|awk -F' ' '{print $2}') ; do docker volume rm $VOL ; done

Elasticsearch:

docker volume rm sherpa_esdata

MongoDB:

docker volume rm sherpa_mongo

JWT:

rm -rf /opt/sherpa/sherpa-core/jwt

Grafana (if deployed):

docker volume rm sherpa_grafana

5. Purge deployed files

rm -rf /opt/sherpa/*

6. Deploy new `docker-compose.yml`

cp **/docker-compose.yml /opt/sherpa/docker-compose.yml

7. Setup `.env` files

SHERPA_INSTALL='/opt/sherpa'
for ENV in $(cat "${SHERPA_INSTALL}/docker-compose.yml"|grep '\.env'|grep -v '#'|sort -n|uniq|awk -F '- ' '{print $2}') ; do echo "touch ${SHERPA_INSTALL}/${ENV}" ; done

ℹ️ Required:

cd "${SHERPA_INSTALL}"
ln -s mongodb-credentials.env .env
cat elasticsearch-credentials.env >> .env

👉 Here the content of the new environment files:

apollo-client.env
APOLLO_API=''
APOLLO_OAUTH=''

apollo-credentials.env
APOLLO_CLIENT_ID=''
APOLLO_CLIENT_SECRET=''

azure-openai-client.env
AZURE_OPENAI_API_BASE=''
AZURE_OPENAI_DEPLOYMENT_ID=''
AZURE_OPENAI_API_VERSION=''

azure-openai-credentials.env
AZURE_OPENAI_API_KEY=''

azure-sso-credentials.env
OAUTH_AZURE_AD_CLIENT_ID=''
OAUTH_AZURE_AD_CLIENT_SECRET=''

azure-sso.env
CHAINLIT_URL=''
OAUTH_AZURE_AD_TENANT_ID=''
OAUTH_AZURE_AD_ENABLE_SINGLE_TENANT=true

chatbot.env
SHERPA_URL=''
CHAINLIT_PORT= 
CHAT_ROOT_PATH='/chatbot'

deepinfra-client.env
DEEPINFRA_OPENAI_API_BASE='https://api.deepinfra.com/v1/openai'

deepinfra-credentials.env
DEEPINFRA_OPENAI_API_KEY=''

deepl-credentials.env
DEEPL_API_KEY=''

deeptranscript-credentials.env
DT_API_KEY=''

elasticsearch-client.env
ELASTICSEARCH_INDEX_PREFIX=

elasticsearch-credentials.env
ELASTICSEARCH_USERNAME='elastic'
ELASTICSEARCH_PASSWORD=''

grafana.env
GF_SERVER_ROOT_URL=''

huggingface-credentials.env
HF_API_TOKEN=''

literalai-credentials.env
LITERAL_API_KEY=''

mistral-credentials.env
MISTRAL_API_KEY=''

mongodb-client.env
MONGODB_DATABASE_PREFIX=

mongodb-credentials.env
MONGODB_USERNAME='admin'
MONGODB_PASSWORD=''

ncbi-credentials.env
NCBI_API_KEY=''

openai-credentials.env
OPENAI_API_KEY=''

rabbitmq-credentials.env
RABBITMQ_USERNAME='sherpa'
RABBITMQ_PASSWORD=''

sherpa-client.env
SHERPA_CONTEXT_PATH=

sherpa-credentials.env
SHERPA_ADMIN_PASSWORD=''

8. Copy required directories

cp -rf **/sherpa-chat/ /opt/sherpa
cp -rf **/sherpa-core/ /opt/sherpa
cp -rf **/sherpa-elasticsearch/ /opt/sherpa
cp -rf **/sherpa-rabbitmq/ /opt/sherpa
cp -rf **/sherpa-haproxy/ /opt/sherpa

cp -rf **/sherpa-filebeat/ /opt/sherpa
cp -rf **/sherpa-prometheus/ /opt/sherpa
cp -rf **/sherpa-grafana/ /opt/sherpa

9. Update passwords

RabbitMQ:

docker exec -i sherpa-rabbitmq rabbitmqctl hash_password <mypassword>
vi /opt/sherpa/sherpa-rabbitmq/definitions/02-kairntech-users.json

Grafana (if deployed):

vi /opt/sherpa/sherpa-grafana/conf/.passwd-grafana-admin
vi /opt/sherpa/sherpa-grafana/conf/.passwd-grafana-smtp

10. Download Docker images

cd /opt/sherpa
docker compose -f docker-compose.yml pull

11. Start Docker images

cd /opt/sherpa
docker compose -f docker-compose.yml up -d

12. Restore MongoDump