{"id":5510,"date":"2025-11-04T14:36:26","date_gmt":"2025-11-04T13:36:26","guid":{"rendered":"https:\/\/kairntech.com\/doc\/?page_id=5510"},"modified":"2025-11-04T14:36:28","modified_gmt":"2025-11-04T13:36:28","slug":"migration-guide-kubernetes","status":"publish","type":"page","link":"https:\/\/kairntech.com\/doc\/migration-guide-kubernetes\/","title":{"rendered":"Migration Guide (kubernetes)"},"content":{"rendered":"\n

The following lines will guide you trough the migration from the version manta<\/code> <\/strong>to the version naja<\/strong><\/code>.<\/p>\n\n\n\n

These notes are based on a kubernetes<\/code> deployment.<\/p>\n\n\n\n

\n

\u2139\ufe0f Note<\/strong>: <\/p>\n\n\n\n

The new naja<\/code> version comes with Elastic Search autentication (in addition to MongoDB authentication).<\/p>\n<\/blockquote>\n\n\n\n

1. Create MongoDB dump (migration 4.0.6 \u2192 7.0.14) – optionnal<\/h2>\n\n\n\n

In order to create a MongoDB dump, a cron job will have to be created.<\/p>\n\n\n\n

Enter the folder kubernetes\/mongodb-backup-restore\/<\/code> and pay attention to file
manifest mongodb-backup-job.yaml<\/code>. <\/p>\n\n\n\n

Specifications may have to be adapted to match the deployed release name:<\/p>\n\n\n\n

    spec:\n      containers:\n        - env:\n            - name: COMMAND\n              value: dump\n            - name: MAX_DUMPS\n              value: \"30\"\n            - name: MONGODB_HOST\n              value: sherpa-mongodb:27017\n            - name: MONGODB_PASSWORD\n              valueFrom:\n                secretKeyRef:\n                  key: MONGODB_PASSWORD\n                  name: mongodb-credentials<\/strong>\n            - name: MONGODB_USERNAME\n              valueFrom:\n                secretKeyRef:\n                  key: MONGODB_USERNAME\n                  name: mongodb-credentials<\/strong>\n          image: kairntech\/mongodb-backup:naja-preview-3\n          imagePullPolicy: Always\n          name: mongodb-backup-job\n          volumeMounts:\n            - mountPath: \/data\n              name: mongodb-dumps<\/strong>\n      imagePullSecrets:\n        - name: kairntech-docker-credentials\n      restartPolicy: Never\n      volumes:\n        - name: mongodb-dumps<\/strong>\n          persistentVolumeClaim:\n            claimName: mongodb-dumps<\/strong><\/code><\/pre>\n\n\n\n
<\/div>\n\n\n\n
Apply the restoration job:<\/p>\n\n\n\n
kubectl apply -f mongodb-backup-job.yaml<\/code><\/pre>\n\n\n\n
<\/div>\n\n\n\n
Check that the dump is being created:<\/p>\n\n\n\n
kubectl logs -f job.batch\/mongodb-backup-job<\/code><\/pre>\n\n\n\n
<\/div>\n\n\n\n
Wait for the end of the dump creation (note the MongoDB dump name):<\/p>\n\n\n\n
kubectl logs -f job.batch\/mongodb-backup-job\n\nRunning: mongodump  --host=sherpa-mongodb:27017 --authenticationDatabase=admin --username=admin --password=<hidden> --gzip --archive=mongodump-sherpa-2025-08-29-08h56_06.gz\n2025-08-29T08:56:07.050+0200\twriting admin.system.users to archive 'mongodump-sherpa-2025-08-29-08h56_06.gz<\/strong>'\n2025-08-29T08:56:07.055+0200\tdone dumping admin.system.users (1 document)\n...\nDump is ready: \/data\/dumps\/mongodump-sherpa-2025-08-29-08h56_06.gz\nRunning: mongosh  --host=sherpa-mongodb:27017 --authenticationDatabase=admin --username=admin --password=<hidden> --eval 'db.adminCommand(\"listDatabases\").databases.sort(function(l, r) {return r.sizeOnDisk - l.sizeOnDisk}).forEach(function(d) {print(d.name + \" - \" + (d.sizeOnDisk\/1024\/1024) + \"M\")});' > mongodump-sherpa-2025-08-29-08h56_06.txt\nDatabase list is ready: \/data\/dumps\/mongodump-sherpa-2025-08-29-08h56_06.txt\nDone<\/code><\/pre>\n\n\n\n
<\/div>\n\n\n\n
Store the dump in your localhost:<\/p>\n\n\n\n
kubectl cp $(kubectl get pods --field-selector=status.phase=Running|grep sherpa-mongodb|awk '{print $1}'):\/data\/dumps\/mongodump-sherpa-2025-08-29-08h56_06.gz mongodump-sherpa-2025-08-29-08h56_06.gz<\/code><\/pre>\n\n\n\n
<\/div>\n\n\n\n
2. Adapt custom-values.yaml<\/strong> file<\/h2>\n\n\n\n
Our helm chart is customizable. Specific parameters have been described in a file called custom-values.yaml<\/strong>, delivered in the archive. In case there are modifications to be applied, this file should be adapted. <\/p>\n\n\n\n
Storage classes<\/h3>\n\n\n\n
The Helm chart defines three storage class aliases: local<\/code><\/strong>, remote<\/code><\/strong>, and shared<\/code><\/strong>.<\/p>\n\n\n\n
    \n
  • local<\/code><\/strong>: Storage class intended for MongoDB and Elasticsearch, which require high-performance storage (ideally node-local).<\/li>\n\n\n\n
  • remote<\/code><\/strong>: Storage class intended for workloads that do not require high performance (e.g., volumes for machine learning models used by suggesters, configuration file volumes, etc.).<\/li>\n\n\n\n
  • shared<\/code><\/strong>: Storage class intended for resources shared across multiple pods. This storage class must support ReadWriteMany (RWX)<\/strong> access mode.<\/li>\n<\/ul>\n\n\n\n
    After selecting your storage classes, you will need to update the following lines in the custom-values.yaml<\/code> file (which currently contain our test values):<\/p>\n\n\n\n
    storageClass:\n  local: local-path\n  remote: longhorn<\/code><\/pre>\n\n\n\n
    In case you do not have an RWX-enabled storage class, the Helm chart provides an option to replace volumes using the storage class alias shared<\/code><\/strong> with emptyDir<\/code> volumes:<\/p>\n\n\n\n
    storage:\n  rwx:\n    available: false<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    CPU versus GPU suggester engines<\/h3>\n\n\n\n
    In case you have suggesters in your pipeline that run on GPU<\/code>, in your environment, you will need to set the gpu<\/code> parameter of the inference suggester to true<\/code> in the custom-values.yaml<\/code> file (our test value is set to false<\/code>):<\/p>\n\n\n\n
    suggester:\n  flair:\n#    training:\n#      image:\n#        gpu: false\n    inference:\n      image:\n        gpu: true<\/strong><\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Expose a NodePort<\/h3>\n\n\n\n
    If you want to expose a NodePort<\/strong> for the Sherpa server, you can add the following option:<\/p>\n\n\n\n
    sherpa:\n  svc:\n    type: NodePort\n    nodePort: 30010 # example\n<\/code><\/pre>\n\n\n\n
    \n
    \u2139\ufe0f Note<\/strong>:<\/p>\n\n\n\n
    You can do the same for all other services (see the various svc<\/code> keys in config.yaml<\/code>).<\/p>\n<\/blockquote>\n\n\n\n
    <\/div>\n\n\n\n
    Passwords<\/h3>\n\n\n\n
    Random passwords will be generated for MongoDB<\/strong> and Elasticsearch<\/strong>.
    The Sherpa admin password will default to secret<\/code><\/strong>, unless you add a key in a custom-secrets.yaml<\/code> file:<\/p>\n\n\n\n
    SHERPA_ADMIN_PASSWORD: \"your password\"<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    3. Install naja version<\/h2>\n\n\n\n
    Install via Helm chart<\/p>\n\n\n\n
    To deploy the naja<\/strong> version, first extract the provided .tgz<\/code> package:<\/p>\n\n\n\n
    mkdir sherpa\ncd sherpa\/\ntar xfz ..\/sherpa.naja.tgz<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Then run the installation with:<\/p>\n\n\n\n
    helm install kairntech helm -f custom-values.yaml -f custom-secrets.yaml<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Check that all is running<\/p>\n\n\n\n
    kubectl get all<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    \n
    \u2139\ufe0f Note<\/strong>:<\/p>\n\n\n\n
    Note that the system is not yet operational: the pods sherpa-entityfishing<\/code><\/strong>, sherpa-fasttext-test-suggester<\/code><\/strong>, sherpa-fasttext-train-suggester<\/code><\/strong>, sherpa-flair-test-suggester<\/code><\/strong>, and sherpa-flair-train-suggester<\/code><\/strong> remain in Init<\/strong> status because their init containers are waiting for resources.<\/p>\n\n\n\n
    The rsync server that will manage resources is exposed on NodePort 31873.<\/p>\n<\/blockquote>\n\n\n\n
    <\/div>\n\n\n\n
    Init-containers to fetch resources<\/h3>\n\n\n\n
    The init-containers should be waiting for files availability; you can check with the following commands:<\/p>\n\n\n\n
    FastText:<\/p>\n\n\n\n
    kubectl logs $(kubectl get pods -o=name|grep sherpa-fasttext-train-suggester) -c fasttext-suggester-resources-init-job<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Flair:<\/p>\n\n\n\n
    kubectl logs $(kubectl get pods -o=name|grep sherpa-flair-train-suggester) -c flair-suggester-resources-init-job<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Entity-Fishing:<\/p>\n\n\n\n
    kubectl logs $(kubectl get pods -o=name|grep sherpa-entityfishing|grep -v suggester) -c entity-fishing-resources-init-job<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Fetching online resources<\/h3>\n\n\n\n
    This step must be performed on a machine with internet access<\/strong>, and a Docker installation<\/strong>.<\/p>\n\n\n\n
    Extract the tgz archive<\/h4>\n\n\n\n
    mkdir sherpa\ncd sherpa\/\ntar xfz ..\/sherpa.naja.tgz<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Download resources<\/h4>\n\n\n\n
    Log in to Docker with your Kairntech credentials<\/strong>, then run:<\/p>\n\n\n\n
    cd kubernetes\/init\/\n.\/download-resources.sh\n<\/code><\/pre>\n\n\n\n
    Wait until the script finishes executing.
    This script retrieves data stored in a Kairntech Amazon S3 bucket.<\/p>\n\n\n\n
    <\/div>\n\n\n\n
    Import resources into Sherpa<\/h4>\n\n\n\n
    Run the following commands:<\/p>\n\n\n\n
    cd resources-mirror\n.\/upload-to-mirror.sh<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Completion of the script and resource initialization<\/h3>\n\n\n\n
    The script ends by copying a _ready<\/code><\/strong> file, signaling to the init containers that the resources are ready:<\/p>\n\n\n\n
    The resource init containers can now start downloading data from the mirror and extracting it into the volumes. You can monitor the operations with:<\/p>\n\n\n\n
    FastText:<\/p>\n\n\n\n
    kubectl logs $(kubectl get pods -o=name | grep sherpa-fasttext-train-suggester) -c fasttext-suggester-resources-init-job<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Flair:<\/p>\n\n\n\n
    kubectl logs $(kubectl get pods -o=name | grep sherpa-flair-train-suggester) -c flair-suggester-resources-init-job<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Entity-Fishing<\/p>\n\n\n\n
    kubectl logs $(kubectl get pods -o=name | grep sherpa-entityfishing | grep -v suggester) -c entity-fishing-resources-init-job<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Finalizing the installation<\/h3>\n\n\n\n
    Check that the init containers have completed successfully: all pods should now be in Running<\/strong> status:<\/p>\n\n\n\n
    kubectl get pods<\/code><\/pre>\n\n\n\n
    Sherpa is now ready to operate.<\/p>\n\n\n\n
    <\/div>\n\n\n\n
    4. Restoring a MongoDB dump with the latest projects – optionnal<\/h2>\n\n\n\n
    This step can be replaced by importing multiple SPA project archives.<\/p>\n\n\n\n
    Copy the MongoDB dump into the MongoDB pod<\/h4>\n\n\n\n
    kubectl cp mongodump-sherpa-2025-08-29-08h56_06.gz<\/strong> \\\n$(kubectl get pods --field-selector=status.phase=Running | grep sherpa-mongodb | awk '{print $1}'):\/data\/dumps<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Navigate to the MongoDB backup\/restore directory<\/h4>\n\n\n\n
    Return to the archive extraction directory and go to kubernetes\/mongodb-backup-restore<\/code>:<\/p>\n\n\n\n
    cd kubernetes\/mongodb-backup-restore\/<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Apply the restore job<\/h4>\n\n\n\n
    kubectl apply -f mongodb-restore-projects-only-job.yaml<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Wait for the restore job to complete<\/h4>\n\n\n\n
    kubectl logs -f job.batch\/mongodb-restore-projects-only-job<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Restart sherpa-core<\/code><\/h4>\n\n\n\n
    kubectl rollout restart deployment.apps\/$(kubectl get deployments | grep sherpa-core | awk '{print $1}')<\/code><\/pre>\n\n\n\n
    <\/div>\n\n\n\n
    Monitor sherpa-core<\/code> logs<\/h4>\n\n\n\n
    Watch the logs until all projects are deployed and indexed:<\/p>\n\n\n\n
    kubectl logs -f deployment.apps\/$(kubectl get deployments | grep sherpa-core | awk '{print $1}') | grep \"Projects deployment complete\"<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"
    The following lines will guide you trough the migration from the version manta to the version naja. These notes are based on a kubernetes deployment. \u2139\ufe0f Note: The new naja version comes with Elastic Search autentication (in addition to MongoDB authentication). 1. Create MongoDB dump (migration 4.0.6 \u2192 7.0.14) – optionnal In order to create […]<\/p>\n","protected":false},"author":9,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-5510","page","type-page","status-publish","hentry"],"yoast_head":"\nMigration Guide (kubernetes) - Kairntech Documentation<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/kairntech.com\/doc\/migration-guide-kubernetes\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Migration Guide (kubernetes) - Kairntech Documentation\" \/>\n<meta property=\"og:description\" content=\"The following lines will guide you trough the migration from the version manta to the version naja. These notes are based on a kubernetes deployment. \u2139\ufe0f Note: The new naja version comes with Elastic Search autentication (in addition to MongoDB authentication). 1. Create MongoDB dump (migration 4.0.6 \u2192 7.0.14) – optionnal In order to create […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/kairntech.com\/doc\/migration-guide-kubernetes\/\" \/>\n<meta property=\"og:site_name\" content=\"Kairntech Documentation\" \/>\n<meta property=\"article:modified_time\" content=\"2025-11-04T13:36:28+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/kairntech.com\/doc\/migration-guide-kubernetes\/\",\"url\":\"https:\/\/kairntech.com\/doc\/migration-guide-kubernetes\/\",\"name\":\"Migration Guide (kubernetes) - Kairntech Documentation\",\"isPartOf\":{\"@id\":\"https:\/\/kairntech.com\/doc\/#website\"},\"datePublished\":\"2025-11-04T13:36:26+00:00\",\"dateModified\":\"2025-11-04T13:36:28+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/kairntech.com\/doc\/migration-guide-kubernetes\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/kairntech.com\/doc\/migration-guide-kubernetes\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/kairntech.com\/doc\/migration-guide-kubernetes\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/kairntech.com\/doc\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Migration Guide (kubernetes)\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/kairntech.com\/doc\/#website\",\"url\":\"https:\/\/kairntech.com\/doc\/\",\"name\":\"Kairntech Documentation\",\"description\":\"All the information you need to use Kairntech Software, methodology,  user and installation guides.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/kairntech.com\/doc\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-GB\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Migration Guide (kubernetes) - Kairntech Documentation","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/kairntech.com\/doc\/migration-guide-kubernetes\/","og_locale":"en_GB","og_type":"article","og_title":"Migration Guide (kubernetes) - Kairntech Documentation","og_description":"The following lines will guide you trough the migration from the version manta to the version naja. These notes are based on a kubernetes deployment. \u2139\ufe0f Note: The new naja version comes with Elastic Search autentication (in addition to MongoDB authentication). 1. Create MongoDB dump (migration 4.0.6 \u2192 7.0.14) – optionnal In order to create […]","og_url":"https:\/\/kairntech.com\/doc\/migration-guide-kubernetes\/","og_site_name":"Kairntech Documentation","article_modified_time":"2025-11-04T13:36:28+00:00","twitter_card":"summary_large_image","twitter_misc":{"Estimated reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/kairntech.com\/doc\/migration-guide-kubernetes\/","url":"https:\/\/kairntech.com\/doc\/migration-guide-kubernetes\/","name":"Migration Guide (kubernetes) - Kairntech Documentation","isPartOf":{"@id":"https:\/\/kairntech.com\/doc\/#website"},"datePublished":"2025-11-04T13:36:26+00:00","dateModified":"2025-11-04T13:36:28+00:00","breadcrumb":{"@id":"https:\/\/kairntech.com\/doc\/migration-guide-kubernetes\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/kairntech.com\/doc\/migration-guide-kubernetes\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/kairntech.com\/doc\/migration-guide-kubernetes\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/kairntech.com\/doc\/"},{"@type":"ListItem","position":2,"name":"Migration Guide (kubernetes)"}]},{"@type":"WebSite","@id":"https:\/\/kairntech.com\/doc\/#website","url":"https:\/\/kairntech.com\/doc\/","name":"Kairntech Documentation","description":"All the information you need to use Kairntech Software, methodology,  user and installation guides.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/kairntech.com\/doc\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-GB"}]}},"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/kairntech.com\/doc\/wp-json\/wp\/v2\/pages\/5510","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kairntech.com\/doc\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/kairntech.com\/doc\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/kairntech.com\/doc\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/kairntech.com\/doc\/wp-json\/wp\/v2\/comments?post=5510"}],"version-history":[{"count":12,"href":"https:\/\/kairntech.com\/doc\/wp-json\/wp\/v2\/pages\/5510\/revisions"}],"predecessor-version":[{"id":5659,"href":"https:\/\/kairntech.com\/doc\/wp-json\/wp\/v2\/pages\/5510\/revisions\/5659"}],"wp:attachment":[{"href":"https:\/\/kairntech.com\/doc\/wp-json\/wp\/v2\/media?parent=5510"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}