I'm having a similar issue and I'm thankful for this thread; however the additional annotation on my ingress-nginx didn't quite do the trick. Currently my annotations look like this:

      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: /
        kubernetes.io/ingress.class: "nginx"
        nginx.ingress.kubernetes.io/whitelist-source-range: <secret  IPs>
        nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
        nginx.ingress.kubernetes.io/proxy-body-size: 100m
        karpenter.sh/do-not-disrupt: "true"
        nginx.ingress.kubernetes.io/affinity: "cookie"
        nginx.ingress.kubernetes.io/session-cookie-name: "route"
        nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
        nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"

We are using okta for authentication. With a single replica there are no issues. As soon as a 2nd replica is introduced the log in just loops over and over again and I see this message in the logs:

 ERROR:    Exception in ASGI application
 Traceback (most recent call last):
   File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
     self.dialect.do_execute(
   File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 941, in do_execute
     cursor.execute(statement, parameters)
 sqlite3.OperationalError: no such table: group

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: group
[SQL: SELECT "group".id AS group_id, "group".user_id AS group_user_id, "group".name AS group_name, "group".description AS group_description, "group".data AS group_data, "group
FROM "group"
WHERE json_array_length("group".user_ids) > ? AND CAST("group".user_ids AS VARCHAR) LIKE ? ORDER BY "group".updated_at DESC]
[parameters: (0, '%"21d82c5b-6496-40ea-a02d-6d395e9e0e6d"%')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)

Any help would be appreciated. We are using the standard Helm chart.

It looks like your replica isn't using the same database as the other. I would highly recommend not using Sqlite as your application database and use Postgres instead. Otherwise you are going to have to share a persistent volume between your replicas and they'll all be accessing the same backing sqlite database file.

If you have your replicas, postgres, and redis configured correctly. These annotations below are not necessary:

        nginx.ingress.kubernetes.io/affinity: "cookie"
        nginx.ingress.kubernetes.io/session-cookie-name: "route"
        nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
        nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"

Thanks @jk-f5 ... Is there thorough documentation on this? Running multiple replicas seems to be something that should be documented better unless I'm just missing something.

I thought using the EFS storageclass using ReadWriteMany would take care of all that business.

@nwhobart would genuinely appreciate a community contribution for documenting the process here!

@jk-f5 @tjbck I'm now on a postgres db in aws rds and i still get auth errors with multiple replicas. is redis required for multiple replicas?

WARNI [open_webui.utils.oauth] OAuth callback error: mismatching_state: CSRF Warning! State not equal in request and response.

the ui shows:

{"detail":"The email or password provided is incorrect. Please check for typos and try logging in again."}

On a single replica the oauth portion works just fine.

it does work with uploaded files though, but that I think is because the metadata info is stored and retrieved from the DB. Couldn't we do something similar?

Also If I am not mistaken the retrieval of the images there is no check on the user ID, any user could potentially retieve any other user's generated images.

Definitely agree that we need to eliminate the need for a volume mount.

Assuming your second comment is true, you might want to create an issue specifically for that.

@rragundez would that have anything to do with permissions checks needing a tweak as in #8889 & #8483? it might not be multiple replicas

I don't think so, is just how the images are served. They are stored in the cached/images/generations directory which is public and served directly from there in img html components.

We change the source code to use a GCS bucket as intermediary and add it a middleware endpoint to check authentication and served the images from there.
Ideally I think it should follow the same of the files, where the metadata is saved to the DB

I created a PR to solve the issue:
#9486

edit: added ingress

So i think according to this ticket, since we start off before the S3 feature, we still have to have persistence for our uploaded files since there isn't a way (that i know of) to move them to S3: open-webui/helm-charts#83 but that is good to know otherwise.

i have tested again with a .5.7 cluster from the start with a new namespace/ingress so everything is fresh and still have the MIME issue. If i downgrade to .5.4 from .5.7 it works immediately.

Install process:

• helm install psql-test bitnami/postgresql --namespace owui-test
• Get password from configmap and place in values file
• helm upgrade --install open-webui open-webui/open-webui --values .\new-owui-values.yaml --namespace owui-test --set 'extraEnvVars[0].name=SOCKET_LOG_LEVEL' --set 'extraEnvVars[0].value=DEBUG'
• Add ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: owui-test
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: 4096m
spec:
  ingressClassName: nginx
  rules:
    - host: owui-test.mydomain.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: open-webui
                port:
                  number: 80

Here is the helm values file attached, will want to rename to .yaml
new-owui-values - Copy.txt

What you have above works for me.

I did a fresh install using KinD 1.32.1 with exactly what you have above except I first altered your values.yaml to use 0.5.4. I verified it was behaving with 0.5.4, then I modified your values.yaml to change 0.5.4 to 0.5.7 and reran the helm upgrade --install again. Works fine after the upgrade.

If you're interested, I can post exactly what I ran to create my environment from scratch.

This was helpful, i am doing more testing and have made some progress where i believe i am able to replicate what you did in KinD and have it work. I am now trying to diagnose our environmental problems as to what might be the issue. Will keep this updated

Hey @tkg61,
I moved over from local uploads to s3 today. All you need to do is get your s3 setup, copy over the documents from the upload folder to the s3 instance, then update the path for the files in the file table:
UPDATE file SET path = REPLACE(path,'/app/backend/data/uploads/','s3://$BUCKET_NAME/') where path like '/app%';

To copy over the docs, I mounted the volume to a pod with an s3 CLI. You can usually mount the ReadWriteOnce volume at the same time as your openwebui instance if they're running on the same node. I used a podAffinity to get them scheduled together.

This was super helpful. WEBUI_SECRET_KEY was my issue, each instance had a different key because I didn't set it as an env variable in my helm chart.

Kubernetes Version: v1.25.10 +k3s1
CONTENT_EXTRACTION_ENGINE : Tika
DATABASE_URL : postgresql://postgres:PASSWORD@<internal_postgres>:5432/postgres
ENABLE_OLLAMA_API : False
ENABLE_WEBSOCKET_SUPPORT : True
OPENAI_API_BASE_URL : http://<internal_pipelines>:9099
SOCKET_LOG_LEVEL : DEBUG
TIKA_SERVER_URL : http://<internal_tika>:9998
WEBSOCKET_MANAGER : redis
WEBSOCKET_REDIS_URL : redis://<internal_redis>:6379/1
WEBUI_SECRET_KEY :

socket.io (websocket) session management in a multi replica deployment specifically

It is not needed in one instance deployment, right ?
It is needed only in multi instance deployment to share the session in central location, am i understand it right ?

That's correct. Though you could still use Redis in the single replica setup as well.

Thank you 👍

Hi all. Did the above. Expanded my setup to 3 open webui docker nodes behind a load balancer. (HAPROXY)
Enabled the socket handling and setup a redis node in the same compose.yml file. Everything comes up fine. No error messages. If I start on node1 and take it. Down. I’m transferred to node 2 and 3.. etc.. with stickiness by cookie.

this strange thing is, even though I had the env commands for this in all 3 services in the docker-compose.yml file. I couldn’t see any traffic between any of the open webui nodes to the redis in tcpdump -nni any port 6379
How can I verify that redis is infact being used?

Thanks!

@roeizavida Hello, I deployed open-webui with HA using https://github.com/open-webui/helm-charts .

following helm values can help you to make it.

persistence:
  enabled: true
replicaCount: 3
websocket:
  enabled: true
ingress:
  enabled: true
  class: nginx
  annotations:
    nginx.ingress.kubernetes.io/session-cookie-name: "route"
    nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
    nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"

you can see full example values in my cluster in ArgoCD CRD. Please let me know how you tried!

I think you are also running into this issue: #10365 where multiple replicas don’t sync settings.

in the end I actually had to switch clusters then I was able to remove the nginx annotations to have the multiple replicas work. Never figured out why since it was taking too long to figure out

Edit: Oh also make sure you are using the same secret key talked about in this thread

@jyje Thanks! Seems like using Nginx + the annotations you provided solved this.

@roeizavida Is it because you set the sticky session so the one who made configuration change is routed to the same instance that has been updated but other replicas still do not get the synced? I have followed @tkg61 issue and found out that I only set WEBSOCKET_REDIS_URL. Now I have both WEBSOCKET_REDIS_URL and REDIS_URL configured, the configs seem to share between each replica now.