Deleting a tenant or migrating a tenant-only model change causes errors

[davidoh14]

I am able to create tenants and access subdomained django admins just fine, but the errors I'm running into make me think something is wrong with the non-shared side.

Deleting a tenant by the command 'delete_tenant' causes the error in the codeblock below. The schema actually gets deleted in postgres (I am no longer able to see it in pgAdmin), but django does not recognize these changes and shows the model, domain, and schema as existing still.

The only way I can avoid this is by using the python shell, getting the schema_context, deleting the tenant without making auto_drop_schema to true, then deleting the schema itself by executing a connection.cursor SQL command.

I then tried the latter flow to delete, but this time with auto_drop_schema set to true, but that errored out.

Traceback (most recent call last):
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UndefinedTable: relation "websites_website" does not exist
LINE 1: ...e"."active", "websites_website"."created_at" FROM "websites_...
                                                             ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/davidoh/Workspace/trail_api/manage.py", line 26, in <module>
    main()
  File "/Users/davidoh/Workspace/trail_api/manage.py", line 22, in main
    execute_from_command_line(sys.argv)
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/core/management/__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/core/management/base.py", line 402, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/core/management/base.py", line 448, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django_tenants/management/commands/delete_tenant.py", line 25, in handle
    tenant.delete()
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django_tenants/models.py", line 162, in delete
    super().delete(*args, **kwargs)
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/db/models/base.py", line 1117, in delete
    collector.collect([self], keep_parents=keep_parents)
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/db/models/deletion.py", line 343, in collect
    if sub_objs:
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/db/models/query.py", line 408, in __bool__
    self._fetch_all()
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/db/models/query.py", line 1867, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/db/models/query.py", line 87, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1398, in execute_sql
    cursor.execute(sql, params)
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/db/backends/utils.py", line 102, in execute
    return super().execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/Users/davidoh/virtualenvs/trail_api/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.ProgrammingError: relation "websites_website" does not exist
LINE 1: ...e"."active", "websites_website"."created_at" FROM "websites_...

The schema gets deleted in pgAdmin, but the tenant model instance and domain still remain in django. When I run delete_tenant again and list out the schemas, the schema is returned in the list of results, despite not showing in pgAdmin.

Why does deleting a tenant in the public schema cause django to look for a table that should only exist in non-shared schemas? I initially suspected that it may be because the website model foreign key is set to cascade deletion from the tenant model, but I tested set_null and still ran into the same issue.

settings.py:

from pathlib import Path
import os
import sys
from .custom_filters import SkipStaticFiles

ALLOWED_HOSTS = [********]

CORS_ALLOWED_ORIGINS = [*********]

CORS_ALLOW_METHODS = [
    'GET',
    'OPTIONS',
    'POST',
]

CORS_ALLOW_HEADERS = [
    'accept',
    'accept-encoding',
    'authorization',
    'content-type',
    'origin',
    'user-agent',
    'x-api-key',  # add your custom header here
    'x-csrftoken',
]

# Application definition

SHARED_APPS = [
    'django_tenants',
    'whitenoise.runserver_nostatic', # keep this at top
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',

    # third party packages
    'rest_framework',
    'corsheaders',

    # internal apps
    'companies',
]

TENANT_APPS = (
    'whitenoise.runserver_nostatic', # keep this at top
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',

    # third party packages
    'rest_framework',
    'corsheaders',
    
    #internal apps
    'apps.events',
    'apps.websites',
)

INSTALLED_APPS = list(SHARED_APPS) + [app for app in TENANT_APPS if app not in SHARED_APPS]

TENANT_MODEL = "companies.Company"

TENANT_DOMAIN_MODEL = "companies.Domain"

SHOW_PUBLIC_IF_NO_TENANT_FOUND = True

MIDDLEWARE = [
    'django_tenants.middleware.main.TenantMainMiddleware',
    'corsheaders.middleware.CorsMiddleware',
    'django.middleware.security.SecurityMiddleware',
    'whitenoise.middleware.WhiteNoiseMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'trail_api.urls'
PUBLIC_SCHEMA_URLCONF = 'trail_api.urls_public'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'trail_api.wsgi.application'


# Database
# https://docs.djangoproject.com/en/4.1/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django_tenants.postgresql_backend',
        'NAME': os.environ.get('DB_NAME'),
        'USER': os.environ.get('DB_USER'),
        'PASSWORD': os.environ.get('DB_PW'),
        'HOST': os.environ.get('DB_HOST'),
        'PORT': os.environ.get('DB_PORT'),
    }
}

DATABASE_ROUTERS = (
    'django_tenants.routers.TenantSyncRouter',
)


# Password validation
# https://docs.djangoproject.com/en/4.1/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


# Internationalization
# https://docs.djangoproject.com/en/4.1/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_TZ = True


STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATICFILES_STORAGE = "whitenoise.storage.CompressedManifestStaticFilesStorage"

DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'

requirements.txt:

asgiref==3.6.0
certifi==2022.12.7
charset-normalizer==3.1.0
Django==4.1.7
django-cors-headers==3.14.0
django-dotenv==1.4.2
django-tenants==3.4.8
djangorestframework==3.14.0
gunicorn==20.1.0
idna==3.4
psycopg2==2.9.6
pytz==2022.7.1
requests==2.28.2
sqlparse==0.4.3
urllib3==1.26.15
uuid==1.30
whitenoise==6.4.0

[HansvanHespen]

Got the same problem myself. Keep on stumbling on issues with django-tenants: cannot use subfolders, cannot use asgi, and cannot delete tenants. Python is supposed to be the most popular language of today, but I wonder what these people then do with it, if basic things don't work? I spend 30% of time programming, and 70% debugging errors of someone else.

In my case the delete function of my model is called. Since there isn't any, it calls the delete function of the inherited models.py of django_tenants. It then drops the schema, allright. From there is goes super().delete. There it searches correctly for csb_tenants_domain. Then it tries to do a select from a table in the schema that doesn't exist anymore, and looks for that table in the public schema, where it never existed of course.

I wonder why it still does a select, in stead of immediately doing a delete of that particular row in the csb_tenants_domain table. Then, I ask myself, why would this result in a hard unhandled exception?

Well, I will keep you posted if I have found anything. I have to admit that I start wondering if I shouldn't go back to delphi or c-sharp...

[HansvanHespen]

I found it!

And it turns out I can't blame this one on bad coding of django-tenants. Honest is honest, this was my mistake.

I had coded my class in the models.py of my tenant app like this:
class csbguest(TenantMixin):
# type: csbtenants
sbtenant = models.ForeignKey(csbtenants, on_delete=models.cascade, related_name='guests')

I was ill advised, but there you are. It makes sense to make that you want a foreign key if you want to link this class of the tenant to the list of tenants in the public schema. And then, it also made sense to me that you do the on_delete in cascading.

What happens though, is that first the schema is deleted, and then the tenant is deleted. The deletion of the denant goes first through the delete_tenant.py of django_tenants, where a few checks are being done, and then it is being sent to the core of django, who doesn't know anything about schema's. And django does it job and goes looking for this table, of course in the public schema since it doesn't know any better, where it doesn't find it of course.

By changing the line into this:
csbtenant = models.ForeignKey(csbtenants, on_delete=models.DO_NOTHING, related_name='guests')

@davidoh14 tried with set_null and that didn't help for him. However, using the DO_NOTHING did the trick for me.

Now it thoroughly delets the row of the specific tenant in the public schema, it deletes the line of the domain in the public schema, drops the schema of the tenant, and with that all of the tables inside of course.

Hence, if you create a class in the models.py of a tenant, use the DO_NOTHING, but if you create a class in the app of the public schema (client app in the example of Tom, and customers class) then you have to create it with the on_delete to CASCADE, because then they will not be automatically dropped by dropping the schema.

So, there you are. It took me a couple of hours, but I learned a lot about how Django, the ORM and django_tenants work.

Problem solved, what a great feeling!

[benshaji-sequoiaat]

I found it!

And it turns out I can't blame this one on bad coding of django-tenants. Honest is honest, this was my mistake.

I had coded my class in the models.py of my tenant app like this: class csbguest(TenantMixin): # type: csbtenants sbtenant = models.ForeignKey(csbtenants, on_delete=models.cascade, related_name='guests')

I was ill advised, but there you are. It makes sense to make that you want a foreign key if you want to link this class of the tenant to the list of tenants in the public schema. And then, it also made sense to me that you do the on_delete in cascading.

What happens though, is that first the schema is deleted, and then the tenant is deleted. The deletion of the denant goes first through the delete_tenant.py of django_tenants, where a few checks are being done, and then it is being sent to the core of django, who doesn't know anything about schema's. And django does it job and goes looking for this table, of course in the public schema since it doesn't know any better, where it doesn't find it of course.

By changing the line into this: csbtenant = models.ForeignKey(csbtenants, on_delete=models.DO_NOTHING, related_name='guests')

@davidoh14 tried with set_null and that didn't help for him. However, using the DO_NOTHING did the trick for me.

Now it thoroughly delets the row of the specific tenant in the public schema, it deletes the line of the domain in the public schema, drops the schema of the tenant, and with that all of the tables inside of course.

Hence, if you create a class in the models.py of a tenant, use the DO_NOTHING, but if you create a class in the app of the public schema (client app in the example of Tom, and customers class) then you have to create it with the on_delete to CASCADE, because then they will not be automatically dropped by dropping the schema.

So, there you are. It took me a couple of hours, but I learned a lot about how Django, the ORM and django_tenants work.

Problem solved, what a great feeling!

I'm having the same issue now. When i try to delete a tenant using ./migrate delete_tenant this issue "relation "app_table" does not exist". I believe this has started when I've added Tenant model as FK in other app's model as:

customer = models.ForeignKey(Tenant, on_delete=models.CASCADE, related_name='customer_site')

where Tenant is the actual Tenant model itself.

[benshaji-sequoiaat]

Suppose: If your Tenant model is used by Student model as FK.
i.e models.ForeignKey(Tenant, on_delete=models.CASCADE, related_name='students') << THIS WILL cause the issue due to reasons @HansvanHespen suggested above and should be models.DO_NOTHING