Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix index collision in 0008_gfk_index migration #458

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

broxeph
Copy link

@broxeph broxeph commented Oct 30, 2019

When schedule migrations are run for the first time, the following exception occurs for migration 0008_gfk_index: psycopg2.errors.DuplicateTable: relation "schedule_calendar_slug_ba17e861_like" already exists.

This appears to be due to Django attempting to set the _like index after adding unique=True to the Calendar.slug field, for which db_index=True was already set due to being a SlugField. The initial migration created the _like index, and making an already indexed field unique causes the index collision.

This is the best fix I could come up with. I'll continue investigating and potentially file a bug with Django if it turns out to be trivially reproducible.

@broxeph
Copy link
Author

broxeph commented Oct 30, 2019

@chrisspen I believe this now fully addresses the problem you reported in #457. I think the root cause of that problem lies more with the index collision than with migration dependency resolution.

@coveralls
Copy link

coveralls commented Oct 30, 2019

Coverage Status

Coverage remained the same at 79.902% when pulling 249886f on broxeph:patch-1 into f80ed3b on llazzaro:develop.

@llazzaro
Copy link
Owner

llazzaro commented Oct 31, 2019

test were done against sqlite3 and postgresql migrations were successful using empty databases
could you tell us which database engine are you using?

@chrisspen
Copy link

We're using PostgreSQL, although non-empty. Sqlite3 sometimes allows subtle errors like this because it doesn't support a lot of features other databases have.

@broxeph
Copy link
Author

broxeph commented Oct 31, 2019

Yeah, after further testing last night I'm not 100% sure this is a you problem... might be an us problem, since I was unable to reproduce it with a new model created in Postgres with db_index=True, and then migrated to unique=True—Django correctly drops the indexes created for db_index=True, and then creates unique indexes (indices?) as expected.

I'm now tempted to close this PR and override the migration in our project with the DROP INDEX IF EXISTS stuff, since it might result instead from some arcane fixture loading/migration dependency thing that's not inherent to the django-scheduler project.

Will continue looking into this until I get too frustrated/bored :)

@broxeph broxeph closed this Oct 31, 2019
@broxeph broxeph reopened this Oct 31, 2019
@broxeph
Copy link
Author

broxeph commented Oct 31, 2019

sqlmigrate output for the relevant schedule migrations, 0001 and 0008, on Postgres:

(note how schedule_calendar_slug_ba17e861_like is created twice)

m sqlmigrate schedule 0001
Loaded settings for site "mysite" in role "local" at domain "mysite.com".
BEGIN;
--
-- Create model Calendar
--
CREATE TABLE "schedule_calendar" ("id" serial NOT NULL PRIMARY KEY, "name" varchar(200) NOT NULL, "slug" varchar(200) NOT NULL);
--
-- Create model CalendarRelation
--
CREATE TABLE "schedule_calendarrelation" ("id" serial NOT NULL PRIMARY KEY, "object_id" integer NOT NULL, "distinction" varchar(20) NULL, "inheritable" boolean NOT NULL, "calendar_id" integer NOT NULL, "content_type_id" integer NOT NULL);
--
-- Create model Event
--
CREATE TABLE "schedule_event" ("id" serial NOT NULL PRIMARY KEY, "start" timestamp with time zone NOT NULL, "end" timestamp with time zone NOT NULL, "title" varchar(255) NOT NULL, "description" text NULL, "created_on" timestamp with time zone NOT NULL, "updated_on" timestamp with time zone NOT NULL, "end_recurring_period" timestamp with time zone NULL, "calendar_id" integer NULL, "creator_id" integer NULL);
--
-- Create model EventRelation
--
CREATE TABLE "schedule_eventrelation" ("id" serial NOT NULL PRIMARY KEY, "object_id" integer NOT NULL, "distinction" varchar(20) NULL, "content_type_id" integer NOT NULL, "event_id" integer NOT NULL);
--
-- Create model Occurrence
--
CREATE TABLE "schedule_occurrence" ("id" serial NOT NULL PRIMARY KEY, "title" varchar(255) NULL, "description" text NULL, "start" timestamp with time zone NOT NULL, "end" timestamp with time zone NOT NULL, "cancelled" boolean NOT NULL, "original_start" timestamp with time zone NOT NULL, "original_end" timestamp with time zone NOT NULL, "created_on" timestamp with time zone NOT NULL, "updated_on" timestamp with time zone NOT NULL, "event_id" integer NOT NULL);
--
-- Create model Rule
--
CREATE TABLE "schedule_rule" ("id" serial NOT NULL PRIMARY KEY, "name" varchar(32) NOT NULL, "description" text NOT NULL, "frequency" varchar(10) NOT NULL, "params" text NULL);
--
-- Add field rule to event
--
ALTER TABLE "schedule_event" ADD COLUMN "rule_id" integer NULL;
CREATE INDEX "schedule_calendar_slug_ba17e861" ON "schedule_calendar" ("slug");
CREATE INDEX "schedule_calendar_slug_ba17e861_like" ON "schedule_calendar" ("slug" varchar_pattern_ops);
ALTER TABLE "schedule_calendarrelation" ADD CONSTRAINT "schedule_calendarrel_calendar_id_0a50be2e_fk_schedule_" FOREIGN KEY ("calendar_id") REFERENCES "schedule_calendar" ("id") DEFERRABLE INITIALLY DEFERRED;
ALTER TABLE "schedule_calendarrelation" ADD CONSTRAINT "schedule_calendarrel_content_type_id_f2a42f5b_fk_django_co" FOREIGN KEY ("content_type_id") REFERENCES "django_content_type" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX "schedule_calendarrelation_calendar_id_0a50be2e" ON "schedule_calendarrelation" ("calendar_id");
CREATE INDEX "schedule_calendarrelation_content_type_id_f2a42f5b" ON "schedule_calendarrelation" ("content_type_id");
ALTER TABLE "schedule_event" ADD CONSTRAINT "schedule_event_calendar_id_eb1c700f_fk_schedule_calendar_id" FOREIGN KEY ("calendar_id") REFERENCES "schedule_calendar" ("id") DEFERRABLE INITIALLY DEFERRED;
ALTER TABLE "schedule_event" ADD CONSTRAINT "schedule_event_creator_id_d2ffab6e_fk_auth_user_id" FOREIGN KEY ("creator_id") REFERENCES "auth_user" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX "schedule_event_calendar_id_eb1c700f" ON "schedule_event" ("calendar_id");
CREATE INDEX "schedule_event_creator_id_d2ffab6e" ON "schedule_event" ("creator_id");
ALTER TABLE "schedule_eventrelation" ADD CONSTRAINT "schedule_eventrelati_content_type_id_d4187723_fk_django_co" FOREIGN KEY ("content_type_id") REFERENCES "django_content_type" ("id") DEFERRABLE INITIALLY DEFERRED;
ALTER TABLE "schedule_eventrelation" ADD CONSTRAINT "schedule_eventrelation_event_id_8c57a7b4_fk_schedule_event_id" FOREIGN KEY ("event_id") REFERENCES "schedule_event" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX "schedule_eventrelation_content_type_id_d4187723" ON "schedule_eventrelation" ("content_type_id");
CREATE INDEX "schedule_eventrelation_event_id_8c57a7b4" ON "schedule_eventrelation" ("event_id");
ALTER TABLE "schedule_occurrence" ADD CONSTRAINT "schedule_occurrence_event_id_ade47cd8_fk_schedule_event_id" FOREIGN KEY ("event_id") REFERENCES "schedule_event" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX "schedule_occurrence_event_id_ade47cd8" ON "schedule_occurrence" ("event_id");
CREATE INDEX "schedule_event_rule_id_90b83d31" ON "schedule_event" ("rule_id");
ALTER TABLE "schedule_event" ADD CONSTRAINT "schedule_event_rule_id_90b83d31_fk_schedule_rule_id" FOREIGN KEY ("rule_id") REFERENCES "schedule_rule" ("id") DEFERRABLE INITIALLY DEFERRED;
COMMIT;
m sqlmigrate schedule 0008
Loaded settings for site "mysite" in role "local" at domain "mysite.com".
BEGIN;
--
-- Alter field object_id on calendarrelation
--
CREATE INDEX "schedule_calendarrelation_object_id_1743bce6" ON "schedule_calendarrelation" ("object_id");
--
-- Alter field object_id on eventrelation
--
CREATE INDEX "schedule_eventrelation_object_id_e22334a2" ON "schedule_eventrelation" ("object_id");
--
-- Alter index_together for calendarrelation (1 constraint(s))
--
CREATE INDEX "schedule_calendarrelatio_content_type_id_object_i_3378a516_idx" ON "schedule_calendarrelation" ("content_type_id", "object_id");
--
-- Alter index_together for eventrelation (1 constraint(s))
--
CREATE INDEX "schedule_eventrelation_content_type_id_object_id_c1b1e893_idx" ON "schedule_eventrelation" ("content_type_id", "object_id");
--
-- Alter field slug on calendar
--
ALTER TABLE "schedule_calendar" ADD CONSTRAINT "schedule_calendar_slug_ba17e861_uniq" UNIQUE ("slug");
CREATE INDEX "schedule_calendar_slug_ba17e861_like" ON "schedule_calendar" ("slug" varchar_pattern_ops);
COMMIT;

@jdufresne
Copy link
Collaborator

I use psycopg2 and PostgreSQL and do not see these errors. What versions of those are you using?

Can you try the latest release and see if you still experience the issue? I have made some fixes for migrations that I noticed, but the error is different.

@broxeph
Copy link
Author

broxeph commented Nov 4, 2019

Postgres 11.5, psycopg2 2.8.3, django 2.2.6, django-scheduler 0.9.2

Will try again from master and continue investigating.

@broxeph
Copy link
Author

broxeph commented Nov 4, 2019

Okay, at this point I suppose I'll have to settle for overriding migrations in our project (MIGRATION_MODULES). Feel free to close this if you can't reproduce it :/

@jdufresne jdufresne removed their request for review September 27, 2022 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants