Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2170 introduce gender model #2485

Open
wants to merge 37 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
3e4b85d
draft create update delete + tests + testupdates + migration
hjanott Jun 21, 2024
36f1d70
update meta rep
hjanott Jun 21, 2024
a025ba2
general error removal and improvement for draft
hjanott Jun 21, 2024
275aed2
remove meeting import error
hjanott Jun 24, 2024
9a79a5e
update meeting import so that checker removes gender_id
hjanott Jun 24, 2024
42b63b6
general code improvements delete gender checks for back relation in o…
hjanott Jun 25, 2024
b4f9e8f
md documentation + pleasing mypy
hjanott Jun 25, 2024
9162ee6
merge upstream main
hjanott Jun 26, 2024
6e74b9c
Merge commit 'b1d1e6fd' into 2170_introduce_gender_model
hjanott Jun 26, 2024
8dfbd80
use defaultdict
hjanott Jun 26, 2024
c356c42
use new in memory flag of datastore
hjanott Jun 27, 2024
468f9eb
fix mypy error by upping datastore version
hjanott Jun 27, 2024
87dfb9e
change the permission from can_manage_organization to can_manage_users
hjanott Jun 27, 2024
3aec96d
improve documentation
hjanott Jun 28, 2024
21addc2
lock result false + no orga check
hjanott Jun 28, 2024
443cb58
add test for gender import
r-peschke Jul 5, 2024
773582b
make name and org id required
hjanott Jul 30, 2024
280f8b2
Use gender string for saml and meeting import. Refine gender actions …
hjanott Aug 1, 2024
ffbd52d
cleanup and documentation improvement
hjanott Aug 2, 2024
32e3cd9
beautify code
hjanott Aug 2, 2024
8be71fe
merge main by adding gender stuff (and locked_out in merge_mixin)
hjanott Aug 9, 2024
abfbefd
update from upstream/main
hjanott Aug 12, 2024
35f739f
add test update gender on user merge.
hjanott Aug 12, 2024
fe3e77e
update docs test for meeting import and saml account (+ new test) cre…
hjanott Aug 16, 2024
8ac6300
Merge remote-tracking branch 'upstream/main' into 2170_introduce_gend…
hjanott Aug 16, 2024
6b9d772
updated meta
hjanott Aug 16, 2024
a9341b0
meeting import create user with gender. export gender strings. improv…
hjanott Aug 20, 2024
4633eed
Separate test for gender on meeting import. General code improvements…
hjanott Aug 21, 2024
2641a8e
merge upstream main and generate models
hjanott Aug 22, 2024
7054aed
extend test for gender import, plus fixes. docs update
hjanott Aug 27, 2024
92fa165
update meta repo
hjanott Aug 28, 2024
5b26bf1
improve user and gender updates/creation
r-peschke Sep 4, 2024
20b6d5c
Gender will not be created without being used anymore. Fix error wher…
hjanott Sep 5, 2024
ee2cff4
Improve gender action documentation
hjanott Sep 5, 2024
033ebde
fixed typo in migration gender female
hjanott Sep 6, 2024
e922d46
Merge main and update migration index.
hjanott Sep 18, 2024
9b51eff
Merge branch 'main' into 2170_introduce_gender_model
hjanott Sep 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/actions/account.json_upload.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The types noted below are the internal types after conversion in the backend. Se
member_number: string, // unique member_number, info: done (used as matching field), new (newly added) or error
title: string,
pronoun: string,
gender: string, // as defined in organization/genders, info: done or warning
gender: string, // info: done or warning
default_password: string, // info: generated, done or warning
is_active: boolean,
is_physical_person: boolean,
Expand Down
13 changes: 13 additions & 0 deletions docs/actions/gender.create.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
## Payload
```
{
// Required
name: string;
}
```

## Action
Creates the gender.

## Permissions
The user needs to have the organization management level `can_manage_organization`.
10 changes: 10 additions & 0 deletions docs/actions/gender.delete.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
## Payload
```
{id: Id;}
```

## Action
Deletes the gender.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add some text like The first 4 gender names are fixed and cannot be changed or delete.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


## Permissions
The user needs to have the organization management level `can_manage_organization`.
17 changes: 17 additions & 0 deletions docs/actions/gender.update.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
## Payload
```
{
// Required
id: number;

// Optional
// Group A
name: string;
r-peschke marked this conversation as resolved.
Show resolved Hide resolved
}
```

## Action
Updates the gender.

## Permissions
- Group A: The user needs the OML `can_manage_organization`
3 changes: 2 additions & 1 deletion docs/actions/meeting.import.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,11 @@ Import one meeting from a file. The file must only contain exactly one meeting.
- The request user is assigned to the admin group.
- meeting.is_active_in_organization_id is set.
- It has to be checked, whether the organization.limit_of_meetings is unlimited(=0) or lower than the active meetings in organization.active_meeting_ids, if the new meeting is not archived (`is_active_in_organization_id` is set)
- Search for users and if username, first-name, last-name and email are identical use this user instead creating a duplicate. Keep the data, including password, of the exiating user.
- Search for users and if username, first-name, last-name and email are identical use this user instead of creating a duplicate. Keep the data, including password, of the existing user.
- Users, that still have to be duplicated:
- Imported usernames will be checked for uniqueness and adjusted in the case of collisions.
- All previously set user passwords will be replaced
- Genders will not be updated or imported.


## Permissions
Expand Down
1 change: 0 additions & 1 deletion docs/actions/organization.update.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@
users_email_replyto: string;
users_email_subject: string;
users_email_body: text;
genders: string[];

// Group B
enable_electronic_voting: boolean;
Expand Down
2 changes: 1 addition & 1 deletion docs/actions/participant.json_upload.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The types noted below are the internal types after conversion in the backend. Se
member_number: string, // unique member_number, info: done, error, new (newly added) or remove (missing field permission)
title: string, // info: done or remove (missing field permission)
pronoun: string, // info: done or remove (missing field permission)
gender: string, // as defined in organization/genders, info: done, warning (undefined gender) or remove (missing field permission)
gender: string, // info: done, warning (undefined gender) or remove (missing field permission)
default_password: string, // info: generated, done, warning or remove (missing field permission)
is_active: boolean, // info: done or remove (missing field permission)
is_physical_person: boolean, // info: done or remove (missing field permission)
Expand Down
4 changes: 2 additions & 2 deletions docs/actions/user.create.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
is_active: boolean;
is_physical_person: boolean;
can_change_own_password: boolean;
gender: string;
gender_id: Id;
pronoun: string;
email: string;
default_vote_weight: decimal(6);
Expand Down Expand Up @@ -56,7 +56,7 @@ Creates a user.
* If `username` is given, it has to be unique within all users. If there already exists a user with the same username, an error must be returned. If the `username` is not given, 1. the saml_id will be used or 2. it has to be generated (see [user.create#generate-a-username](user.create.md#generate-a-username) below). Also the username may not contain spaces.
* The `organization_management_level` as restring can be taken from the enum of this user field.
* Remove starting and trailing spaces from `username`, `first_name` and `last_name`
* The given `gender` must be present in `organization/genders`
* The given `gender_id` must be present in `organization/gender_ids`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better: ... must be present in database. The gender lived only in organization/genders-field, the gender_id is presented by an gender-instance in the database.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

* If `saml_id` is set in payload, there may be no `password` or `default_password` set or generated and `set_change_own_password` will be set to False.
* The `member_number` must be unique within all users.

Expand Down
2 changes: 1 addition & 1 deletion docs/actions/user.save_saml_account.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
first_name: string,
last_name: string,
email: string,
gender: string,
gender_id: Id,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep with gender, we will transform it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

pronoun: string,
is_active: boolean,
is_physical_person: boolean,
Expand Down
2 changes: 1 addition & 1 deletion docs/actions/user.update.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
is_active: boolean;
is_physical_person: boolean;
can_change_own_password: boolean;
gender: string;
gender_id: Id;
pronoun: string;
email: string;
default_vote_weight: decimal(6);
Expand Down
2 changes: 1 addition & 1 deletion docs/actions/user.update_self.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
// Optional
username: string;
email: string;
gender: string;
gender_id: Id;
pronoun: string;
}
```
Expand Down
38 changes: 32 additions & 6 deletions global/data/example-data.json
Original file line number Diff line number Diff line change
@@ -1,13 +1,39 @@
{
"_migration_index": 54,
"_migration_index": 55,
"gender":{
"1":{
"id": 1,
"name": "male",
"organization_id": 1,
"user_ids":[1]
},
"2":{
"id": 2,
"name": "female",
"organization_id": 1,
"user_ids":[2]
},
"3":{
"id": 3,
"name": "diverse",
"organization_id": 1,
"user_ids":[3]
},
"4":{
"id": 4,
"name": "non-binary",
"organization_id": 1,
"user_ids":[]
}
},
"organization": {
"1": {
"id": 1,
"name": "Test Organization",
"legal_notice": "<a href=\"http://www.openslides.org\">OpenSlides</a> is a free web based presentation and assembly system for visualizing and controlling agenda, motions and elections of an assembly.",
"login_text": "Good Morning!",
"default_language": "en",
"genders": ["male", "female", "diverse", "non-binary"],
"gender_ids": [1, 2 ,3 ,4],
"enable_electronic_voting": true,
"enable_chat": true,
"reset_password_verbose_errors": true,
Expand Down Expand Up @@ -42,7 +68,7 @@
"first_name": "firstName",
"last_name": "lastName",
"email": "email",
"gender": "gender",
"gender_id": "gender_id",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave "gender": "gender", we will transform it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

"pronoun": "pronoun",
"is_active": "is_active",
"is_physical_person": "is_person"
Expand All @@ -59,7 +85,7 @@
"password": "316af7b2ddc20ead599c38541fbe87e9a9e4e960d4017d6e59de188b41b2758flD5BVZAZ8jLy4nYW9iomHcnkXWkfk3PgBjeiTSxjGG7+fBjMBxsaS1vIiAMxYh+K38l0gDW4wcP+i8tgoc4UBg==",
"default_password": "admin",
"can_change_own_password": true,
"gender": "male",
"gender_id": 1,
"default_vote_weight": "1.000000",
"organization_management_level": "superadmin",
"is_present_in_meeting_ids": [
Expand Down Expand Up @@ -88,7 +114,7 @@
"password": "316af7b2ddc20ead599c38541fbe87e9a9e4e960d4017d6e59de188b41b2758fDB3tv5HcCtPRREt7bPGqerTf1AbmoKXt/fVFkLY4znDRh2Yy0m3ZjXD0nHI8oa6KrGlHH/cvysfvf8i2fWIzmw==",
"default_password": "a",
"can_change_own_password": true,
"gender": "female",
"gender_id": 2,
"default_vote_weight": "1.000000",
"committee_ids": [
1
Expand All @@ -109,7 +135,7 @@
"password": "316af7b2ddc20ead599c38541fbe87e9a9e4e960d4017d6e59de188b41b2758fIxDxvpkn6dDLRxT9DxJhZ/f04AL2oK2beICRFobSw53CI93U+dfN+w+NaL7BvrcR4JWuMj9NkH4dVjnnI0YTkg==",
"default_password": "jKwSLGCk",
"can_change_own_password": true,
"gender": "diverse",
"gender_id": 3,
"default_vote_weight": "1.000000",
"option_ids": [8, 11],
"meeting_user_ids": [3],
Expand Down
30 changes: 28 additions & 2 deletions global/data/initial-data.json
Original file line number Diff line number Diff line change
@@ -1,12 +1,38 @@
{
"_migration_index": 54,
"_migration_index": 55,
"gender":{
"1":{
"id": 1,
"name": "male",
"organization_id": 1,
"user_ids":[]
},
"2":{
"id": 2,
"name": "female",
"organization_id": 1,
"user_ids":[]
},
"3":{
"id": 3,
"name": "diverse",
"organization_id": 1,
"user_ids":[]
},
"4":{
"id": 4,
"name": "non-binary",
"organization_id": 1,
"user_ids":[]
}
},
"organization": {
"1": {
"id": 1,
"name": "[Your organization]",
"legal_notice": "<a href=\"http://www.openslides.org\">OpenSlides</a> is a free web based presentation and assembly system for visualizing and controlling agenda, motions and elections of an assembly. The event organizer is resposible for the content.",
"login_text": "Welcome to OpenSlides. Please login.",
"genders": ["male", "female", "diverse", "non-binary"],
"gender_ids": [1, 2, 3, 4],
"default_language": "en",
"enable_electronic_voting": true,
"limit_of_meetings": 0,
Expand Down
2 changes: 1 addition & 1 deletion global/meta
Submodule meta updated 1 files
+24 −10 models.yml
1 change: 1 addition & 0 deletions openslides_backend/action/actions/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ def prepare_actions_map() -> None:
chat_group,
chat_message,
committee,
gender,
group,
list_of_speakers,
mediafile,
Expand Down
1 change: 1 addition & 0 deletions openslides_backend/action/actions/gender/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from . import create, delete, update # noqa
27 changes: 27 additions & 0 deletions openslides_backend/action/actions/gender/create.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
from typing import Any

from ....action.mixins.archived_meeting_check_mixin import CheckForArchivedMeetingMixin
from ....models.models import Gender
from ....permissions.management_levels import OrganizationManagementLevel
from ....shared.util import ONE_ORGANIZATION_ID
from ...generics.create import CreateAction
from ...util.default_schema import DefaultSchema
from ...util.register import register_action
from .mixins import GenderUniqueMixin


@register_action("gender.create")
class GenderCreate(CreateAction, CheckForArchivedMeetingMixin, GenderUniqueMixin):
"""
Action to create a gender.
"""

model = Gender()
schema = DefaultSchema(Gender()).get_create_schema(
required_properties=["name"],
)
permission = OrganizationManagementLevel.CAN_MANAGE_ORGANIZATION

def update_instance(self, instance: dict[str, Any]) -> dict[str, Any]:
instance["organization_id"] = ONE_ORGANIZATION_ID
return instance
r-peschke marked this conversation as resolved.
Show resolved Hide resolved
24 changes: 24 additions & 0 deletions openslides_backend/action/actions/gender/delete.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
from typing import Any

from ....models.models import Gender
from ....permissions.management_levels import OrganizationManagementLevel
from ...generics.delete import DeleteAction
from ...util.default_schema import DefaultSchema
from ...util.register import register_action
from .mixins import GenderPermissionMixin


@register_action("gender.delete")
class GenderDeleteAction(DeleteAction, GenderPermissionMixin):
"""
Action to delete a gender.
"""

model = Gender()
schema = DefaultSchema(Gender()).get_delete_schema()
permission = OrganizationManagementLevel.CAN_MANAGE_ORGANIZATION
skip_archived_meeting_check = True

def update_instance(self, instance: dict[str, Any]) -> dict[str, Any]:
super().check_editable(instance)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you rename the method in the mixin from check_editable to check_permissions you remove the update_instance method here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

return instance
33 changes: 33 additions & 0 deletions openslides_backend/action/actions/gender/mixins.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
from typing import Any

from datastore.shared.util import fqid_from_collection_and_id

from openslides_backend.action.action import Action
from openslides_backend.shared.exceptions import ActionException

from ...mixins.check_unique_name_mixin import CheckUniqueInContextMixin


class GenderUniqueMixin(CheckUniqueInContextMixin):
def validate_instance(self, instance: dict[str, Any]) -> None:
super().validate_instance(instance)
if instance.get("name") == "":
raise ActionException("Empty gender name not allowed.")
self.check_unique_in_context(
"name",
instance.get("name", ""),
"Gender '" + instance.get("name", "") + "' already exists.",
instance.get("id"),
)


class GenderPermissionMixin(Action):
def check_editable(self, instance: dict[str, Any]) -> None:
gender = instance.get("id", 0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if it is an id, give it the name gender_id ;-)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

if 0 < gender < 5:
if gender_name := self.datastore.get(
fqid_from_collection_and_id("gender", instance.get("id", 0)), ["name"]
).get("name"):
gender = gender_name
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stay with the field name gender_name. You have chosen it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I restructured this a bit. I think now it's even better to read.

msg = f"Cannot delete or update gender '{gender}' from default selection."
raise ActionException(msg)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use a more specific exception, i.e. PermissionException

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

29 changes: 29 additions & 0 deletions openslides_backend/action/actions/gender/update.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
from typing import Any

from ....action.mixins.archived_meeting_check_mixin import CheckForArchivedMeetingMixin
from ....models.models import Gender
from ....permissions.management_levels import OrganizationManagementLevel
from ...generics.update import UpdateAction
from ...util.default_schema import DefaultSchema
from ...util.register import register_action
from .mixins import GenderPermissionMixin, GenderUniqueMixin


@register_action("gender.update")
class GenderUpdateAction(
UpdateAction, CheckForArchivedMeetingMixin, GenderPermissionMixin, GenderUniqueMixin
):
"""
Action to update a gender.
"""

model = Gender()
schema = DefaultSchema(Gender()).get_update_schema(
required_properties=["name"],
)
permission = OrganizationManagementLevel.CAN_MANAGE_ORGANIZATION

def update_instance(self, instance: dict[str, Any]) -> dict[str, Any]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renaming the mixin method from check_editable to check_permissions you could remove the update_instance in this place.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense

super().check_editable(instance)
instance = super().update_instance(instance)
return instance
5 changes: 4 additions & 1 deletion openslides_backend/action/actions/meeting/import_.py
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,10 @@ def update_instance(self, instance: dict[str, Any]) -> dict[str, Any]:
"last_email_sent",
"last_login",
"committee_ids",
"gender",
r-peschke marked this conversation as resolved.
Show resolved Hide resolved
"gender_id", # this is fine since we do not delete actual user data and imported genders can be omitted as superseded outdated
],
"organization": ["genders"],
},
)
try:
Expand Down Expand Up @@ -747,7 +750,7 @@ def migrate_data(self, instance: dict[str, Any]) -> dict[str, Any]:
instance["meeting"] = defaultdict(dict)
for fqid, model in migrated_models.items():
collection, id = collection_and_id_from_fqid(fqid)
if collection not in ("organization", "committee", "theme"):
if collection not in ("organization", "committee", "theme", "gender"):
instance["meeting"][collection][str(id)] = model

instance["meeting"]["_migration_index"] = backend_migration_index
Expand Down
Loading
Loading