Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow sites and devices to belong to more than one Group #4038

Merged
merged 3 commits into from
Dec 10, 2024

Conversation

Baalmart
Copy link
Contributor

@Baalmart Baalmart commented Dec 10, 2024

Description

allow sites and devices to belong to more than one Group

Changes Made

  • allow sites and devices to belong to more than one Group

Testing

  • Tested locally
  • Tested against staging environment
  • Relevant tests passed: [List test names]

Affected Services

  • Which services were modified:
    • Device Registry

Endpoints Ready for Testing

  • New endpoints ready for testing:
    • Get Sites
    • Get Devices
    • Create Sites
    • Create Devices
    • Update Devices

Request Body:

{
    "groups": [
        "bbaale",
        "martin",
        "martin",
        "kika"
    ]
}

API Documentation Updated?

  • Yes, API documentation was updated
  • No, API documentation does not need updating

Additional Notes

allow sites and devices to belong to more than one Group

Summary by CodeRabbit

  • New Features

    • Introduced a new optional groups parameter for creating and updating sites.
    • Added validation for the groups field in device creation and update.
    • Enhanced filtering capabilities to support multiple group matches.
  • Bug Fixes

    • Updated various schemas and methods to consistently use the plural groups instead of group.
  • Documentation

    • Updated relevant documentation to reflect changes in the groups field across multiple models.
  • Chores

    • Refactored existing methods to improve clarity and maintainability, particularly in error handling and duplicate checks.

Copy link
Contributor

coderabbitai bot commented Dec 10, 2024

Warning

Rate limit exceeded

@Baalmart has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 21 minutes and 30 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 99b38f9 and 68a0b0f.

📒 Files selected for processing (3)
  • src/device-registry/models/Location.js (5 hunks)
  • src/device-registry/routes/v2/cohorts.js (2 hunks)
  • src/device-registry/routes/v2/grids.js (2 hunks)
📝 Walkthrough
📝 Walkthrough
📝 Walkthrough

Walkthrough

The changes in this pull request primarily involve renaming the group property to groups across various schemas and models within the device registry. This includes updates to the data types, changing from a single string to an array of strings for the groups field. Additionally, new validations and methods have been introduced to enhance error handling and data integrity. The modifications are consistent across multiple files, ensuring that the new naming convention is uniformly applied throughout the codebase.

Changes

File Path Change Summary
src/device-registry/config/global/db-projections.js Renamed group to groups in several projection definitions: SITES_INCLUSION_PROJECTION, DEVICES_INCLUSION_PROJECTION, GRIDS_INCLUSION_PROJECTION, COHORTS_INCLUSION_PROJECTION, AIRQLOUDS_INCLUSION_PROJECTION, SITE_ACTIVITIES_INCLUSION_PROJECTION.
src/device-registry/models/Activity.js Renamed group to groups, changed type from String to [String]. Updated toJSON method to return this.groups.
src/device-registry/models/Airqloud.js Renamed group to groups, changed type from String to [String]. Updated toJSON method to return groups.
src/device-registry/models/Cohort.js Renamed group to groups, changed type from String to [String]. Updated toJSON method and aggregation pipeline to use groups.
src/device-registry/models/Device.js Renamed group to groups, changed type from String to [String]. Added checkDuplicates function for duplicate checks. Updated pre-save middleware and toJSON method.
src/device-registry/models/Grid.js Renamed group to groups, changed type from String to [String]. Updated toJSON method to reflect this change.
src/device-registry/models/Location.js Added new field groups as [String]. Updated toJSON, list, and remove methods to include groups.
src/device-registry/models/Photo.js Renamed group to groups, changed type from String to [String]. Updated toJSON and list methods to use groups.
src/device-registry/models/Site.js Renamed group to groups, changed type from String to [String]. Added checkDuplicates function. Updated pre-save middleware and toJSON method.
src/device-registry/routes/v2/sites.js Added optional body parameter groups in POST and PUT routes for site creation and updates.
src/device-registry/utils/generate-filter.js Renamed filter.group to filter.groups in multiple functions to support multiple group filtering.
src/device-registry/validators/device.validators.js Added validation for groups field in validateCreateDevice and validateUpdateDevice.

Poem

In the realm of code, a shift so bright,
From group to groups, a naming delight.
Arrays now hold what once was a string,
Enhancing our models, oh what joy they bring!
With checks for duplicates and routes anew,
A cleaner design, for me and for you! 🌟


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Experiment)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

codecov bot commented Dec 10, 2024

Codecov Report

Attention: Patch coverage is 5.26316% with 36 lines in your changes missing coverage. Please review.

Project coverage is 11.73%. Comparing base (93914f8) to head (68a0b0f).
Report is 26 commits behind head on staging.

Files with missing lines Patch % Lines
src/device-registry/models/Device.js 5.55% 17 Missing ⚠️
src/device-registry/models/Site.js 8.33% 11 Missing ⚠️
src/device-registry/routes/v2/cohorts.js 0.00% 2 Missing ⚠️
src/device-registry/routes/v2/grids.js 0.00% 2 Missing ⚠️
src/device-registry/routes/v2/sites.js 0.00% 2 Missing ⚠️
...rc/device-registry/validators/device.validators.js 0.00% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           staging    #4038      +/-   ##
===========================================
- Coverage    11.73%   11.73%   -0.01%     
===========================================
  Files          114      114              
  Lines        15331    15349      +18     
  Branches       319      319              
===========================================
+ Hits          1799     1801       +2     
- Misses       13532    13548      +16     
Files with missing lines Coverage Δ
src/device-registry/models/Activity.js 18.47% <ø> (ø)
src/device-registry/models/Airqloud.js 25.71% <ø> (ø)
src/device-registry/models/Cohort.js 24.50% <ø> (ø)
src/device-registry/models/Grid.js 23.14% <ø> (ø)
src/device-registry/models/Location.js 20.75% <ø> (ø)
src/device-registry/models/Photo.js 13.19% <ø> (ø)
src/device-registry/routes/v2/cohorts.js 41.93% <0.00%> (-0.93%) ⬇️
src/device-registry/routes/v2/grids.js 32.82% <0.00%> (-0.51%) ⬇️
src/device-registry/routes/v2/sites.js 21.01% <0.00%> (-0.31%) ⬇️
...rc/device-registry/validators/device.validators.js 32.78% <0.00%> (-1.12%) ⬇️
... and 2 more

Copy link
Contributor

Device registry changes in this PR available for preview here

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (9)
src/device-registry/models/Airqloud.js (1)

116-117: LGTM! Consider adding validation for groups

The groups field implementation is consistent with other models. Consider adding validation to ensure group values are unique within the array.

 groups: {
   type: [String],
   trim: true,
+  validate: {
+    validator: function(v) {
+      return new Set(v).size === v.length;
+    },
+    message: 'Duplicate group values are not allowed'
+  }
 },
src/device-registry/models/Location.js (1)

87-90: Consider adding validation for the groups array

While the schema change supports multiple groups, consider adding validation to ensure data integrity:

  • Minimum/maximum array length constraints
  • String format validation for group identifiers
  • Default empty array value
     groups: {
       type: [String],
       trim: true,
+      default: [],
+      validate: {
+        validator: function(v) {
+          return v.every(group => /^[a-zA-Z0-9_-]+$/.test(group));
+        },
+        message: props => 'Group identifiers must contain only alphanumeric characters, underscores, and hyphens'
+      }
     },
src/device-registry/models/Cohort.js (1)

253-253: LGTM! Consider indexing for performance

The groups field is correctly mapped in the device response. Given that this field will be frequently queried, consider adding an index for it.

 cohortSchema.index({ geoHash: 1 });
+cohortSchema.index({ groups: 1 });
src/device-registry/models/Photo.js (1)

Line range hint 86-90: Fix duplicate image_url field in toJSON

While the groups field is correctly added, there's an unrelated issue: image_url appears twice in the output (lines 86 and 87).

     return {
       image_url: this.image_url,
       metadata: this.metadata,
       id: this._id,
       tags: this.tags,
       name: this.name,
       network: this.network,
       groups: this.groups,
-      image_url: this.image_url,
       device_id: this.device_id,
src/device-registry/models/Device.js (1)

77-79: Consider removing trim from groups array field

The trim option on the array field may not work as expected since it applies to the array itself rather than the individual string elements.

    groups: {
      type: [String],
-     trim: true,
    },

Consider validating and trimming individual strings during input processing if needed.

src/device-registry/validators/device.validators.js (2)

129-137: Consider adding validation for individual group strings

While the array validation is good, consider adding checks for individual string elements to ensure they meet your requirements (e.g., non-empty strings, format, length).

      body("groups")
        .optional()
        .custom((value) => {
-         return Array.isArray(value);
+         return Array.isArray(value) && value.every(item => 
+           typeof item === 'string' && item.trim().length > 0
+         );
        })
-       .withMessage("the groups should be an array")
+       .withMessage("groups should be an array of non-empty strings")
        .bail()
        .notEmpty()
        .withMessage("the groups should not be empty"),

396-404: Maintain consistency with create validation

When implementing the suggested improvements for create validation, ensure the same validation logic is applied here for consistency.

src/device-registry/models/Site.js (2)

78-80: Remove trim from groups array field for consistency

Similar to the Device model, the trim option on the array field should be removed as it won't work as expected on the array type.

    groups: {
      type: [String],
-     trim: true,
    },

367-379: Consider moving checkDuplicates to a shared utility

The checkDuplicates function is identical in both Device and Site models. Consider moving it to a shared utility file to maintain DRY principles.

Consider creating a new utility file (e.g., src/device-registry/utils/array-validators.js) and moving the function there:

// src/device-registry/utils/array-validators.js
const { HttpError } = require("@utils/errors");
const httpStatus = require("http-status");

exports.checkDuplicates = (arr, fieldName) => {
  const duplicateValues = arr.filter(
    (value, index, self) => self.indexOf(value) !== index
  );

  if (duplicateValues.length > 0) {
    return new HttpError(
      `Duplicate values found in ${fieldName} array.`,
      httpStatus.BAD_REQUEST
    );
  }
  return null;
};
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 9cab881 and 99b38f9.

📒 Files selected for processing (12)
  • src/device-registry/config/global/db-projections.js (6 hunks)
  • src/device-registry/models/Activity.js (2 hunks)
  • src/device-registry/models/Airqloud.js (2 hunks)
  • src/device-registry/models/Cohort.js (7 hunks)
  • src/device-registry/models/Device.js (5 hunks)
  • src/device-registry/models/Grid.js (3 hunks)
  • src/device-registry/models/Location.js (5 hunks)
  • src/device-registry/models/Photo.js (3 hunks)
  • src/device-registry/models/Site.js (5 hunks)
  • src/device-registry/routes/v2/sites.js (2 hunks)
  • src/device-registry/utils/generate-filter.js (9 hunks)
  • src/device-registry/validators/device.validators.js (2 hunks)
👮 Files not reviewed due to content moderation or server errors (3)
  • src/device-registry/routes/v2/sites.js
  • src/device-registry/config/global/db-projections.js
  • src/device-registry/utils/generate-filter.js
🔇 Additional comments (16)
src/device-registry/models/Activity.js (2)

73-73: LGTM! Consistent toJSON update

The toJSON method correctly reflects the schema change from 'group' to 'groups'.


43-44: LGTM! Schema change for multiple group support

The change from single group to groups array is well-implemented with appropriate type definition and trim support.

Let's verify the schema migration impact:

✅ Verification successful

The previous search patterns might have been too strict. Let's broaden the search to catch any variations in the schema definition and also look for related migrations to understand the schema change better.


Schema change for multiple groups is consistently implemented across models

The verification shows that the groups field with type [String] and trim: true is consistently implemented across all relevant models:

  • Activity.js
  • Airqloud.js
  • Cohort.js
  • Device.js
  • Grid.js
  • Location.js
  • Photo.js
  • Site.js

No instances of the old single group field were found, indicating a complete migration to the new array-based schema.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check for any remaining single 'group' field references
rg -l 'group: \{.*type: String' src/device-registry/models/
# Check for consistent array type usage
rg -l 'groups: \{.*type: \[String\]' src/device-registry/models/

Length of output: 128


Script:

#!/bin/bash
# Let's try a broader search for group/groups fields in the schema
rg -A 2 -B 2 'group[s]?:\s*\{' src/device-registry/models/

# Also search for any group-related migrations
fd migration src/device-registry/ --exec rg -l 'group'

Length of output: 4655

src/device-registry/models/Grid.js (2)

46-47: LGTM! Consistent schema implementation

The groups field implementation maintains consistency with other models in the system.


124-124: LGTM! Well-structured toJSON update

The toJSON method properly includes the groups field in both destructuring and return statement.

Also applies to: 142-142

src/device-registry/models/Airqloud.js (1)

155-155: LGTM! Consistent toJSON implementation

The groups field is properly included in the toJSON output, maintaining consistent field ordering with other models.

src/device-registry/models/Location.js (1)

131-131: LGTM!

The toJSON method correctly includes the new groups field in the serialized output.

src/device-registry/models/Cohort.js (3)

20-22: Apply consistent validation across models

Similar to the Location model, consider adding validation for the groups array here.


84-84: LGTM!

The toJSON method correctly handles the new groups field in both destructuring and return value.

Also applies to: 95-95


205-205: LGTM!

The aggregation pipeline correctly projects the groups field.

src/device-registry/models/Photo.js (2)

21-24: Apply consistent validation across models

Similar to the Location and Cohort models, consider adding validation for the groups array here.


161-161: LGTM!

The groups field is correctly included in the aggregation projection.

src/device-registry/models/Device.js (3)

240-252: Well-implemented duplicate check utility!

The checkDuplicates function is clean, reusable, and handles error cases appropriately.


343-352: Good validation implementation in pre-save middleware!

The duplicate checks for both cohorts and groups are properly implemented with clear error handling.


386-399: Excellent handling of array updates!

The use of $addToSet for the groups field ensures no duplicates during updates while maintaining consistency with other array fields.

src/device-registry/models/Site.js (2)

427-429: Good consistency in array field handling!

The addition of groups to arrayFieldsToAddToSet maintains consistency with the Device model and ensures proper handling of array updates.


466-469: Well-implemented groups validation in pre-save middleware!

The duplicate checking for groups is properly implemented with clear error handling, maintaining consistency with the rest of the codebase.

@@ -200,6 +206,7 @@ locationSchema.statics = {
admin_level: 1,
description: 1,
network: 1,
group: 1,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Critical: Fix inconsistent field name in projectSummary

The projection still uses the old field name 'group' instead of 'groups', which could cause data inconsistency in summary responses.

-        group: 1,
+        groups: 1,
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
group: 1,
groups: 1,

Copy link
Contributor

Device registry changes in this PR available for preview here

1 similar comment
Copy link
Contributor

Device registry changes in this PR available for preview here

@Baalmart Baalmart merged commit c1e369b into staging Dec 10, 2024
50 of 52 checks passed
@Baalmart Baalmart deleted the en-entity-groups branch December 10, 2024 15:20
@Baalmart Baalmart mentioned this pull request Dec 10, 2024
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant