Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add query parser library #73

Merged
merged 11 commits into from
Feb 17, 2025
Merged

feat: add query parser library #73

merged 11 commits into from
Feb 17, 2025

Conversation

whoAbhishekSah
Copy link
Member

@whoAbhishekSah whoAbhishekSah commented Feb 14, 2025

Description

Add a golang subpackage rql which provides RESTful APIs capability to parse advanced REST API query parameters like (filter, pagination, sort, group, search) and logical operators on the keys (like eq, neq, like, gt, lt etc).

Testing

  1. Unit tests are added.
  2. Manual testing is done locally using this main.go file and input.json file
main.go
package main

import (
	"encoding/json"
	"fmt"
	"io/ioutil"
	"log"
	"os"
	"time"

	"github.com/doug-martin/goqu/v9"
	"github.com/raystack/salt/rql"
)

type Organization struct {
	Id              int       `rql:"type=number,min=10,max=200"`
	BillingPlanName string    `rql:"type=string"`
	CreatedAt       time.Time `rql:"type=datetime"`
	MemberCount     int       `rql:"type=number"`
	Title           string    `rql:"type=string"`
	Enabled         bool      `rql:"type=bool"`
}

func main() {
	userInput := &rql.Query{}
	// org := Organization{10, "standard plan", time.Now(), 10, "pixxel space pvt ltd"}
	jsonFile, err := os.Open("input.json")
	if err != nil {
		log.Fatalf(err.Error())
	}
	defer jsonFile.Close()
	byteValue, err := ioutil.ReadAll(jsonFile)
	if err != nil {
		log.Fatalf(err.Error())
	}
	err = json.Unmarshal(byteValue, userInput)
	if err != nil {
		panic(fmt.Sprintf("failed to unmarshal query string to parser query struct, err:%s", err.Error()))
	}

	err = rql.ValidateQuery(userInput, Organization{})
	if err != nil {
		panic(err)
	}
	query := goqu.From("organizations")

	fuzzySearchColumns := []string{"id", "billing_plan_name", "title"}

	for _, filter_item := range userInput.Filters {
		query = query.Where(goqu.Ex{
			filter_item.Name: goqu.Op{filter_item.Operator: filter_item.Value},
		})
	}

	listOfExpressions := make([]goqu.Expression, 0)

	if userInput.Search != "" {
		for _, col := range fuzzySearchColumns {
			listOfExpressions = append(listOfExpressions, goqu.Ex{
				col: goqu.Op{"LIKE": userInput.Search},
			})
		}
	}

	query = query.Where(goqu.Or(listOfExpressions...))

	query = query.Offset(uint(userInput.Offset))
	for _, sort_item := range userInput.Sort {
		switch sort_item.Order {
		case "asc":
			query = query.OrderAppend(goqu.C(sort_item.Key).Asc())
		case "desc":
			query = query.OrderAppend(goqu.C(sort_item.Key).Desc())
		default:
		}
	}
	query = query.Limit(uint(userInput.Limit))
	sql, _, _ := query.ToSQL()
	fmt.Println(sql)
}
{
    "filters": [
      { "name": "id", "operator": "neq", "value": 20 },
      { "name": "title", "operator": "neq", "value": "nasa" },
      { "name": "enabled", "operator": "eq", "value": false },
      {
        "name": "createdAt",
        "operator": "gte",
        "value": "2025-02-05T11:25:37.957Z"
      },
      { "name": "title", "operator": "like", "value": "xyz" }
    ],
    "group_by": ["billingPlanName"],
    "offset": 20,
    "limit": 50,
    "search": "abcd",
    "sort": [
      { "key": "title", "order": "desc" },
      { "key": "createdAt", "order": "asc" }
    ]
  }

The output SQL is:

SELECT * FROM "organizations" WHERE (("id" != 20) AND ("title" != 'nasa') AND ("enabled" IS FALSE) AND ("createdAt" >= '2025-02-05T11:25:37.957Z') AND ("title" LIKE 'xyz') AND (("id" LIKE 'abcd') OR ("billing_plan_name" LIKE 'abcd') OR ("title" LIKE 'abcd'))) ORDER BY "title" DESC, "createdAt" ASC LIMIT 50 OFFSET 20

@ravisuhag
Copy link
Member

ravisuhag commented Feb 14, 2025

@whoAbhishekSah Better to call it rql ? or something better?

We can also move out parse outside sub folder. So that import becomes simple.

@whoAbhishekSah
Copy link
Member Author

@ravisuhag Yeah, rql sounds good. I've updated it as per rql literal.
Also, I removed the extra nesting of the directory for now.

@punit-kulal punit-kulal merged commit 78904a0 into main Feb 17, 2025
3 checks passed
@punit-kulal punit-kulal deleted the query_parser branch February 17, 2025 07:05
whoAbhishekSah added a commit that referenced this pull request Feb 26, 2025
* feat: add query parser library

* feat: return error if invalid type

* refactor: remove main.go

* refactor: lint fix

* refactor: rename to rql

* refactor: make data type key private

* refactor: remove custom util function

* refactor: fix tests

* fix golangci.yml

* refactor: create constants

* refactor: create constants

---------

Co-authored-by: Punit Kulal <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants