-
-
Notifications
You must be signed in to change notification settings - Fork 225
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
all: address performance issues and refactor structure (#58)
* core: addressing performance issues This patch introduces a very simple cache (needs to be replaced with LRU and needs a lock) and also a check if regex is used, and if not a simple string match is done. * LRU * LRU * more bench tests * sql all * finalize fetching everything from sql * improve sql adapter performance * fix tests * all: improve performance of regexp matches This patch introduces an LRU cache for compiled regular expressions. Manager implementations have been moved to their own packages. The SQL manager has been improved for better performance. * #58 (comment) * implement has_regex * implement has_regex * implement has_regex * implement has_regex * implement has_regex * vendor: resolves glide issues * readme: examples for new manager instantiation * readme: examples for new manager instantiation * vendor: updates ory-am/common to 0.2.2 * all: goimports and remove redis/rethinkdb * all: get tests passing * all: goimports * sql: implement migration * all: move to new org * vendor: add glide files * all: resolve broken import references, clean up * all: goimports * docs: update history file * manager: implement and test GetAll function
- Loading branch information
Showing
29 changed files
with
1,631 additions
and
1,003 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,3 +2,4 @@ | |
| *.iml | ||
| vendor/ | ||
| sqlite-test.db | ||
| tests/ | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| # History of breaking changes | ||
|
|
||
| <!-- START doctoc generated TOC please keep comment here to allow auto update --> | ||
| <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> | ||
|
|
||
| - [0.6.0](#060) | ||
| - [New location](#new-location) | ||
| - [Deprecating Redis and RethinkDB](#deprecating-redis-and-rethinkdb) | ||
| - [New packages](#new-packages) | ||
| - [IMPORTANT: SQL Changes](#important-sql-changes) | ||
| - [Manager API Changes](#manager-api-changes) | ||
|
|
||
| <!-- END doctoc generated TOC please keep comment here to allow auto update --> | ||
|
|
||
|
|
||
| ## 0.6.0 | ||
|
|
||
| Version 0.6.0 includes some larger BC breaks. This version focuses on various | ||
| performance boosts for both in-memory and SQL adapters, removes some technical debt | ||
| and restructures the repository. | ||
|
|
||
| ### New location | ||
|
|
||
| The location of this library changed from `github.com/ory-am/ladon` to `github.com/ory/ladon`. | ||
|
|
||
| ### Deprecating Redis and RethinkDB | ||
|
|
||
| Redis and RethinkDB are no longer maintained by ORY and were moved to | ||
| [ory/ladon-community](https://github.com/ory/ladon-community). The adapters had various | ||
| bugs and performance issues which is why they were removed from the official repository. | ||
|
|
||
| ### New packages | ||
|
|
||
| The SQLManager and MemoryManager moved to their own packages in `ladon/manager/sql` and `ladon/manager/memory`. | ||
| This change was made to avoid pulling dependencies that are not required by the user. | ||
|
|
||
| ### IMPORTANT: SQL Changes | ||
|
|
||
| The SQLManager was rewritten completely. Now, the database is 3NF (normalized) and includes | ||
| various improvements over the previous, naive adapter. The greatest challenge is matching | ||
| regular expressions within SQL databases, which causes significant overhead. | ||
|
|
||
| While there is an auto-migration for the schema, the data **is not automatically transferred to | ||
| the new schema**. | ||
|
|
||
| However, we provided a migration helper. For usage, check out | ||
| [xxx_manager_sql_migrator_test.go](xxx_manager_sql_migrator_test.go) or this short example: | ||
|
|
||
| ```go | ||
| var db = getSqlDatabaseFromSomewhere() | ||
| s := NewSQLManager(db, nil) | ||
|
|
||
| if err := s.CreateSchemas(); err != nil { | ||
| log.Fatalf("Could not create mysql schema: %v", err) | ||
| } | ||
|
|
||
| migrator := &SQLManagerMigrateFromMajor0Minor6ToMajor0Minor7{ | ||
| DB:db, | ||
| SQLManager:s, | ||
| } | ||
|
|
||
| err := migrator.Migrate() | ||
| ``` | ||
|
|
||
| Please run this migrator **only once and make back ups before you run it**. | ||
|
|
||
| ### Manager API Changes | ||
|
|
||
| `Manager.FindPoliciesForSubject` is now `Manager.FindRequestCandidates` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| package ladon_test | ||
|
|
||
| import ( | ||
| "fmt" | ||
| "strconv" | ||
| "testing" | ||
|
|
||
| "github.com/ory/ladon" | ||
| "github.com/ory/ladon/manager/memory" | ||
| "github.com/pborman/uuid" | ||
| "github.com/pkg/errors" | ||
| ) | ||
|
|
||
| func benchmarkLadon(i int, b *testing.B, warden *ladon.Ladon) { | ||
| //var concurrency = 30 | ||
| //var sem = make(chan bool, concurrency) | ||
| // | ||
| //for _, pol := range generatePolicies(i) { | ||
| // sem <- true | ||
| // go func(pol ladon.Policy) { | ||
| // defer func() { <-sem }() | ||
| // if err := warden.Manager.Create(pol); err != nil { | ||
| // b.Logf("Got error from warden.Manager.Create: %s", err) | ||
| // } | ||
| // }(pol) | ||
| //} | ||
| // | ||
| //for i := 0; i < cap(sem); i++ { | ||
| // sem <- true | ||
| //} | ||
|
|
||
| for _, pol := range generatePolicies(i) { | ||
| if err := warden.Manager.Create(pol); err != nil { | ||
| b.Logf("Got error from warden.Manager.Create: %s", err) | ||
| } | ||
| } | ||
|
|
||
| b.ResetTimer() | ||
| var err error | ||
| for n := 0; n < b.N; n++ { | ||
| if err = warden.IsAllowed(&ladon.Request{ | ||
| Subject: "5", | ||
| Action: "bar", | ||
| Resource: "baz", | ||
| }); errors.Cause(err) == ladon.ErrRequestDenied || errors.Cause(err) == ladon.ErrRequestForcefullyDenied || err == nil { | ||
| } else { | ||
| b.Logf("Got error from warden: %s", err) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| func BenchmarkLadon(b *testing.B) { | ||
| for _, num := range []int{10, 100, 1000, 10000, 100000, 1000000} { | ||
| b.Run(fmt.Sprintf("store=memory/policies=%d", num), func(b *testing.B) { | ||
| matcher := ladon.NewRegexpMatcher(4096) | ||
| benchmarkLadon(num, b, &ladon.Ladon{ | ||
| Manager: memory.NewMemoryManager(), | ||
| Matcher: matcher, | ||
| }) | ||
| }) | ||
|
|
||
| b.Run(fmt.Sprintf("store=mysql/policies=%d", num), func(b *testing.B) { | ||
| benchmarkLadon(num, b, &ladon.Ladon{ | ||
| Manager: managers["mysql"], | ||
| Matcher: ladon.NewRegexpMatcher(4096), | ||
| }) | ||
| }) | ||
|
|
||
| b.Run(fmt.Sprintf("store=postgres/policies=%d", num), func(b *testing.B) { | ||
| benchmarkLadon(num, b, &ladon.Ladon{ | ||
| Manager: managers["postgres"], | ||
| Matcher: ladon.NewRegexpMatcher(4096), | ||
| }) | ||
| }) | ||
| } | ||
| } | ||
|
|
||
| func generatePolicies(n int) map[string]ladon.Policy { | ||
| policies := map[string]ladon.Policy{} | ||
| for i := 0; i <= n; i++ { | ||
| id := uuid.New() | ||
| policies[id] = &ladon.DefaultPolicy{ | ||
| ID: id, | ||
| Subjects: []string{"foobar", "some-resource" + fmt.Sprintf("%d", i%100), strconv.Itoa(i)}, | ||
| Actions: []string{"foobar", "foobar", "foobar", "foobar", "foobar"}, | ||
| Resources: []string{"foobar", id}, | ||
| Effect: ladon.AllowAccess, | ||
| } | ||
| } | ||
| return policies | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.