-
Notifications
You must be signed in to change notification settings - Fork 156
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Removed OSCache. Using EHCache and Guava instead. (#67)
* Gateway HA implementation * perf: Utilize EHCache and Guava cache only * fix: changes after review * docs: update gateway-ha README.md * fix: updated after review * fix: replaced cacheDir with gatewayStateDir
- Loading branch information
1 parent
c8b3bee
commit 0c52b95
Showing
20 changed files
with
454 additions
and
223 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,155 +1,21 @@ | ||
# presto-gateway | ||
A load balancer / proxy / gateway for presto compute engine. | ||
|
||
## Getting Started | ||
|
||
### Build and run | ||
run `mvn clean install` to build `presto-gateway` | ||
|
||
Edit the [config file](gateway/src/main/resources/config.yml.template) and update backend urls | ||
A load balancer / proxy / gateway for presto compute engine. | ||
|
||
``` | ||
cd gateway/target/ | ||
java -jar gateway-{{VERSION}}-jar-with-dependencies.jar server ../src/presto-gateway/gateway/src/main/resources/config.yml.template | ||
``` | ||
Now you can access load balanced presto at localhost:8080 port. We will refer to this as `prestogateway.lyft.com` | ||
|
||
### Query History UI - check query plans etc. | ||
PrestoGateway records history of recent queries and displays links to check query details page in respective presto cluster. | ||
 | ||
## Getting started | ||
|
||
### Adhoc vs Scheduled query routing | ||
In the [config](gateway/src/main/resources/config.yml.template) | ||
you can specify if a backend cluster is a part of a routing group. | ||
If not specified, cluster will be part of `adhoc` routing group by default. | ||
Presto Gateway can be deployed as a standalone stateless service or with high availability and external database. For help with setting up a standalone gateway service, please visit [Standalone Gateway](/gateway/#standalone-gateway). For help with setting up a gateway service with high availability, please visit [Gateway-HA](/gateway-ha#gateway-ha). | ||
|
||
PrestoGateway router will route any request to `scheduled` group of clusters if request contains header `X-Presto-Routing-Group: scheduled` and there are clusters present with config `routingGroup: scheduled`. | ||
For further information regarding the design of the gateway, please visit the [Gateway Design Document](/docs/design.md) | ||
|
||
If no matching `routingGroup` found, router will route to `adhoc` group of clusters. Please make sure there are clusters under `adhoc` routing group. | ||
|
||
## Gateway API | ||
|
||
### Get all backends behind the gateway | ||
|
||
`curl -X GET prestogateway.lyft.com/gateway/backend/all | python -m json.tool` | ||
``` | ||
[ | ||
{ | ||
"active": true, | ||
"includeInRouter": true, | ||
"localPort": 8081, | ||
"name": "presto1", | ||
"proxyTo": "http://presto1.lyft.com", | ||
"routingGroup": "adhoc" | ||
}, | ||
{ | ||
"active": true, | ||
"includeInRouter": true, | ||
"localPort": 8083, | ||
"name": "presto3", | ||
"proxyTo": "http://presto3.lyft.com", | ||
"routingGroup": "adhoc" | ||
}, | ||
{ | ||
"active": true, | ||
"includeInRouter": true, | ||
"localPort": 8082, | ||
"name": "presto2", | ||
"proxyTo": "http://presto2.lyft.com", | ||
"routingGroup": "adhoc" | ||
} | ||
] | ||
``` | ||
|
||
### Get active backends behind the Gateway | ||
|
||
`curl -X GET prestogateway.lyft.com/gateway/backend/active | python -m json.tool` | ||
``` | ||
[ | ||
{ | ||
"active": true, | ||
"includeInRouter": true, | ||
"localPort": 8081, | ||
"name": "presto1", | ||
"proxyTo": "http://presto1.lyft.com", | ||
"routingGroup": "adhoc" | ||
}, | ||
{ | ||
"active": true, | ||
"includeInRouter": true, | ||
"localPort": 8082, | ||
"name": "presto2", | ||
"proxyTo": "http://presto2.lyft.com", | ||
"routingGroup": "adhoc" | ||
}, | ||
{ | ||
"active": true, | ||
"includeInRouter": true, | ||
"localPort": 8083, | ||
"name": "presto3", | ||
"proxyTo": "http://presto3.lyft.com", | ||
"routingGroup": "adhoc" | ||
} | ||
] | ||
``` | ||
### Deactivate a backend | ||
|
||
`curl -X POST prestogateway.lyft.com/gateway/backend/deactivate/presto2` | ||
|
||
Verify this by calling get active backends | ||
``` | ||
curl -X GET prestogateway.lyft.com/gateway/backend/active | python -m json.tool | ||
[ | ||
{ | ||
"active": true, | ||
"includeInRouter": true, | ||
"localPort": 8081, | ||
"name": "presto1", | ||
"proxyTo": "http://presto1.lyft.com", | ||
"routingGroup": "adhoc" | ||
}, | ||
{ | ||
"active": true, | ||
"includeInRouter": true, | ||
"localPort": 8083, | ||
"name": "presto3", | ||
"proxyTo": "http://presto3.lyft.com", | ||
"routingGroup": "adhoc" | ||
} | ||
] | ||
``` | ||
### Activate a backend | ||
|
||
curl -X POST prestogateway.lyft.com/gateway/backend/activate/presto2 | ||
* [Get all backends behind the gateway](/gateway/#get-all-backends-behind-the-gateway) | ||
* [Get active backends behind the Gateway](/gateway/#get-active-backends-behind-the-gateway) | ||
* [Deactivate a backend](/gateway/#deactivate-a-backend) | ||
* [Activate a backend](/gateway/#activate-a-backend) | ||
|
||
Verify this by calling get active backends | ||
``` | ||
curl -X GET localhost:8090/gateway/backend/active | python -m json.tool | ||
## Contributing | ||
|
||
[ | ||
{ | ||
"active": true, | ||
"includeInRouter": true, | ||
"localPort": 8081, | ||
"name": "presto1", | ||
"proxyTo": "http://presto1.lyft.com", | ||
"routingGroup": "adhoc" | ||
}, | ||
{ | ||
"active": true, | ||
"includeInRouter": true, | ||
"localPort": 8082, | ||
"name": "presto2", | ||
"proxyTo": "http://presto2.lyft.com", | ||
"routingGroup": "adhoc" | ||
}, | ||
{ | ||
"active": true, | ||
"includeInRouter": true, | ||
"localPort": 8083, | ||
"name": "presto3", | ||
"proxyTo": "http://presto3.lyft.com", | ||
"routingGroup": "adhoc" | ||
} | ||
] | ||
``` | ||
Want to help build Presto Gateway? Check out our [contributing documentation](CONTRIBUTING.md) |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.