Skip to content

Commit

Permalink
Removed OSCache. Using EHCache and Guava instead. (#67)
Browse files Browse the repository at this point in the history
* Gateway HA implementation

* perf: Utilize EHCache and Guava cache only

* fix: changes after review

* docs: update gateway-ha README.md

* fix: updated after review

* fix: replaced cacheDir with gatewayStateDir
  • Loading branch information
Preethi G authored and puneetjaiswal committed Sep 6, 2019
1 parent c8b3bee commit 0c52b95
Show file tree
Hide file tree
Showing 20 changed files with 454 additions and 223 deletions.
154 changes: 10 additions & 144 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,155 +1,21 @@
# presto-gateway
A load balancer / proxy / gateway for presto compute engine.

## Getting Started

### Build and run
run `mvn clean install` to build `presto-gateway`

Edit the [config file](gateway/src/main/resources/config.yml.template) and update backend urls
A load balancer / proxy / gateway for presto compute engine.

```
cd gateway/target/
java -jar gateway-{{VERSION}}-jar-with-dependencies.jar server ../src/presto-gateway/gateway/src/main/resources/config.yml.template
```
Now you can access load balanced presto at localhost:8080 port. We will refer to this as `prestogateway.lyft.com`

### Query History UI - check query plans etc.
PrestoGateway records history of recent queries and displays links to check query details page in respective presto cluster.
![prestogateway.lyft.com](docs/assets/prestogateway_query_history.png)
## Getting started

### Adhoc vs Scheduled query routing
In the [config](gateway/src/main/resources/config.yml.template)
you can specify if a backend cluster is a part of a routing group.
If not specified, cluster will be part of `adhoc` routing group by default.
Presto Gateway can be deployed as a standalone stateless service or with high availability and external database. For help with setting up a standalone gateway service, please visit [Standalone Gateway](/gateway/#standalone-gateway). For help with setting up a gateway service with high availability, please visit [Gateway-HA](/gateway-ha#gateway-ha).

PrestoGateway router will route any request to `scheduled` group of clusters if request contains header `X-Presto-Routing-Group: scheduled` and there are clusters present with config `routingGroup: scheduled`.
For further information regarding the design of the gateway, please visit the [Gateway Design Document](/docs/design.md)

If no matching `routingGroup` found, router will route to `adhoc` group of clusters. Please make sure there are clusters under `adhoc` routing group.

## Gateway API

### Get all backends behind the gateway

`curl -X GET prestogateway.lyft.com/gateway/backend/all | python -m json.tool`
```
[
{
"active": true,
"includeInRouter": true,
"localPort": 8081,
"name": "presto1",
"proxyTo": "http://presto1.lyft.com",
"routingGroup": "adhoc"
},
{
"active": true,
"includeInRouter": true,
"localPort": 8083,
"name": "presto3",
"proxyTo": "http://presto3.lyft.com",
"routingGroup": "adhoc"
},
{
"active": true,
"includeInRouter": true,
"localPort": 8082,
"name": "presto2",
"proxyTo": "http://presto2.lyft.com",
"routingGroup": "adhoc"
}
]
```

### Get active backends behind the Gateway

`curl -X GET prestogateway.lyft.com/gateway/backend/active | python -m json.tool`
```
[
{
"active": true,
"includeInRouter": true,
"localPort": 8081,
"name": "presto1",
"proxyTo": "http://presto1.lyft.com",
"routingGroup": "adhoc"
},
{
"active": true,
"includeInRouter": true,
"localPort": 8082,
"name": "presto2",
"proxyTo": "http://presto2.lyft.com",
"routingGroup": "adhoc"
},
{
"active": true,
"includeInRouter": true,
"localPort": 8083,
"name": "presto3",
"proxyTo": "http://presto3.lyft.com",
"routingGroup": "adhoc"
}
]
```
### Deactivate a backend

`curl -X POST prestogateway.lyft.com/gateway/backend/deactivate/presto2`

Verify this by calling get active backends
```
curl -X GET prestogateway.lyft.com/gateway/backend/active | python -m json.tool
[
{
"active": true,
"includeInRouter": true,
"localPort": 8081,
"name": "presto1",
"proxyTo": "http://presto1.lyft.com",
"routingGroup": "adhoc"
},
{
"active": true,
"includeInRouter": true,
"localPort": 8083,
"name": "presto3",
"proxyTo": "http://presto3.lyft.com",
"routingGroup": "adhoc"
}
]
```
### Activate a backend

curl -X POST prestogateway.lyft.com/gateway/backend/activate/presto2
* [Get all backends behind the gateway](/gateway/#get-all-backends-behind-the-gateway)
* [Get active backends behind the Gateway](/gateway/#get-active-backends-behind-the-gateway)
* [Deactivate a backend](/gateway/#deactivate-a-backend)
* [Activate a backend](/gateway/#activate-a-backend)

Verify this by calling get active backends
```
curl -X GET localhost:8090/gateway/backend/active | python -m json.tool
## Contributing

[
{
"active": true,
"includeInRouter": true,
"localPort": 8081,
"name": "presto1",
"proxyTo": "http://presto1.lyft.com",
"routingGroup": "adhoc"
},
{
"active": true,
"includeInRouter": true,
"localPort": 8082,
"name": "presto2",
"proxyTo": "http://presto2.lyft.com",
"routingGroup": "adhoc"
},
{
"active": true,
"includeInRouter": true,
"localPort": 8083,
"name": "presto3",
"proxyTo": "http://presto3.lyft.com",
"routingGroup": "adhoc"
}
]
```
Want to help build Presto Gateway? Check out our [contributing documentation](CONTRIBUTING.md)
Binary file added docs/assets/prestogateway_ha_admin.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 0c52b95

Please sign in to comment.