-
Notifications
You must be signed in to change notification settings - Fork 1k
MultipleGraphs
If you want to provide OpenTripPlanner services for several distinct geographic areas, you do not need to run several OTP webapps. With a bit of configuration a single OTP instance can load multiple graphs and direct incoming requests to the proper graph.
Most OTP API methods support an optional routerId query parameter specifying which graph the query is to be executed against. When no routerId is provided in a query, OTP falls back on a default routerId which can be configured via the defaultRouterId property of the GraphServiceImpl. The defaultRouterId is the empty string unless set to a different value. RouterIds are strings composed of alphanumeric characters, dashes, and underscores only.
Before handling each request, an OTP server resolves the routerId to a specific graph object. The server maintains a mapping from routerIds to graphs, and a routerId must be registered and associated with a loaded graph to be valid for use in OTP API requests. You can register graphs via Spring configuration or using the routers API at otp/routers/.
A client must send a routerId if it wants to use any graph other than the default one.
A routerId is invalid for use in API requests until it is registered (put), and becomes invalid again if it is de-registered (deleted). When a routerId is registered it is associated with an OTP Graph object, which is typically loaded from a serialized graph file on disk. The filesystem or resource path provided to the GraphServiceImpl via the Spring XML config file is in fact a base path. When a graph is loaded from disk, it will always be found in an immediate subdirectory of that base path, and the subdirectory's name will be the same as the routerId. When the routerId is the empty string, the graph file will be found in the base path itself rather than any subdirectory of that base path.
The config.js configuration file in the opentripplanner-webapp module contains a default empty value for routerId that you can modify for this purpose (NB - if you're specifying the routerId value in this .js file, remember to put the name in quotation marks - e.g. "seattle").
When an OTP server starts up, it will look for a graph on disk corresponding to the defaultRouterId and load it (unless told not to by setting the attemptLoadDefault property to 'false'). Additional routerIds can be registered automatically via the autoRegister property of the GraphServiceImpl component, which takes a list of Strings specifying the routerIds to attempt to register on startup. Here is an example GraphServiceImpl configuration for use in data-sources.xml:
<!-- specify a GraphService, configuring the path to the serialized Graphs -->
<bean id="graphService" class="org.opentripplanner.routing.impl.GraphServiceImpl">
<property name="path" value="/var/otp/graphs/" />
<property name="defaultRouterId" value="pdx" />
<property name="autoRegister">
<list>
<value>nyc</value>
<value>seattle</value>
</list>
</property>
</bean>When using this configuration, upon startup OTP will attempt to register the routerIds pdx, nyc, and seattle, which will resolve to graphs loaded from /var/otp/graphs/pdx/Graph.obj, /var/otp/graphs/nyc/Graph.obj, and /var/otp/graphs/seattle/Graph.obj respectively. If any of these files does not exist or is corrupted or otherwise unreadable, the corresponding routerId will not be registered. All of these properties are optional. Not setting them will result in basic single-graph operation, where the OTP server has a single mapping from the default defaultRouterId (the empty string) to a graph loaded from a serialized graph object found in the GraphServiceImpl's base directory (which defaults to /var/otp/graphs/Graph.obj).
At startup, routerIds can be registered and graphs loaded via the Spring configuration files. We also provide a REST API that allows remotely loading, reloading, and evicting graphs on an OTP server once it is up and running.
A GraphService maintains a mapping between routerIds and specific graph objects. The HTTP verbs are used as follows to manipulate that mapping:
- GET - see registered routerIds and graphs, or verify whether a particular routerId is registered
- PUT - create or replace a mapping from a routerId to a graph loaded from the server filesystem
- POST - create or replace a mapping from a routerId to a serialized graph sent in the request
- DELETE - de-register a routerId, releasing the reference to the associated graph
The HTTP request URLs are of the form /otp/routers/{routerId}, where the routerId is optional. If a routerId is supplied in the URL, the verb will act upon the mapping for that specific routerId. If no routerId is given, the verb will act upon all routerIds currently registered. More specific examples are given below in the sections for each verb. All examples assume that the base path is left at the default of /var/otp/graphs/.
The GET methods are not secured, but all other methods are secured under ROLE_ROUTERS (see the final section of this page on security). Results will be returned in JSON or XML depending on the Accept header of the HTTP request.
For testing or even regular use of this API in scripts, you can use the curl command as follows:
curl -u user:PASS -v -X VERB http://localhost/otp/routers/{routerId}
The -v switch will allow you to see response codes and other information, -u is used to specify the username and password, and -X is used to specify the HTTP verb for the request.
GET http://localhost/otp/routers
will retrieve a list of all registered routerId to graph mappings as well as the geographic bounds of the graphs.
GET http://localhost/otp/routers/london
will return status code 200 and a brief description of the 'london' graph including geographic bounds, or a 404 if the 'london' routerId is not registered.
This documentation is outdated. The current documentation for this method is here.
PUT http://localhost/otp/routers/{routerId}
will attempt to load the graph at /var/otp/graphs/{routerId}/Graph.obj and associate it with the given routerId. If the routerId is already registered, it will be overwritten. If the operation succeeds, OTP will return status code 201 ("created"); upon failure it will return a 404.
This method has a boolean query parameter preEvict, which defaults to true. When preEvict is true, the existing graph will be removed (see DELETE entry below) before loading the new one. This avoids having two copies of the same graph in memory at once, which can cause memory use to double during the reload. If pre-eviction is performed, routing will be unavailable on this routerId for the duration of the load operation.
Thus, PUT http://localhost/otp/routers/berlin?preEvict=false will load or reload the berlin graph from disk, leaving the current graph (if any) available for routing during the load.
If a graph load fails (i.e. if the file is corrupted), the existing routerId mapping will remain unchanged if pre-eviction is turned off, but the mapping will be lost if pre-eviction is on.
Graphs can be reloaded from disk if they have changed (i.e. if you have some automated system producing new graphs when GTFS files change). The PUT verb can also be used for this purpose. Graph reloads must be triggered via the routers API. A graph will not be automatically reloaded when its corresponding file changes on disk because a) this carries a risk of loading a partially written graph file, and b) because graph loading is a relatively slow operation that will either make a routerId unavailable or double memory use for up to several minutes.
The put verb can also be used on the base routers API URL with no routerId:
PUT http://localhost/otp/routers/
will reload the graphs for all currently registered routerIds from disk, just as if you had retrieved a list of routerIds and called PUT on each of them.
There is internal protection against truncated graph files, but you should be careful not to request a reload while another process may be writing to the graph file. If you are still concerned about partially written graphs being loaded, you can use an atomic copy/move.
- On UNIX, use "mv -f"
- Within JAVA, use the Files.move(...) function with the "ATOMIC_MOVE" option set (see http://download.oracle.com/javase/tutorial/essential/io/move.html). It throws an exception if atomic move is not supported by the underlying file system
- On Windows NTFS, use a transactional move (TODO: is there a scriptable command existing here?) (see http://msdn.microsoft.com/en-us/magazine/cc163388.aspx)
(NOTE: was PUT with query parameter upload=true through release 0.9.1)
It is also possible to send a serialized graph over the network to an OTP server using the POST verb. For example, if you have an OTP instance deployed at otp.example.org and a graph saved locally in /var/otp/graphs/london, the command
curl -v -u user:PWD -T /var/otp/graphs/london/Graph.obj -H "Content-Type: application/octet-stream" -X POST "http://otp.example.org/otp/routers/test"
will send the serialized graph over the network, causing it to be deserialized on the server and associated with the routerId 'test'. Like a PUT, it will return status code 201 if the operation succeeds, and 404 if it fails. This request is authenticated, and requires you to configure a user with ROLE_ROUTERS (see last section below).
DELETE http://localhost/otp/routers/{routerId}
will de-register the given routerId and evict the associated graph from memory. It will no longer be available for routing. It will return status code 200 if the routerId was successfully de-registered or a 404 if the routerId was not previously registered.
DELETE http://localhost/otp/routers
will de-register all currently registered routerIds. The OTP server will not be usable for routing until more graphs are loaded and registered.
You may not see memory use drop immediately upon graph eviction because Java is a garbage-collected language. We simply release all references to the graph. Externally visible indicators like used/free memory and the overall size of the heap may not reflect this change until the next time the JVM goes looking for some free memory.
OpenTripPlanner API endpoints that allow modifying sensitive information, as well as those that might put an unreasonable amount of load on the server are secured. This includes the routers API, since it allows you to load graphs into memory or evict them. Security is handled by Spring and is configured in otp-rest-servlet/src/defaults/resources/org/opentripplanner/api/security-application-context.xml. You will need to configure credentials for ROLE_ROUTERS to use the routers API. Below is an example file configuring HTTP basic access authentication, which is far from truly secure but will get you started. Besides ROLE_ROUTERS which grants access to the routers API, you can configure ROLE_USER which grants access to secured endpoints in the OTP internals API.
(NOTE: the role was called ROLE_DEPLOYER rather than ROLE_ROUTERS in the 0.9.0 release but was changed for consistency with the class name)
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:security="http://www.springframework.org/schema/security"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/security
http://www.springframework.org/schema/security/spring-security-3.0.xsd">
<security:global-method-security secured-annotations="enabled" jsr250-annotations="enabled" />
<!-- This is the simple basic auth configuration.-->
<security:http>
<security:http-basic></security:http-basic>
<security:intercept-url method="POST" pattern="/**" access="ROLE_USER" />
</security:http>
<security:authentication-manager alias="authenticationManager">
<security:authentication-provider>
<security:user-service>
<security:user name="{user1}" password="{password1}" authorities="ROLE_USER" />
<security:user name="{user2}" password="{password2}" authorities="ROLE_ROUTERS" />
</security:user-service>
</security:authentication-provider>
</security:authentication-manager>
</beans>unless you are intentionally working with legacy versions of OpenTripPlanner. Please consult the current documentation at readthedocs