From a196b93e38353ec991c3de4791300439eb5f2ba9 Mon Sep 17 00:00:00 2001 From: paryoja Date: Thu, 4 Nov 2021 12:17:26 +0900 Subject: [PATCH] Update 01-ug-quick-start-guide.md --- docs/_docs/01-ug-quick-start-guide.md | 72 ++++++++++++++++++++++----- 1 file changed, 59 insertions(+), 13 deletions(-) diff --git a/docs/_docs/01-ug-quick-start-guide.md b/docs/_docs/01-ug-quick-start-guide.md index 659f17144..0b921fb8e 100644 --- a/docs/_docs/01-ug-quick-start-guide.md +++ b/docs/_docs/01-ug-quick-start-guide.md @@ -239,16 +239,62 @@ hs.explain(query, verbose = True) #### Enable Hyperspace -Now that you have created an index that your query can utilize, you can enable Hyperspace and execute your query: - -Scala: -```scala -spark.enableHyperspace -query.show -``` - -Python: -```python -Hyperspace.enable(spark) -query.show() -``` +Now that you have created an index that your query can utilize, you can enable Hyperspace and execute your query. +There are two ways to enable Hyperspace: + +1. Using Hyperspace extension + + You can start a session with Hyperspace extension. + The follows are examples to use Sparkā„¢ with Hyperspace. + + spark-submit: + ``` + spark-submit -c spark.sql.extensions=com.microsoft.hyperspace.HyperspaceSparkSessionExtension ... + ``` + + Scala: + ```scala + val spark = SparkSession + .builder() + .appName("...") + .master("...") + .config("spark.sql.extensions", "com.microsoft.hyperspace.HyperspaceSparkSessionExtension") + .getOrCreate() + ``` + + Python: + ```python + from pyspark.sql import SparkSession + + spark = SparkSession \ + .builder \ + .appName("...") \ + .master("...") \ + .config("spark.sql.extensions", "com.microsoft.hyperspace.HyperspaceSparkSessionExtension") \ + .getOrCreate() + ``` + + Spark configuration file (default: `$SPARK_HOME/conf/spark-defaults.conf`): + + ```cs + # Add the following line to the configuration file. + + spark.sql.extensions com.microsoft.hyperspace.HyperspaceSparkSessionExtension + ``` + + +2. Calling enableHyperspace function explicitly + + By explicitly calling enableHyperspace (for Scala) or Hyperspace.enable (for Python), Spark will use Hyperspace indexes when it is applicable. + + Scala: + ```scala + spark.enableHyperspace + query.show + ``` + + Python: + ```python + Hyperspace.enable(spark) + query.show() + ```