Skip to content

Latest commit

 

History

History
70 lines (57 loc) · 1.45 KB

README.md

File metadata and controls

70 lines (57 loc) · 1.45 KB
Raw

Import data files from different directories

In this recipe we'll learn how to import data files from different directories.

Pinot Version 1.0.0
Schema config/schema.json
Table Config config/table.json
Ingestion Job config/job-spec.yml

Clone this repository and navigate to this recipe:

git clone [email protected]:startreedata/pinot-recipes.git
cd pinot-recipes/recipes/import-csv-files-different-directories

Spin up a Pinot cluster using Docker Compose:

docker-compose up

Open another tab to add the movies table:

docker run \
   --network csv \
   -v $PWD/config:/config \
   apachepinot/pinot:1.0.0 AddTable \
     -tableConfigFile /config/table.json   \
     -schemaFile /config/schema.json \
     -controllerHost "pinot-controller-csv" \
    -exec

Import the CSV files from the input directory into Pinot:

docker run \
   --network csv \
   -v $PWD/config:/config \
   -v $PWD/data:/data \
   -v $PWD/input:/input \
   apachepinot/pinot:1.0.0 LaunchDataIngestionJob \
  -jobSpecFile /config/job-spec.yml

Navigate to http://localhost:9000/#/query and run the following query:

select * 
from movies 
limit 10