csv-files-spaces-column-names

Importing CSV files with columns containing spaces

Pinot can transform data at ingestion. In this recipe, we'll learn how to use a transformation to change the name of a column. We will ingest a CSV file with a column containing spaces in its name. We will then use a transformation function to remove the space while the data is being ingested into Pinot.

ID,Case Number
10224738,HY411648
10224739,HY411615
11646166,JC213529
10224740,HY411595

Pinot Version	1.0.0
Schema	config/schema.json
Table Config	config/table.json
Ingestion Job	config/job-spec.yml

This is the code for the following recipe: https://dev.startree.ai/docs/pinot/recipes/csv-files-spaces-column-names

---

Clone this repository and navigate to this recipe:

git clone [email protected]:startreedata/pinot-recipes.git
cd pinot-recipes/recipes/csv-files-spaces-column-names

Spin up a Pinot cluster using Docker Compose:

docker-compose up

Open another tab to add the crimes table:

docker run \
   --network csv \
   -v $PWD/config:/config \
   apachepinot/pinot:1.0.0 AddTable \
     -schemaFile /config/schema.json \
     -tableConfigFile /config/table.json \
     -controllerHost "pinot-controller-csv" \
    -exec

Import data/ingest.csv into Pinot:

docker run \
   --network csv \
   -v $PWD/config:/config \
   -v $PWD/data:/data \
   apachepinot/pinot:1.0.0 LaunchDataIngestionJob \
  -jobSpecFile /config/job-spec.yml

Navigate to http://localhost:9000/#/query and run the following query:

select * 
from crimes 
limit 10