Apache Arrow

• arrow.apache.org
  • github
  • docs
  • install

Developing inside a Container

• containers.dev
  • reference
• github: devcontainers
  • images
  • features
• vscode
  • docs: Developing inside a Container

Launch the container

Launch a Apache Arrow development environment:

1. Start VS Code
2. Open Command Palette with F1
3. Run: Dev Containers: Open Folder in Container...

Hello World

• python: Hello World

cd /workspace/python/helloworld
python -m helloworld

Attach to Hadoop Nodes

docker exec -it apache-arrow_devcontainer-namenode-1 bash
docker exec -it apache-arrow_devcontainer-datanode-1 bash

hdfs dfs -ls -R /

drwxr-xr-x   - vscode supergroup          0 2024-03-25 02:59 /user
drwxr-xr-x   - vscode supergroup          0 2024-03-25 02:59 /user/vscode
-rw-r--r--   3 vscode supergroup       1247 2024-03-25 02:59 /user/vscode/birthdays.parquet
drwxr-xr-x   - vscode supergroup          0 2024-03-25 02:59 /user/vscode/output
-rw-r--r--   3 vscode supergroup       1207 2024-03-25 02:59 /user/vscode/output/0.parquet
-rw-r--r--   3 vscode supergroup       1207 2024-03-25 02:59 /user/vscode/output/1.parquet
-rw-r--r--   3 vscode supergroup       1207 2024-03-25 02:59 /user/vscode/output/2.parquet