Initial commit

.gitignore

@@ -0,0 +1,129 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/

.pre-commit-config.yaml

@@ -0,0 +1,20 @@
+- repo: https://github.com/pre-commit/pre-commit-hooks
+  rev: v2.3.0
+  hooks:
+  - id: trailing-whitespace
+- repo: https://github.com/pre-commit/mirrors-isort
+  rev: master
+  hooks:
+  - id: isort
+- repo: https://github.com/ambv/black
+  rev: stable
+  hooks:
+  - id: black
+    language_version: python3.6
+- repo: https://github.com/pre-commit/pre-commit-hooks
+  rev: v2.4.0
+  hooks:
+  - id: flake8
+    additional_dependencies: ["flake8-blind-except", "flake8-bugbear", "flake8-builtins",
+                              "flake8-comprehensions", "flake8-debugger", "flake8-isort",
+                              "flake8-quotes", "flake8-string-format"]

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2021 Larissa Triess
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,72 @@
+## TF1 Keras implementation for PointNet++ Model
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+![version](https://img.shields.io/badge/version-0.1.0-blue)
+
+This repository holds the implementation of the PointNet++ Model.
+There is no training code contained, it is simply to use the model as a place-in feature extractor in other projects.
+
+The original PointNet++ code can be found [here](https://github.com/charlesq34/pointnet2).
+It also provides the entire training pipeline for PointNet++.
+
+### Installation
+The code requires Python 3.6 and TensorFlow 1.15 GPU version.
+```commandline
+pip install git+https://github.com/ltriess/pointnet2_keras
+```
+If you want to install TF15 alongside, use
+```commandline
+pip install git+https://github.com/ltriess/pointnet2_keras[tf-gpu]  # for gpu support
+pip install git+https://github.com/ltriess/pointnet2_keras[tf-cpu]  # for cpu support
+```
+
+#### Compile Customized TF Operators
+The TF operators are included under `tf_ops`.
+Check `tf_xxx_compile.sh` under each ops subfolder to compile the operators.
+The scripts are tested under TF1.15.
+
+First, find TensorFlow include and library paths.
+```commandline
+TF_CFLAGS=( $(python -c 'import tensorflow as tf; print(" ".join(tf.sysconfig.get_compile_flags()))') )
+TF_LFLAGS=( $(python -c 'import tensorflow as tf; print(" ".join(tf.sysconfig.get_link_flags()))') )
+```
+Then, build the op (name indicated as `xxx`) located in each subfolder.
+```commandline
+/usr/local/cuda/bin/nvcc -std=c++11 -c -o tf_xxx_g.cu.o tf_xxx_g.cu ${TF_CFLAGS[@]} -D GOOGLE_CUDA=1 -x cu -Xcompiler -fPIC
+g++ -std=c++11 -shared -o tf_xxx_so.so tf_xxx.cpp tf_xxx_g.cu.o ${TF_CFLAGS[@]} -fPIC -lcudart ${TF_LFLAGS[@]} -I/usr/local/cuda/include -L/usr/local/cuda/lib64
+```
+
+This implementation does not provide the nearest neighbor op.
+The original implementation is not suited to deal with very large point clouds (the target of our work).
+However, we cannot provide our implementation of the efficient KNN op.
+You can plug in your custom TF ops `my_tf_ops` and use your KNN implementation in `layers/sample_and_group.py`
+(refer to call `from my_tf_ops.knn_op import k_nearest_neighbor_op as get_knn`).
+
+### Usage
+This repo provides the implementation of the PointNet++ model without any additional code.
+In your project, you may use the feature extractor from this implementation.
+We do not provide the implementation of the output heads.
+
+```python
+import tensorflow as tf
+from pointnet2 import PointNet2
+
+feature_extractor = PointNet2(
+    mlp_point=[[64, 64, 128], [128, 128, 256]],
+    num_queries=[100, 10],
+    num_neighbors=[10, 10],
+    radius=[0.2, 0.4],
+    use_knn=False,
+    use_xyz=True,
+    sampling="random",
+)
+
+classifier = ...
+
+points = tf.random.normal(shape=(2, 1000, 3))
+features, _ = feature_extractor(points)
+predictions = classifier(features)
+```
+
+### License
+This code is released under MIT License (see [LICENSE](LICENSE) for details).
+This code is partially adapted from [charlesq34/pointnet2](https://github.com/charlesq34/pointnet2), as indicated in each affected file.

pointnet2/__init__.py

@@ -0,0 +1,9 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+__author__ = "Larissa Triess"
+__email__ = "[email protected]"
+
+from .model import PointNet2
+
+__all__ = ["PointNet2"]

pointnet2/layers/__init__.py

@@ -0,0 +1,5 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+__author__ = "Larissa Triess"
+__email__ = "[email protected]"

pointnet2/layers/normalization.py

@@ -0,0 +1,79 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+__author__ = "Larissa Triess"
+__email__ = "[email protected]"
+
+
+from typing import List
+
+import tensorflow as tf
+
+
+class NeighborhoodNormalization(tf.keras.layers.Layer):
+    def __init__(self):
+        """ Normalize neighborhoods towards their search points """
+        super().__init__()
+
+    def call(self, inputs: List[tf.Tensor], **_) -> tf.Tensor:
+        points = inputs[0]  # tf.Tensor(shape=(*, 3), dtype=tf.float32)
+        neighborhoods = inputs[1]  # tf.Tensor(shape=(*, K, 3), dtype=tf.float32)
+
+        # Get the unit vector from the new origin for the transformed x axis.
+        # u = v / |v| with v being the search point -> new transformed origin.
+        with tf.name_scope("unit_vector"):
+            unit_vec = tf.div_no_nan(
+                points, tf.linalg.norm(points, ord=2, axis=-1, keepdims=True)
+            )
+
+        # The transformation matrix T (from local into global) is defined as
+        # T = [
+        #       [R_00, R_01, R_02, t_0],
+        #       [R_10, R_11, R_12, t_1],
+        #       [R_20, R_21, R_22, t_2],
+        #       [   0,    0,    0,   1]
+        #     ]
+        # with R being the rotation matrix and t being the translation vector.
+        # To transform all coordinates from global to local we use p' = T^(-1) * p
+        # with p being homogeneous coordinates (see below).
+        # The axes are only rotated in the xy-plane. The z-axis is not converted.
+        # The translation vector is equal to the global vector to the search point,
+        # since the local coordinate system has its origin in this point.
+        # Therefore exactly N points are located in the local neighborhoods.
+        # The local x axis is equal to the unit vector of the global search point.
+        with tf.name_scope("transformation_matrix"):
+            ones = tf.ones_like(unit_vec[..., 0])
+            transformation_matrix = tf.convert_to_tensor(
+                [
+                    [unit_vec[..., 0], -unit_vec[..., 1], 0 * ones, points[..., 0]],
+                    [unit_vec[..., 1], unit_vec[..., 0], 0 * ones, points[..., 1]],
+                    [0 * ones, 0 * ones, 1 * ones, points[..., 2]],
+                    [0 * ones, 0 * ones, 0 * ones, 1 * ones],
+                ]
+            )  # --> [4, 4, *]
+            transformation_matrix = tf.broadcast_to(
+                transformation_matrix[..., tf.newaxis],
+                shape=tf.concat([[4, 4], tf.shape(neighborhoods)[:-1]], 0),
+            )  # --> [4, 4, *, K]
+            transformation_matrix = tf.transpose(
+                transformation_matrix,
+                perm=tf.concat(
+                    [tf.range(2, tf.rank(neighborhoods) + 1), [0, 1]], axis=0
+                ),
+            )  # --> [*, K, 4, 4]
+
+        with tf.name_scope("homogeneous_transformation"):
+            # Get homogeneous coordinates for p with p_homo = (p_x, p_y, p_z, 1)
+            homo_knn_points = tf.concat(
+                [neighborhoods, tf.ones_like(neighborhoods[..., 0:1])], axis=-1
+            )  # [*, K, 3] + [*, K, 1] --> [*, K, 4]
+            homo_knn_points = homo_knn_points[..., tf.newaxis]  # --> [*, K, 4, 1]
+            # Transform neighborhood points into their own coordinate system with
+            # p_homo' = T^(-1) * p_homo
+            homo_knn_points = tf.matmul(
+                tf.linalg.inv(transformation_matrix), homo_knn_points
+            )
+            # Retrieve points from homogeneous form.
+            neighborhoods = homo_knn_points[..., :-1, 0]  # [*, K, 4, 1] --> [*, K, 3]
+
+        return neighborhoods