Command line

cLASpy_T could be used through command line or with Graphical User Interface.

cLASpy_T is devided into 3 main modules: train, predict and segment.

  • train: performs training model according the selected supervised machine learning algorithm and the provided dataset. The data file must contain fields of features that describe each point and the target field of labels as integers.

  • predict: performs predictions for new dataset according a previous trained model. The new dataset must contain the same fields of features that the dataset used to train the model. cLASpy_T ignore any ‘target’ field if the new dataset has one.

  • segment: performs cluster segmentation of a dataset according KMeans algorithm (see scikit-learn documentation).

Note

Command line mode is the main and stablest way to use cLASpy_T. Do not forget to activate your python virtual environment before calling cLASpy_T.

Command line mode uses the 3 main modules as subcommands of the main software, i.e. train, predict and segment.

For example, if you want to train model:

python cLASpy_T.py train -a=rf -i=/home/me/data/lidar_training_dataset.las

To make predictions:

python cLASpy_T.py predict -m=/home/me/results/lidar_dataset.model -i=/home/me/data/lidar_survey.las

And to segment a dataset:

python cLASpy_T.py segment -i=/home/me/data/lidar_survey.las

‘train’ module

The ‘train’ module is used to create a supervised model from the machine learning algorithm that you select. cLASpy_T uses scikit-learn library as main machine learning library, so do not hesitate to look up the documentation.

Available supervised algorithms

Currently, there are 3 available supervised machine learning algorithms from scikit-learn:

Format of data files

The input data must be in LAS or CSV (sep=’,’) formats.

Example of CSV file:

X,Y,Z,Target,Intensity,Red,Green,Blue,Roughness (5),Omnivariance (5),Sphericity (5)...
638.957,916.201,-2.953,1,39.0,104,133,113,0.11013,0.63586,0.00095...

Data file must contain:

  • Target field named ‘target’ (non case-sensitive): contains the labels for training as integer.

  • Features that describe each point.

If X, Y and/or Z fields provided, they are discard for training, but re-used to write the output file.

To use ‘Intensity’ field from LAS file, rename it as, for example, ‘Original_Intensity’ or ‘Amplitude’.

Arguments

  • -h, --help Show this help message and exit.

  • -a, --algo Set the supervised machine learning algorithm: ‘rf’, ‘gb’, ‘ann’.

    • rf > RandomForestClassifier

    • gb > GradientBoostingClassifier

    • ann > MLPClassifier

  • -c, --config Give the configuration file with all parameters and selected scalar fields.

    • on Windows: C:\path\to\the\config.json

    • on Linux: /path/to/the/config.json

  • -i, --input_data Set the input file of the dataset (LAS or CSV).

    • on Windows: C:\path\to\the\input_data.las

    • on Linux: /path/to/the/input_data.las

  • -o, --output Set the output folder to save all results. Default: Create folder with the path of the input file.

    • on Windows: C:\path\to\the\output_folder

    • on Linux: /path/to/the/output_folder

  • -f, --features Select the features to used to train the model. Give a list of feature names. Caution: Replace whitespaces by underscores ‘_’.

-f=['Anisotropy_5m', 'R', 'G', 'B', ...]

  • -g, --grid_search Perform the training with GridSearchCV (see `scikit-learn`_ documentation).

  • -k, --param_grid Set the parameters to pass to the GridSearchCV as lists in a dictionary. If empty, GridSearchCV uses presets. Wrong parameters will be ignored. Caution: Replace whitespaces by underscores ‘_’.

-k="{'n_estimators':[50,100,500],'loss':['deviance', 'exponential'],'hidden_layer_sizes':[[100,100],[50,100,50]]}"

  • -n, --n_jobs Set the number of threads to use, ‘-1’ means all available threads. Default: -1.

  • -p, --parameters Set the parameters to pass to the classifier for training, as a dictionary. Caution: Replace whitespaces by underscores ‘_’.

-p="{'n_estimators':50,'max_depth':5,'max_iter':500}"

  • --pca Set the Principal Component Analysis and the number of principal components.

  • --png_features Export the feature importnaces from RandomForest and GradientBoosting algorithms as PNG image.

  • --random_state Set the random_state to split dataset in the GridSearchCV and cross-validation.

  • -s, --samples Set the number of samples for large dataset (float in million points). samples = train_set + test_set.

  • --scaler Set the method to scale the dataset before training. Default: ‘Standard’.

    • Standard: StandardScaler > Standardize features by removing the mean and scaling to unit variance.

    • MinMax: MinMaxScaler > Transform features by scaling each feature to a given range, e.g. betwen zero and one.

    • Robust: RobustScaler > Scale features using statistics that are robust to outliers, e.g. between 1st and 3rd quartile.

  • --scoring Set scorer for GridSearchCV or cross_val_score. Default: ‘accuracy’. See the `scikit-learn_ documentation.

  • --train_r Set the train ratio as float [0.0 - 1.0] to split data into train and test datasets. Default: 0.5.

‘predict’ module

The ‘predict’ module allows to make predictions of an entire dataset using a pre-trained model, created with the ‘train’ module.

To make predictions, 2 files are required:

  • The input dataset, as LAS or CSV (sep=’,’) formats.

  • The pre-trained model, as ‘*.model’.

Input datasets:

The input dataset must contain the same features used to train the model. The ‘predict’ module starts with comparaison between the features used by the model and the features in the input dataset. If one or more features are missing, the ‘predict’ module returns an error. If the input dataset contains more features than necessary, these features are discarded for predictions.

The ‘target’ field (non case-sensitive), containing the labels as integer, is not mandatory. If a ‘target’ field is found in the input dataset, this field is discarded for predictions, but used at the end to make a confusion matrix and compute the scores.

Model:

The ‘*.model’ file is created during the training phase. It contains the model itself, as well as the scaler used to transform the input data, the features used to train the model and the PCA where applicable.

Arguments

  • -h, --help Show this help message and exit.

  • -c, --config Give the configuration file with all parameters and selected scalar fields.

    • on Windows: C:\path\to\the\config.json

    • on Linux: /path/to/the/config.json

  • -i, --input_data Set the input file of the dataset (LAS or CSV).

    • on Windows: C:\path\to\the\input_data.las

    • on Linux: /path/to/the/input_data.las

  • -o, --output Set the output folder to save all results. Default: Create folder with the path of the input file.

    • on Windows: C:\path\to\the\output_folder

    • on Linux: /path/to/the/output_folder

  • -m, --model Import the model file to make predictions.

    • on Windows: C:\path\to\the\model_file.model

    • on Linux: /path/to/the/model_file.model

‘segment’ module

The ‘segment’ module segments an entire dataset according to the number of clusters defined.

To segment a dataset, only 1 file is required:

  • The input dataset, as LAS or CSV (sep=’,’) formats.

Input datasets:

The input dataset must contain some features that describe the points.

Any fields named ‘target’ (non case-sensitive), containing the labels as integer, are discarded for segmentation.

Arguments

  • -h, --help Show this help message and exit.

  • -c, --config Give the configuration file with all parameters and selected scalar fields.

    • on Windows: C:\path\to\the\config.json

    • on Linux: /path/to/the/config.json

  • -i, --input_data Set the input file of the dataset (LAS or CSV).

    • on Windows: C:\path\to\the\input_data.las

    • on Linux: /path/to/the/input_data.las

  • -o, --output Set the output folder to save all results. Default: Create folder with the path of the input file.

    • on Windows: C:\path\to\the\output_folder

    • on Linux: /path/to/the/output_folder

  • -f, --features Set the features to used to segment the dataset, as a list of feature names. Caution: Replace whitespaces by underscores ‘_’.

-f=['Anisotropy_5m', 'R', 'G', 'B', ...]

  • -p, --parameters Set the parameters to pass to the clustering algorithm, as a dictionary. Caution: Replace whitespaces by underscores ‘_’.

-p="{'n_estimators':50,'max_depth':5,'max_iter':500}"