nnictl

Introduction

nnictl is a command line tool, which can be used to control experiments, such as start/stop/resume an experiment, start/stop NNIBoard, etc.

Commands

nnictl support commands:

nnictl create
nnictl resume
nnictl view
nnictl stop
nnictl update
nnictl trial
nnictl top
nnictl experiment
nnictl platform
nnictl config
nnictl log
nnictl webui
nnictl tensorboard
nnictl package
nnictl --version

Manage an experiment

nnictl create

Description

You can use this command to create a new experiment, using the configuration specified in config file.

After this command is successfully done, the context will be set as this experiment, which means the following command you issued is associated with this experiment, unless you explicitly changes the context(not supported yet).
Usage
```
nnictl create [OPTIONS]
```
Options

Name, shorthand Required Default Description

--config, -c True YAML configure file of the experiment

--port, -p False the port of restful server

--debug, -d False set debug mode

Examples

create a new experiment with the default port: 8080

nnictl create --config nni/examples/trials/mnist/config.yml

create a new experiment with specified port 8088

nnictl create --config nni/examples/trials/mnist/config.yml --port 8088

create a new experiment with specified port 8088 and debug mode

nnictl create --config nni/examples/trials/mnist/config.yml --port 8088 --debug

Note:

Debug mode will disable version check function in Trialkeeper.

nnictl resume

Description

You can use this command to resume a stopped experiment.
Usage
```
nnictl resume [OPTIONS]
```

Options

Name, shorthand	Required	Description
id	True	The id of the experiment you want to resume
--port, -p	False	Rest port of the experiment you want to resume
--debug, -d	False	set debug mode

Example

resume an experiment with specified port 8088
```
nnictl resume [experiment_id] --port 8088
```

nnictl view

Description

You can use this command to view a stopped experiment.
Usage
```
nnictl view [OPTIONS]
```
Options

Name, shorthand Required Default Description

id True The id of the experiment you want to view

--port, -p False Rest port of the experiment you want to view
Example

view an experiment with specified port 8088
```
nnictl view [experiment_id] --port 8088
```

nnictl stop

Description

You can use this command to stop a running experiment or multiple experiments.
Usage
```
nnictl stop [Options]
```

Options

Name, shorthand	Required	Description
id	False	The id of the experiment you want to stop
--port, -p	False	Rest port of the experiment you want to stop
--all, -a	False	Stop all of experiments

Details & Examples
1. If there is no id specified, and there is an experiment running, stop the running experiment, or print error message.
```
nnictl stop
```
2. If there is an id specified, and the id matches the running experiment, nnictl will stop the corresponding experiment, or will print error message.
```
nnictl stop [experiment_id]
```
3. If there is a port specified, and an experiment is running on that port, the experiment will be stopped.
```
nnictl stop --port 8080
```
4. Users could use 'nnictl stop --all' to stop all experiments.
```
nnictl stop --all
```
5. If the id ends with *, nnictl will stop all experiments whose ids matchs the regular.
6. If the id does not exist but match the prefix of an experiment id, nnictl will stop the matched experiment.
7. If the id does not exist but match multiple prefix of the experiment ids, nnictl will give id information.

nnictl update

nnictl update searchspace
- Description
  
  You can use this command to update an experiment's search space.
- Usage
```
nnictl update searchspace [OPTIONS]
```
- Options
Name, shorthand Required Default Description

id False ID of the experiment you want to set

--filename, -f True the file storing your new search space
- Example
  
  update experiment's new search space with file dir 'examples/trials/mnist/search_space.json'
```
nnictl update searchspace [experiment_id] --filename examples/trials/mnist/search_space.json
```
nnictl update concurrency
- Description
  
  You can use this command to update an experiment's concurrency.
- Usage
```
nnictl update concurrency [OPTIONS]
```
- Options
Name, shorthand Required Default Description

id False ID of the experiment you want to set

--value, -v True the number of allowed concurrent trials
- Example
  
  update experiment's concurrency
```
nnictl update concurrency [experiment_id] --value [concurrency_number]
```

nnictl update duration

Description

You can use this command to update an experiment's duration.
Usage
```
nnictl update duration [OPTIONS]
```
Options

Name, shorthand	Required	Default	Description
id	False		ID of the experiment you want to set
--value, -v	True		the experiment duration will be NUMBER seconds. SUFFIX may be 's' for seconds (the default), 'm' for minutes, 'h' for hours or 'd' for days.

Example

update experiment's duration

nnictl update duration [experiment_id] --value [duration]

nnictl update trialnum
- Description
  
  You can use this command to update an experiment's maxtrialnum.
- Usage
```
nnictl update trialnum [OPTIONS]
```
- Options
Name, shorthand Required Default Description

id False ID of the experiment you want to set

--value, -v True the new number of maxtrialnum you want to set
- Example
  
  update experiment's trial num
```
nnictl update trialnum --id [experiment_id] --value [trial_num]
```

nnictl trial

nnictl trial ls
- Description
  
  You can use this command to show trial's information.
- Usage
```
nnictl trial ls
```
- Options
Name, shorthand Required Default Description

id False ID of the experiment you want to set
nnictl trial kill
- Description
  
  You can use this command to kill a trial job.
- Usage
```
nnictl trial kill [OPTIONS]
```
- Options
Name, shorthand Required Default Description

id False Experiment ID of the trial

--trial_id, -T True ID of the trial you want to kill.
- Example
  
  kill trail job
```
nnictl trial [trial_id] --experiment [experiment_id]
```

nnictl top

Description

Monitor all of running experiments.
Usage
```
nnictl top
```

Options

Name, shorthand	Required	Default	Description
id	False		ID of the experiment you want to set
--time, -t	False		The interval to update the experiment status, the unit of time is second, and the default value is 3 second.

Manage experiment information

nnictl experiment show
- Description
  
  Show the information of experiment.
- Usage
```
nnictl experiment show
```
- Options
Name, shorthand Required Default Description

id False ID of the experiment you want to set
nnictl experiment status
- Description
  
  Show the status of experiment.
- Usage
```
nnictl experiment status
```
- Options
Name, shorthand Required Default Description

id False ID of the experiment you want to set
nnictl experiment list
- Description
  
  Show the information of all the (running) experiments.
- Usage
```
nnictl experiment list [OPTIONS]
```
- Options
Name, shorthand Required Default Description

--all False list all of experiments
nnictl experiment delete
- Description
  
  Delete one or all experiments, it includes log, result, environment information and cache. It uses to delete useless experiment result, or save disk space.
- Usage
```
nnictl experiment delete [OPTIONS]
```
- Options
Name, shorthand Required Default Description

id False ID of the experiment

--all False delete all of experiments

nnictl experiment export

Description

You can use this command to export reward & hyper-parameter of trial jobs to a csv file.
Usage
```
nnictl experiment export [OPTIONS]
```
Options

Name, shorthand	Required	Description
id	False	ID of the experiment
--filename, -f	True	File path of the output file
--type	True	Type of output file, only support "csv" and "json"

Examples

export all trial data in an experiment as json format

nnictl experiment export [experiment_id] --filename [file_path] --type json

nnictl experiment import
- Description
  
  You can use this command to import several prior or supplementary trial hyperparameters & results for NNI hyperparameter tuning. The data are fed to the tuning algorithm (e.g., tuner or advisor).
- Usage
```
nnictl experiment import [OPTIONS]
```
- Options
Name, shorthand Required Default Description

id False The id of the experiment you want to import data into

--filename, -f True a file with data you want to import in json format
- Details
  
  NNI supports users to import their own data, please express the data in the correct format. An example is shown below:
```
[
  {"parameter": {"x": 0.5, "y": 0.9}, "value": 0.03},
  {"parameter": {"x": 0.4, "y": 0.8}, "value": 0.05},
  {"parameter": {"x": 0.3, "y": 0.7}, "value": 0.04}
]
```
  Every element in the top level list is a sample. For our built-in tuners/advisors, each sample should have at least two keys: parameter and value. The parameter must match this experiment's search space, that is, all the keys (or hyperparameters) in parameter must match the keys in the search space. Otherwise, tuner/advisor may have unpredictable behavior. Value should follow the same rule of the input in nni.report_final_result, that is, either a number or a dict with a key named default. For your customized tuner/advisor, the file could have any json content depending on how you implement the corresponding methods (e.g., import_data).
  
  You also can use nnictl experiment export to export a valid json file including previous experiment trial hyperparameters and results.
  
  Currently, following tuner and advisor support import data:
```
builtinTunerName: TPE, Anneal, GridSearch, MetisTuner
builtinAdvisorName: BOHB
```
  If you want to import data to BOHB advisor, user are suggested to add "TRIAL_BUDGET" in parameter as NNI do, otherwise, BOHB will use max_budget as "TRIAL_BUDGET". Here is an example:
```
[
  {"parameter": {"x": 0.5, "y": 0.9, "TRIAL_BUDGET": 27}, "value": 0.03}
]
```
- Examples
  
  import data to a running experiment
```
nnictl experiment import [experiment_id] -f experiment_data.json
```

Manage platform information

nnictl platform clean
- Description
  
  It uses to clean up disk on a target platform. The provided YAML file includes the information of target platform, and it follows the same schema as the NNI configuration file.
- Note
  
  if the target platform is being used by other users, it may cause unexpected errors to others.
- Usage
```
nnictl platform clean [OPTIONS]
```
- Options
Name, shorthand Required Default Description

--config True the path of yaml config file used when create an experiment

nnictl config show

Description

Display the current context information.
Usage
```
nnictl config show
```

Manage log

nnictl log stdout

Description

Show the stdout log content.
Usage
```
nnictl log stdout [options]
```
Options

Name, shorthand	Required	Description
id	False	ID of the experiment you want to set
--head, -h	False	show head lines of stdout
--tail, -t	False	show tail lines of stdout
--path, -p	False	show the path of stdout file

Example

Show the tail of stdout log content

nnictl log stdout [experiment_id] --tail [lines_number]

nnictl log stderr

Description

Show the stderr log content.
Usage
```
nnictl log stderr [options]
```
Options

Name, shorthand	Required	Description
id	False	ID of the experiment you want to set
--head, -h	False	show head lines of stderr
--tail, -t	False	show tail lines of stderr
--path, -p	False	show the path of stderr file

nnictl log trial
- Description
  
  Show trial log path.
- Usage
```
nnictl log trial [options]
```
- Options
Name, shorthand Required Default Description

id False Experiment ID of the trial

--trial_id, -T False ID of the trial to be found the log path, required when id is not empty.

Manage webui

nnictl webui url

Manage tensorboard

nnictl tensorboard start
- Description
  
  Start the tensorboard process.
- Usage
```
nnictl tensorboard start
```
- Options
Name, shorthand Required Default Description

id False ID of the experiment you want to set

--trial_id, -T False ID of the trial

--port False 6006 The port of the tensorboard process
- Detail
  1. NNICTL support tensorboard function in local and remote platform for the moment, other platforms will be supported later.
  2. If you want to use tensorboard, you need to write your tensorboard log data to environment variable [NNI_OUTPUT_DIR] path.
  3. In local mode, nnictl will set --logdir=[NNI_OUTPUT_DIR] directly and start a tensorboard process.
  4. In remote mode, nnictl will create a ssh client to copy log data from remote machine to local temp directory firstly, and then start a tensorboard process in your local machine. You need to notice that nnictl only copy the log data one time when you use the command, if you want to see the later result of tensorboard, you should execute nnictl tensorboard command again.
  5. If there is only one trial job, you don't need to set trial id. If there are multiple trial jobs running, you should set the trial id, or you could use [nnictl tensorboard start --trial_id all] to map --logdir to all trial log paths.
nnictl tensorboard stop
- Description
  
  Stop all of the tensorboard process.
- Usage
```
nnictl tensorboard stop
```
- Options
Name, shorthand Required Default Description

id False ID of the experiment you want to set

Manage package

nnictl package install
- Description
  
  Install the packages needed in nni experiments.
- Usage
```
nnictl package install [OPTIONS]
```
- Options
Name, shorthand Required Default Description

--name True The name of package to be installed
- Example
  
  Install the packages needed in tuner SMAC
```
nnictl package install --name=SMAC
```
nnictl package show
- Description
  
  List the packages supported.
- Usage
```
nnictl package show
```

Check NNI version

nnictl --version
- Description
  
  Describe the current version of NNI installed.
- Usage
```
nnictl --version
```

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nnictl.md

Nnictl.md

nnictl

Introduction

Commands

Manage an experiment

Name, shorthand	Required	Description
--config, -c	True	YAML configure file of the experiment
--port, -p	False	the port of restful server
--debug, -d	False	set debug mode

Name, shorthand	Required	Default	Description
id	True		The id of the experiment you want to view
--port, -p	False		Rest port of the experiment you want to view

Name, shorthand	Required	Default	Description
id	False		Experiment ID of the trial
--trial_id, -T	True		ID of the trial you want to kill.

Name, shorthand	Required	Default	Description
id	False		ID of the experiment
--all	False		delete all of experiments

Name, shorthand	Required	Default	Description
id	False		The id of the experiment you want to import data into
--filename, -f	True		a file with data you want to import in json format

Files

Nnictl.md

Latest commit

History

Nnictl.md

File metadata and controls

nnictl

Introduction

Commands

Manage an experiment