PRISMS-PF_MC_tools

Tools for integration between PRISMS-PF and Materials Commons

The script files in this repository can be used to assist with automation of the following tasks:

• Locally importing PRISMS-PF code files and simulation results into a Materials Commons project directory
• For each calculation (simulation) directory, sorting the different file types into different subdirectories
• Creating a yaml file containing data and metadata for each simulation and adding these data to an ETL spreadsheet associated with an Experiment in Materials Commons
• Generating image frames and movies for different field variables within a phase-field simulation

Installing the PRISMS-PF_MC_tools

Clone the tools repository

$ git clone https://github.com/prisms-center/PRISMS-PF_MC_tools.git

Go to the directory PRISMS-PF_MC_tools and install the tools

$ cd PRISMS-PF_MC_tools

$ make install

After the tools have been installed you should be able to run them from any directory. Type

$ mcpf

to see the available tools.

To uninstall the tools, type

$ make uninstall

from the PRISMS-PF_MC_tools directory.

To take full advantage of these tools you should:

1. Create an account in Materials Commons if you do not already have one
2. Install and configure the Materials Commons Command Line Interface (CLI) in the computer where you usually run PRISMS-PF.
3. Create a project directory in your computer to compile all the data files to be uploaded into a Materials Commons project. To be able to use the CLI, the name of the Materials Commons project should match that of the project locally. Read more about Materials Commons projects here. It is recommended that the project directory is outside you phaseField directory.
4. Within the project directory, create a corresponding project within Materials Commons by typing

$ mc init

All of the scripts and commands below described below should be run under your local project directory.

Importing data from a simulation into the project directory

The script importsim.sh copies data from a source directory where the simulation code and data files are located into a new destination directory within your project directory. Usage

$ mcpf importsim --copy=<ON/OFF> <source directory> <destination directory>

This will copy most of the contents of <source directory> into <destination directory>, organizing the files in the following way

├── project directory
           ├── study directory 1
                      ├── simulation directory 1 (destination directory)
                                 ├── code
                                 ├── description.txt
                                 ├── observations.txt
                                 ├── data
                                            ├── images
                                            ├── movies
                                            ├── postprocess
                                            ├── vtk
                      ├── simulation directory 2 (destination directory)

Options:

--copy=<ON/OFF> Moves vtu, vtk, and pvtu files from <source directory> to <destination directory> instead of copying them.

<destination directory> The destination directory. If this is left blank, the script will use the last folder of the <source directory>.

Generating a simulation data/metadata file

The script generate_yaml.py parses the parameters.prm file, extracts key-value-type triples, and generates a yaml-type file with the data. Usage

$ mcpf generate_yaml <simulation directory> 

The script requires the python packages re, ruamel.yaml, and argparse to run. The yaml file (simlog.yaml) will be generated in the <simulation directory>.

Generating image frames and movies from the simulation results

The script plot_series uses the LLNL visit CLI to generate 2D pseudocolor frames (in png format)from a series of simulation's vtu/vtk files at different time increments. The user can specify the fields for which to generate the frames. Usage

$ mcpf plot_series <var1> <var2> ... <simulation directory>

This script requires python packages os, sys and visit.

The script make_movies uses the ffmpeg package to generate movies from the image series generated by plot_series.

Usage

$ mcpf make_movies <var1> <var2> ... <simulation directory>

The list of variables, <var1> <var2> ... <varN>, must only include variables for which a set of images (created using plot_series) already exist.

Creating an ETL spreadsheet with simulation data

The script add_to_spreadsheet does the following tasks:

1. Extract the simulation data and metadata from the file <simulation directory>/simlog.yaml.
2. Create an Excel file, <etl filename.xlsx>, and write the simulation data in a single row, where each column corresponds to a different parameter.
3. If the file <etl filename.xlsx> already exists, the script opens the file and appends the simulation data in the first unpopulated row.
4. Prompt the user for a description of the simulation and create the file <simulation directory>/description.txt. This file can be edited later or left empty.
5. Prompt the user for observations pertaining to the simulation and create the files file <simulation directory>/observations.txt. This file can be edited lateror left empty.
6. Write columns indicating the paths to the code files and simulation files relative to the project directory. This will facilitate access to the data when add_to_spreadsheet is associated with a Materials Commons Study (formerly known as 'Experiment').

Usage

$ mcpf add_to_spreadsheet <simulation directory> <etl filename.xlsx>

This script requires python packages os, sys and yaml, and pandas.

After the steps above are carried out, all the simulation files and directories can be uploaded to Materials Commons under the remote directory

$ mc up -r .

This will essentially update the remote Materials Commons project with all the new files and subdirectories, without uploading existing files that have not been modified.