+
+## Overview
+
+
+## Scope
+This repository was initiated as an entry for the [Bayesian Optimization Hackathon for Chemistry and Materials](https://ac-bo-hackathon.github.io/), held on March 27-28, 2024, and sponsored by the Acceleration Consortium and Merck KGaA. Here, we propose Bayesian optimization within the field of zeolite synthesis. This concept is also explained in a [short video](https://www.youtube.com/watch?v=4lFEUixwkE8).
+
+Despite their significant industrial applications as catalysts, ion exchangers and adsorbents, the synthesis of zeolites predominantly relies on heuristics, experience and a sprinkle of magic. Employing Bayesian optimization has the potential to swiftly navigate the extensive parameter space in zeolite synthesis research and reduce associated costs.
+
+## [Introductory text](./zeolite_synthesis_bo_introduction.md) on zeolites, zeolite synthesis and Bayesian optimization
+In [zeolite_synthesis_bo_introduction.md](./zeolite_synthesis_bo_introduction.md) we provide an overview of the following topics:
+1. **Zeolites:** definition, properties and relevance in industry
+2. **Zeolite Synthesis:** typical procedures, ingredients and equipments
+3. **Zeolite Synthesis Optimization:** the limited literature on active learning for zeolite synthesis is discussed, and considerations regarding BO:
+ 3.1 Parameter space of a zeolite synthesis
+ 3.2 Constraints of a zeolite synthesis
+ 3.3 Objectives that might be pursued in zeolite synthesis
+ 3.4 Zeolite synthesis datasets that are available in literature
+4. **Discussion:** various aspects of BO in zeolite synthesis are discussed, and how they can be tackled
+5. **References**
+
+While numerous references are provided for further exploration, this document is self-contained and aims to be easily understood. We hope it inspires the reader to consider active learning approaches in their zeolite synthesis endeavors.
+This introductory text is also provided as [pdf](./zeolite_synthesis_bo_introduction.pdf).
+
+## [Notebook](./demo_zeolite_synthesis_bo.ipynb) tutorial on Bayesian optimization for zeolite synthesis (with [Ax](https://ax.dev/))
+Within the [demo_zeolite_synthesis_bo.ipynb](./demo_zeolite_synthesis_bo.ipynb) notebook, we illustrate the concepts of the introductory text with code, leveraging real-world literature data acquired through grid search in [Table S4](https://pubs.acs.org/doi/suppl/10.1021/acs.chemmater.9b03738/suppl_file/cm9b03738_si_001.pdf#page=10) and [Table S3](https://pubs.acs.org/doi/suppl/10.1021/jacs.1c07590/suppl_file/ja1c07590_si_001.pdf#page=9) in the Supporting Information of respectively [*Chem. Mater.* **2020**, *32*, 273â285](https://pubs.acs.org/doi/abs/10.1021/acs.chemmater.9b03738) and [*J. Am. Chem. Soc.* **2021**, *143*, 16243â16255](https://pubs.acs.org/doi/10.1021/jacs.1c07590).
+
+This notebook is divided into two sections:
+1. The first section uses Bayesian optimization to **optimize an analytical dummy function** using zeolite synthesis parameters, showing a significant improvement compared to grid search.
+2. The second section uses Bayesian optimization to **propose a new experiment** based on existing literature data.
+
+A common overarching objective in zeolite synthesis is to achieve a **high yield** of the desired zeolite product. In the papers under consideration, a more specialized goal involves **maximizing the presence of proximal Al pairs** within synthesized CHA zeolites, which is required for stabilizing Fe2+ sites (so-called divalent cation capacity, DCC). Upon activation, these sites can selectively oxidize methane to **methanol**. Accordingly, we will provide examples with synthesis yield, DCC and methanol yield as optimization objectives for the Bayesian optimization process.
+
+The various examples touch upon different aspects of Bayesian optimization, including **continuous variables, categorical variables, mixed variable types, parameter constraints, as well as single and multiple objectives**.
+
+
+
+
+
+
diff --git a/demo_zeolite_synthesis_bo.ipynb b/demo_zeolite_synthesis_bo.ipynb
new file mode 100644
index 0000000..deb1fe9
--- /dev/null
+++ b/demo_zeolite_synthesis_bo.ipynb
@@ -0,0 +1,2181 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "33956ea9-21bf-41a8-9c4a-d24f89d1d52d",
+ "metadata": {},
+ "source": [
+ "# Code Demonstration: Bayesian Optimization for Zeolite Synthesis"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "72398f31-f1bc-459f-aca4-6358d7e1a0c8",
+ "metadata": {},
+ "source": [
+ "Author: Plessers Dieter \n",
+ "March 28th, 2024"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e8b0873e-52ee-41ad-8ea6-dda138b77892",
+ "metadata": {},
+ "source": [
+ "## Table of Contents \n",
+ "[Introduction](#Introduction) \n",
+ "[Import necessary packages](#Packages) \n",
+ "\n",
+ "[Part 1: Optimize an Analytical 'Dummy' Function with Zeolite Synthesis Parameters](#Part1)\n",
+ "* [1.1 Define Dummy Analytical Objective Functions](#Part1.1) \n",
+ "* [1.2 Define a Function that Optimizes the Dummy Function with Ax Package](#Part1.2)\n",
+ "* [1.3 Single-Objective with 4 Continuous Variables](#Part1.3) \n",
+ "* [1.4 Multi-Objective with 4 Continuous Variables](#Part1.4)\n",
+ "* [1.5 Multi-Objective with 4 Continuous Variables and 2 Categorical Variables](#Part1.5) \n",
+ "* [1.6 Multi-Objective with 4 Continuous Variables, 2 Categorical Variables and Parameter Constraints](#Part1.6)\n",
+ "* [1.7 Code Example without Using the General `optimize_experiment_ax` Function of Part 1.2](#Part1.7)\n",
+ " \n",
+ "[Part 2: Propose a New Experiment Based on a Batch of Prior (Literature) Experiments](#Part2)\n",
+ "* [2.1 Define a Function that Proposes the Next Best Experiment with Ax Package](#Part2.1) \n",
+ "* [2.2 Collect the Literature Input and Output Data](#Part2.2)\n",
+ "* [2.3 Single-Objective (Synthesis Yield) with 3 Continuous Variables and 2 Categorical Variables](#Part2.3) \n",
+ "* [2.4 Multi-Objective (Synthesis Yield + Methanol Production) with 3 Continuous Variables and 2 Categorical Variables](#Part2.4)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e3c2a160-5c0a-4e9c-bfcd-45491aef19b9",
+ "metadata": {},
+ "source": [
+ "## Introduction \n",
+ "#### Scope \n",
+ "This Jupyter notebook accompanies an introductory text on Bayesian optimization (BO) for zeolite synthesis. In the text, we outlined a **typical parameter space with tunable variables and constraints, along with possible objectives derived from a zeolite target application**. Within this notebook, we exemplify these concepts with code, leveraging real-world literature data acquired through grid search in [Table S4](https://pubs.acs.org/doi/suppl/10.1021/acs.chemmater.9b03738/suppl_file/cm9b03738_si_001.pdf#page=10) and [Table S3](https://pubs.acs.org/doi/suppl/10.1021/jacs.1c07590/suppl_file/ja1c07590_si_001.pdf#page=9) in the Supporting Information of respectively:\n",
+ "\n",
+ "1. Devos, J.; Bols, M. L.; Plessers, D.; Goethem, C. Van; Seo, J. W.; Hwang, S.-J.; Sels, B. F.; Dusselier, M. SynthesisâStructureâActivity Relations in Fe-CHA for CâH Activation: Control of Al Distribution by Interzeolite Conversion. [*Chem. Mater.* **2020**, *32*, 273â285](https://pubs.acs.org/doi/abs/10.1021/acs.chemmater.9b03738). \n",
+ "2. Bols, M. L.; Devos, J.; Rhoda, H. M.; Plessers, D.; Solomon, E. I.; Schoonheydt, R. A.; Sels, B. F.; Dusselier, M. Selective Formation of $\\alpha$-Fe(II) Sites on Fe-Zeolites through One-Pot Synthesis. [*J. Am. Chem. Soc.* **2021**, *143*, 16243â16255](https://pubs.acs.org/doi/10.1021/jacs.1c07590). \n",
+ "\n",
+ "A common overarching objective in zeolite synthesis is to achieve a **high yield** of the desired zeolite product. In the papers under consideration, a more specialized goal involves **maximizing the presence of proximal Al pairs** within synthesized CHA zeolites, which is required for stabilizing Fe2+ sites (so-called divalent cation capacity, DCC). Upon activation, these sites can selectively oxidize methane to **methanol**. Accordingly, we will provide examples with synthesis yield, DCC and methanol yield as optimization objectives for the Bayesian optimization process.\n",
+ "\n",
+ "This notebook is divided into two sections: \n",
+ "1. The first section uses Bayesian optimization to optimize an analytical dummy function using zeolite synthesis parameters. \n",
+ "2. The second section uses Bayesian optimization to propose a new experiment based on existing literature data.\n",
+ "\n",
+ "#### [Part 1: Optimize an Analytical 'Dummy' Function with Zeolite Synthesis Parameters](#Part1)\n",
+ "We create a ['dummy' function](#Part1.1) linking the input parameters of zeolite synthesis to a specific output. The Bayesian optimizer operates without knowledge of the underlying analytical function we have defined, but this approach allows us to run a Bayesian optimization scheme as follows: 1) BO suggests parameters for experimentation based on a list of previous experimental outcomes, 2) our analytical function yields the outcome of the suggested experiment, 3) the outcome is added to the list of previous experimental outcomes and 4) this process is iteratively repeated. \n",
+ "\n",
+ "We run three different experiment modes to demonstrate various options available with BO:\n",
+ "\n",
+ "1. Four **continuous** variables with a **single** objective (high synthesis yield) [[link]](#Part1.3) \n",
+ "2. Four **continuous** variables with **multiple** objectives (high synthesis yield and high DCC) [[link]](#Part1.4) \n",
+ "3. **Mixed variable types** (4 continuous and 2 categorical) with **multiple** objectives (high synthesis yield and high DCC) [[link]](#Part1.5) \n",
+ "4. **Mixed variable types** (4 continuous and 2 categorical) with **parameter constraints** and with **multiple objectives** (high synthesis yield and high DCC) [[link]](#Part1.6) \n",
+ "\n",
+ "In the simplest case of 4 continuous variables, a grid search choosing 3 values per variable would require $3^4=81$ experiments, with no guarantee that any of these experiments will come close to the global optimum. In contrast, it can be seen in the code that the Bayesian optimizer reaches close to the global optimum in less than 30 iterations. \n",
+ " \n",
+ "#### [Part 2: Propose a New Experiment Based on a Batch of Prior (Literature) Experiments](#Part2)\n",
+ "Here Bayesian optimization is applied within a **real-world context** where the underlying target function remains unknown. We start from a batch of experiments collected from literature, and we allow the Bayesian optimizer to **suggest the next experiment**. This process aims to explore the search space further and/or leverage past results. In practice, one would implement the suggested experiment, incorporate its outcome into the existing results, and subsequently rerun the Bayesian optimization algorithm. We provide an example with **mixed variable types** (3 continuous and 2 categorical) with a **single-objective** and **multi-objective** target.\n",
+ "\r\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "293ba810-16e0-4db1-8bd8-ed91705d0fe5",
+ "metadata": {},
+ "source": [
+ "## Import necessary packages "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "id": "bec011d0-be00-428b-b4e3-ace5c2940f3e",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "\n",
+ "# Ax (adaptive experimentation) is the package we use for BO: https://ax.dev/\n",
+ "try:\n",
+ " import ax\n",
+ "except: \n",
+ " %pip install ax-platform\n",
+ "from ax.service.ax_client import AxClient, ObjectiveProperties\n",
+ "from ax.modelbridge.generation_strategy import GenerationStrategy, GenerationStep\n",
+ "from ax.modelbridge.registry import Models\n",
+ "from ax.service.utils.instantiation import ObjectiveProperties\n",
+ "\n",
+ "# We will use plotly to plot the optimization trace of a single-objective experiment\n",
+ "try: \n",
+ " import plotly\n",
+ "except:\n",
+ " %pip install plotly\n",
+ "import plotly.graph_objects as go\n",
+ "\n",
+ "from IPython.display import display # Enables the display of more than one dataframe per code cell\n",
+ "import pandas as pd\n",
+ "pd.set_option('max_colwidth', None) # Show full width Panda columns\n",
+ "from collections import OrderedDict\n",
+ "\n",
+ "# To hide all the info and warning messages of Ax. Comment the next lines if you want to see them.\n",
+ "import logging, sys\n",
+ "logging.disable(sys.maxsize)\n",
+ "import warnings\n",
+ "warnings.filterwarnings('ignore')\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4f326992-3de6-443f-875b-d72e35aaab7b",
+ "metadata": {},
+ "source": [
+ "## Part 1: Optimize an Analytical 'Dummy' Function with Zeolite Synthesis Parameters "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6c7517af-a5d3-4d5c-b7e9-05af59588548",
+ "metadata": {},
+ "source": [
+ "### 1.1 Define Dummy Analytical Objective Functions "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2b43f842-8109-499d-a064-048b377cb9b3",
+ "metadata": {},
+ "source": [
+ "Here we will define analytical functions for BO to **minimize**:\n",
+ "1) The first function has 4 continuous variables with a single objective\n",
+ "2) The second function has 4 continuous variables with two objectives\n",
+ "3) The third fuctions has 4 continuous and 2 categorical variables with 2 objectives\n",
+ "\n",
+ "We choose a simple function definition that has a **clear global minimum** (both for single and multi-objective) around \n",
+ "x1 = 0.35 $\\ \\ \\ $ (molar ratio of organic structure directing agents (OSDA) compared to Si) \n",
+ "x2 = 12.5 $\\ \\ \\ $ (the molar ratio of water compared to Si) \n",
+ "x3 = 140 $\\ \\ \\ \\ $ (synthesis temperature (°C)) \n",
+ "x4 = 24 $\\ \\ \\ \\ \\ \\ $ (synthesis time (h)) \n",
+ "x5 = Static $\\ $ (stirring mode) \n",
+ "x6 = Fumed (silicon source)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "id": "fae9fcab-28a4-4245-8354-677b7a4034d1",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def dummy_function_4_continuous_1_objective(x1, x2, x3, x4, obj_name1):\n",
+ " y1 = float((90*x1- 90*0.35)**2 + (x2-12.5)**2 + (x3-140)**2 + (x4/8-24/8)**2 + np.cos(x1*x2*x3*x4))\n",
+ " return {obj_name1:y1}\n",
+ " \n",
+ "def dummy_function_4_continuous_2_objective(x1, x2, x3, x4, obj_name1, obj_name2):\n",
+ " y1 = float((90*x1- 90*0.35)**2 + (x2-12.5)**2 + (x3-140)**2 + (x4/8-24/8)**2 + np.cos(x1*x2*x3*x4))\n",
+ " y2 = float((90*x1- 90*0.35)**2 + (x2-12.5)**2 + (x3-140)**2 + (x4/8-24/8)**2 + np.sin(x1*x2*x3*x4))\n",
+ "\n",
+ " # In our case, standard error is 0.0, since we are computing a synthetic function.\n",
+ " # See: https://ax.dev/tutorials/multiobjective_optimization.html\n",
+ " return {obj_name1:(y1, 0.0), obj_name2 : (y2, 0.0)}\n",
+ "\n",
+ "def dummy_function_4_continuous_2_categorical_2_objective(x1, x2, x3, x4, c1, c2, obj_name1, obj_name2):\n",
+ "\n",
+ " # The penalty lookup idea comes from https://honegumi.readthedocs.io/en/latest/\n",
+ " penalty_lookup1 = {\"Stirring\": 5000, \"Tumbling\": 2500, \"Static\": 0.0}\n",
+ " penalty_lookup2 = {\"Colloidal\": 5000, \"Fumed\": 0.0}\n",
+ " \n",
+ " y1 = float((90*x1- 90*0.35)**2 + (x2-12.5)**2 + (x3-140)**2 + (x4/8-24/8)**2 + np.cos(x1*x2*x3*x4))\n",
+ " y1 += penalty_lookup1[c1] + penalty_lookup2[c2]\n",
+ " \n",
+ " y2 = float((90*x1- 90*0.35)**2 + (x2-12.5)**2 + (x3-140)**2 + (x4/8-24/8)**2 + np.sin(x1*x2*x3*x4))\n",
+ " y2 += penalty_lookup1[c1] + penalty_lookup2[c2]\n",
+ " \n",
+ " # In our case, standard error is 0.0, since we are computing a synthetic function.\n",
+ " # See: https://ax.dev/tutorials/multiobjective_optimization.html\n",
+ " return {obj_name1:(y1, 0.0), obj_name2 : (y2, 0.0)}\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9b4cc353-4c83-42f8-9bfd-bf1d926e97e0",
+ "metadata": {},
+ "source": [
+ "### 1.2 Define a Function that Optimizes the Dummy Function with Ax Package "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "fdb712df-d17b-4662-a671-ddf0b3335c2c",
+ "metadata": {},
+ "source": [
+ "Here we will create a **general function** `optimize_experiment_ax` to run a BO experiment with the Ax package. This general function allows focus on the output of the experiments further on, without repeatedly modifying similar sections of code. At the end, we offer an example code not using this general function, which may be employed if only one type of experiment is considered. \n",
+ "\n",
+ "The `optimize_experiment_ax` function takes a list of **objective names** (can be any length, so single or multi-objective), an **analytical function** of the objectives (defined in [section 1.1](#Part1.1)), a list of **continuous parameters**, a list of **categorical parameters**, a list of parameter **constraints**, and an amount of **iterations**. This function will be used throughout part 1 of the notebook to run different experiments. \n",
+ "\n",
+ "\n",
+ "This function is inspired by the different options provided by **Honegumi** (https://honegumi.readthedocs.io/en/latest/), and can easily be modified to incorporate batch sizes. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "id": "5bcfb4b9-c580-4061-9b95-34a66f1667c6",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def to_plotly(axplotconfig):\n",
+ " \"\"\"Converts AxPlotConfig to plotly Figure.\"\"\"\n",
+ " data = axplotconfig[0][\"data\"]\n",
+ " layout = axplotconfig[0][\"layout\"]\n",
+ " fig = go.Figure({\"data\": data, \"layout\": layout})\n",
+ " return fig\n",
+ " \n",
+ "def optimize_experiment_ax(list_obj_names, objective_function, continuous_parameters_list, categorical_parameters_list, \n",
+ " parameter_constraints_lists, iterations):\n",
+ " \"\"\"\n",
+ " Creates an optimization experiment with the Ax package. \n",
+ " Inputs: \n",
+ " list_obj_names: list with all the objective names\n",
+ " objective_function: function used to calculate objectives\n",
+ " continuous_parameters_list: list of continuous parameters for the objective function\n",
+ " categorical_parameters_list: list of categorical parameters for the objective function\n",
+ " parameter_constraints_lists: list of constraints in the parameter space\n",
+ " iterations: number of iterations for the BO algorithm\n",
+ "\n",
+ " Outputs:\n",
+ " The best parameters and their objective function outputs are printed.\n",
+ " For single-objective experiments, the optimization trace is plotted. \n",
+ "\n",
+ " Source:\n",
+ " This function is inspired by the different options provided by Honegumi (https://honegumi.readthedocs.io/en/latest/)\n",
+ " \"\"\"\n",
+ "\n",
+ " all_parameters_list = continuous_parameters_list + categorical_parameters_list\n",
+ " objectives_dictionary={}\n",
+ " for obj_name in list_obj_names:\n",
+ " objectives_dictionary[obj_name]=ObjectiveProperties(minimize=True)\n",
+ " \n",
+ " ax_client = AxClient(verbose_logging=False)\n",
+ " ax_client.create_experiment(parameters= all_parameters_list, \n",
+ " parameter_constraints = parameter_constraints_lists,\n",
+ " objectives = objectives_dictionary \n",
+ " )\n",
+ " \n",
+ " for _ in range(iterations):\n",
+ " parameterization, trial_index = ax_client.get_next_trial()\n",
+ " \n",
+ " # Extract parameters and collect in dictionary\n",
+ " dict_extract_parameters={}\n",
+ " for parameter_info in all_parameters_list:\n",
+ " parameter_name = parameter_info[\"name\"]\n",
+ " dict_extract_parameters[parameter_name] = parameterization[parameter_name]\n",
+ " \n",
+ " # Calculate the results by plugging the parameters and objective names in the provided objective function\n",
+ " results = objective_function(*dict_extract_parameters.values(), *list_obj_names)\n",
+ " ax_client.complete_trial(trial_index=trial_index, raw_data=results)\n",
+ "\n",
+ " # Create dataframe with details of all experiment iterations\n",
+ " df_experiments = ax_client.generation_strategy.trials_as_df\n",
+ " number_of_experiments_to_show= min(iterations, 5)\n",
+ " print(f'The details of the last {number_of_experiments_to_show} experiments:')\n",
+ " display(df_experiments.tail())\n",
+ "\n",
+ " if len(list_obj_names) == 1:\n",
+ " # Single-objective optimization\n",
+ " # Get the AxClient's optimization trace \n",
+ " optimization_trace = ax_client.get_optimization_trace()\n",
+ " \n",
+ " # Convert the optimization trace to a Plotly figure and plot it\n",
+ " fig = to_plotly(optimization_trace)\n",
+ " fig.show(renderer='png')\n",
+ "\n",
+ " # Output the best parameters and their corresponding objective values\n",
+ " best_parameters, value = ax_client.get_best_parameters()\n",
+ " print('The best parameters were:')\n",
+ " display(best_parameters)\n",
+ " print('\\nThe objective value for the best parameters was:')\n",
+ " display(value[0])\n",
+ "\n",
+ " else:\n",
+ " # Multi-objective optimization\n",
+ " # Output the Pareto optimal parameters and their corresponding objective values\n",
+ " best_parameters = ax_client.get_pareto_optimal_parameters()\n",
+ " print('The Pareto optimal parameters were:')\n",
+ " display(list(best_parameters.values())[0][0])\n",
+ " print('\\nThe objective values for the Pareto optimal parameters were:')\n",
+ " display(list(best_parameters.values())[0][1][0])\n",
+ " \n",
+ " return None"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e2739dec-a5c4-4a32-bdbb-de0c014cf4dc",
+ "metadata": {},
+ "source": [
+ "### 1.3 Single-Objective with 4 Continuous Variables "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2f9df5b9-9b2a-4785-a347-061da3d110fd",
+ "metadata": {},
+ "source": [
+ "The first demonstration uses the general function [`optimize_experiment_ax`](#Part1.2) to optimize an experiment with **4 continuous variables** with typical ranges that are based on [Table S4](https://pubs.acs.org/doi/suppl/10.1021/acs.chemmater.9b03738/suppl_file/cm9b03738_si_001.pdf#page=10) in the Supporting Information of [reference 1](https://pubs.acs.org/doi/abs/10.1021/acs.chemmater.9b03738): \n",
+ "* the molar ratio of organic structure directing agents (OSDA) compared to Si \n",
+ "* the molar ratio of water compared to Si\n",
+ "* synthesis temperature (°C)\n",
+ "* synthesis time (h)\n",
+ "\n",
+ "The **single target variable** is synthesis yield. We assume that minimizing the above defined [dummy function](#Part1.1) `dummy_function_4_continuous_1_objective` corresponds to maximizing the yield of the zeolite synthesis. We also plot the **optimization trace** of the BO algorithm throughout the different experiment iterations.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "id": "c1825bf3-f5be-4504-834b-f18551cecd63",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "The details of the last 5 experiments:\n"
+ ]
+ },
+ {
+ "data": {
+ "text/html": [
+ "| \n", + " | Generation Step | \n", + "Generation Model | \n", + "Trial Index | \n", + "Trial Status | \n", + "Arm Parameterizations | \n", + "
|---|---|---|---|---|---|
| 25 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "25 | \n", + "COMPLETED | \n", + "{'25_0': {'OSDA_molar_ratio': 0.38, 'H2O_molar_ratio': 13.61, 'Synthesis_temperature': 136.78, 'Synthesis_time': 28.85}} | \n", + "
| 26 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "26 | \n", + "COMPLETED | \n", + "{'26_0': {'OSDA_molar_ratio': 0.38, 'H2O_molar_ratio': 20.35, 'Synthesis_temperature': 143.58, 'Synthesis_time': 2.0}} | \n", + "
| 27 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "27 | \n", + "COMPLETED | \n", + "{'27_0': {'OSDA_molar_ratio': 0.36, 'H2O_molar_ratio': 11.54, 'Synthesis_temperature': 142.55, 'Synthesis_time': 37.52}} | \n", + "
| 28 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "28 | \n", + "COMPLETED | \n", + "{'28_0': {'OSDA_molar_ratio': 0.38, 'H2O_molar_ratio': 5.0, 'Synthesis_temperature': 138.76, 'Synthesis_time': 2.0}} | \n", + "
| 29 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "29 | \n", + "COMPLETED | \n", + "{'29_0': {'OSDA_molar_ratio': 0.35, 'H2O_molar_ratio': 12.39, 'Synthesis_temperature': 138.91, 'Synthesis_time': 25.45}} | \n", + "
| \n", + " | Generation Step | \n", + "Generation Model | \n", + "Trial Index | \n", + "Trial Status | \n", + "Arm Parameterizations | \n", + "
|---|---|---|---|---|---|
| 25 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "25 | \n", + "COMPLETED | \n", + "{'25_0': {'OSDA_molar_ratio': 0.35, 'H2O_molar_ratio': 12.76, 'Synthesis_temperature': 140.14, 'Synthesis_time': 23.7}} | \n", + "
| 26 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "26 | \n", + "COMPLETED | \n", + "{'26_0': {'OSDA_molar_ratio': 0.38, 'H2O_molar_ratio': 11.41, 'Synthesis_temperature': 140.21, 'Synthesis_time': 23.86}} | \n", + "
| 27 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "27 | \n", + "COMPLETED | \n", + "{'27_0': {'OSDA_molar_ratio': 0.36, 'H2O_molar_ratio': 13.2, 'Synthesis_temperature': 140.56, 'Synthesis_time': 23.78}} | \n", + "
| 28 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "28 | \n", + "COMPLETED | \n", + "{'28_0': {'OSDA_molar_ratio': 0.33, 'H2O_molar_ratio': 8.25, 'Synthesis_temperature': 143.34, 'Synthesis_time': 2.0}} | \n", + "
| 29 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "29 | \n", + "COMPLETED | \n", + "{'29_0': {'OSDA_molar_ratio': 0.35, 'H2O_molar_ratio': 12.67, 'Synthesis_temperature': 139.9, 'Synthesis_time': 30.48}} | \n", + "
| \n", + " | Generation Step | \n", + "Generation Model | \n", + "Trial Index | \n", + "Trial Status | \n", + "Arm Parameterizations | \n", + "
|---|---|---|---|---|---|
| 25 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "25 | \n", + "COMPLETED | \n", + "{'25_0': {'OSDA_molar_ratio': 0.35, 'H2O_molar_ratio': 12.04, 'Synthesis_temperature': 139.7, 'Synthesis_time': 27.79, 'Stirring': 'Static', 'Silicon_source': 'Fumed'}} | \n", + "
| 26 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "26 | \n", + "COMPLETED | \n", + "{'26_0': {'OSDA_molar_ratio': 0.34, 'H2O_molar_ratio': 11.67, 'Synthesis_temperature': 140.26, 'Synthesis_time': 25.62, 'Stirring': 'Static', 'Silicon_source': 'Fumed'}} | \n", + "
| 27 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "27 | \n", + "COMPLETED | \n", + "{'27_0': {'OSDA_molar_ratio': 0.35, 'H2O_molar_ratio': 12.09, 'Synthesis_temperature': 140.02, 'Synthesis_time': 29.19, 'Stirring': 'Static', 'Silicon_source': 'Fumed'}} | \n", + "
| 28 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "28 | \n", + "COMPLETED | \n", + "{'28_0': {'OSDA_molar_ratio': 0.34, 'H2O_molar_ratio': 12.8, 'Synthesis_temperature': 140.5, 'Synthesis_time': 26.23, 'Stirring': 'Static', 'Silicon_source': 'Fumed'}} | \n", + "
| 29 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "29 | \n", + "COMPLETED | \n", + "{'29_0': {'OSDA_molar_ratio': 0.34, 'H2O_molar_ratio': 12.48, 'Synthesis_temperature': 140.3, 'Synthesis_time': 27.02, 'Stirring': 'Static', 'Silicon_source': 'Fumed'}} | \n", + "
| \n", + " | Generation Step | \n", + "Generation Model | \n", + "Trial Index | \n", + "Trial Status | \n", + "Arm Parameterizations | \n", + "
|---|---|---|---|---|---|
| 25 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "25 | \n", + "COMPLETED | \n", + "{'25_0': {'OSDA_molar_ratio': 0.35, 'H2O_molar_ratio': 8.65, 'Synthesis_temperature': 136.35, 'Synthesis_time': 22.99, 'Stirring': 'Static', 'Silicon_source': 'Fumed'}} | \n", + "
| 26 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "26 | \n", + "COMPLETED | \n", + "{'26_0': {'OSDA_molar_ratio': 0.34, 'H2O_molar_ratio': 8.55, 'Synthesis_temperature': 136.45, 'Synthesis_time': 23.4, 'Stirring': 'Static', 'Silicon_source': 'Fumed'}} | \n", + "
| 27 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "27 | \n", + "COMPLETED | \n", + "{'27_0': {'OSDA_molar_ratio': 0.34, 'H2O_molar_ratio': 8.08, 'Synthesis_temperature': 136.92, 'Synthesis_time': 20.49, 'Stirring': 'Static', 'Silicon_source': 'Fumed'}} | \n", + "
| 28 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "28 | \n", + "COMPLETED | \n", + "{'28_0': {'OSDA_molar_ratio': 0.34, 'H2O_molar_ratio': 8.05, 'Synthesis_temperature': 136.95, 'Synthesis_time': 18.65, 'Stirring': 'Static', 'Silicon_source': 'Fumed'}} | \n", + "
| 29 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "29 | \n", + "COMPLETED | \n", + "{'29_0': {'OSDA_molar_ratio': 0.34, 'H2O_molar_ratio': 8.49, 'Synthesis_temperature': 136.51, 'Synthesis_time': 23.21, 'Stirring': 'Static', 'Silicon_source': 'Fumed'}} | \n", + "
| \n", + " | Generation Step | \n", + "Generation Model | \n", + "Trial Index | \n", + "Trial Status | \n", + "Arm Parameterizations | \n", + "
|---|---|---|---|---|---|
| 25 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "25 | \n", + "COMPLETED | \n", + "{'25_0': {'OSDA_molar_ratio': 0.36, 'H2O_molar_ratio': 12.41, 'Synthesis_temperature': 140.96, 'Synthesis_time': 21.67}} | \n", + "
| 26 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "26 | \n", + "COMPLETED | \n", + "{'26_0': {'OSDA_molar_ratio': 0.37, 'H2O_molar_ratio': 11.45, 'Synthesis_temperature': 141.27, 'Synthesis_time': 2.0}} | \n", + "
| 27 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "27 | \n", + "COMPLETED | \n", + "{'27_0': {'OSDA_molar_ratio': 0.34, 'H2O_molar_ratio': 12.26, 'Synthesis_temperature': 139.76, 'Synthesis_time': 21.9}} | \n", + "
| 28 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "28 | \n", + "COMPLETED | \n", + "{'28_0': {'OSDA_molar_ratio': 0.1, 'H2O_molar_ratio': 5.0, 'Synthesis_temperature': 200.0, 'Synthesis_time': 2.0}} | \n", + "
| 29 | \n", + "GenerationStep_1 | \n", + "BoTorch | \n", + "29 | \n", + "COMPLETED | \n", + "{'29_0': {'OSDA_molar_ratio': 0.13, 'H2O_molar_ratio': 5.0, 'Synthesis_temperature': 132.33, 'Synthesis_time': 234.7}} | \n", + "
| \n", + " | OSDA_molar_ratio | \n", + "H2O_molar_ratio | \n", + "Fe_molar_ratio | \n", + "Fe_Salt | \n", + "Stirring | \n", + "Synthesis_yield | \n", + "Methanol_yield | \n", + "
|---|---|---|---|---|---|---|---|
| 0 | \n", + "0.36 | \n", + "15.4 | \n", + "0.019 | \n", + "A | \n", + "Static | \n", + "0.83 | \n", + "0.089 | \n", + "
| 1 | \n", + "0.33 | \n", + "14.1 | \n", + "0.016 | \n", + "A | \n", + "Static | \n", + "0.85 | \n", + "0.085 | \n", + "
| 2 | \n", + "0.36 | \n", + "15.5 | \n", + "0.018 | \n", + "N | \n", + "Static | \n", + "0.96 | \n", + "0.084 | \n", + "
| 3 | \n", + "0.36 | \n", + "15.5 | \n", + "0.017 | \n", + "N | \n", + "Static | \n", + "0.84 | \n", + "0.108 | \n", + "
| 4 | \n", + "0.36 | \n", + "15.5 | \n", + "0.035 | \n", + "A | \n", + "Static | \n", + "0.93 | \n", + "0.116 | \n", + "
| 5 | \n", + "0.36 | \n", + "15.5 | \n", + "0.037 | \n", + "N | \n", + "Stirring | \n", + "1.00 | \n", + "0.063 | \n", + "
| 6 | \n", + "0.34 | \n", + "14.4 | \n", + "0.039 | \n", + "N | \n", + "Stirring | \n", + "0.91 | \n", + "0.108 | \n", + "
| 7 | \n", + "0.36 | \n", + "15.5 | \n", + "0.055 | \n", + "A | \n", + "Static | \n", + "0.00 | \n", + "0.053 | \n", + "
| 8 | \n", + "0.36 | \n", + "15.6 | \n", + "0.055 | \n", + "N | \n", + "Static | \n", + "0.00 | \n", + "0.013 | \n", + "
| 9 | \n", + "0.36 | \n", + "15.5 | \n", + "0.090 | \n", + "A | \n", + "Static | \n", + "0.00 | \n", + "0.020 | \n", + "
| 10 | \n", + "0.33 | \n", + "14.1 | \n", + "0.016 | \n", + "A | \n", + "Stirring | \n", + "0.91 | \n", + "0.062 | \n", + "
| \n", + " | OSDA_molar_ratio | \n", + "H2O_molar_ratio | \n", + "Fe_molar_ratio | \n", + "Fe_Salt | \n", + "Stirring | \n", + "Synthesis_yield | \n", + "
|---|---|---|---|---|---|---|
| 0 | \n", + "0.36 | \n", + "15.4 | \n", + "0.019 | \n", + "A | \n", + "Static | \n", + "0.83 | \n", + "
| 1 | \n", + "0.33 | \n", + "14.1 | \n", + "0.016 | \n", + "A | \n", + "Static | \n", + "0.85 | \n", + "
| 2 | \n", + "0.36 | \n", + "15.5 | \n", + "0.018 | \n", + "N | \n", + "Static | \n", + "0.96 | \n", + "
| 3 | \n", + "0.36 | \n", + "15.5 | \n", + "0.017 | \n", + "N | \n", + "Static | \n", + "0.84 | \n", + "
| 4 | \n", + "0.36 | \n", + "15.5 | \n", + "0.035 | \n", + "A | \n", + "Static | \n", + "0.93 | \n", + "
| 5 | \n", + "0.36 | \n", + "15.5 | \n", + "0.037 | \n", + "N | \n", + "Stirring | \n", + "1.00 | \n", + "
| 6 | \n", + "0.34 | \n", + "14.4 | \n", + "0.039 | \n", + "N | \n", + "Stirring | \n", + "0.91 | \n", + "
| 7 | \n", + "0.36 | \n", + "15.5 | \n", + "0.055 | \n", + "A | \n", + "Static | \n", + "0.00 | \n", + "
| 8 | \n", + "0.36 | \n", + "15.6 | \n", + "0.055 | \n", + "N | \n", + "Static | \n", + "0.00 | \n", + "
| 9 | \n", + "0.36 | \n", + "15.5 | \n", + "0.090 | \n", + "A | \n", + "Static | \n", + "0.00 | \n", + "
| 10 | \n", + "0.33 | \n", + "14.1 | \n", + "0.016 | \n", + "A | \n", + "Stirring | \n", + "0.91 | \n", + "
| \n", + " | OSDA_molar_ratio | \n", + "H2O_molar_ratio | \n", + "Fe_molar_ratio | \n", + "Fe_Salt | \n", + "Stirring | \n", + "Synthesis_yield | \n", + "Methanol_yield | \n", + "
|---|---|---|---|---|---|---|---|
| 0 | \n", + "0.36 | \n", + "15.4 | \n", + "0.019 | \n", + "A | \n", + "Static | \n", + "0.83 | \n", + "0.089 | \n", + "
| 1 | \n", + "0.33 | \n", + "14.1 | \n", + "0.016 | \n", + "A | \n", + "Static | \n", + "0.85 | \n", + "0.085 | \n", + "
| 2 | \n", + "0.36 | \n", + "15.5 | \n", + "0.018 | \n", + "N | \n", + "Static | \n", + "0.96 | \n", + "0.084 | \n", + "
| 3 | \n", + "0.36 | \n", + "15.5 | \n", + "0.017 | \n", + "N | \n", + "Static | \n", + "0.84 | \n", + "0.108 | \n", + "
| 4 | \n", + "0.36 | \n", + "15.5 | \n", + "0.035 | \n", + "A | \n", + "Static | \n", + "0.93 | \n", + "0.116 | \n", + "
| 5 | \n", + "0.36 | \n", + "15.5 | \n", + "0.037 | \n", + "N | \n", + "Stirring | \n", + "1.00 | \n", + "0.063 | \n", + "
| 6 | \n", + "0.34 | \n", + "14.4 | \n", + "0.039 | \n", + "N | \n", + "Stirring | \n", + "0.91 | \n", + "0.108 | \n", + "
| 7 | \n", + "0.36 | \n", + "15.5 | \n", + "0.055 | \n", + "A | \n", + "Static | \n", + "0.00 | \n", + "0.053 | \n", + "
| 8 | \n", + "0.36 | \n", + "15.6 | \n", + "0.055 | \n", + "N | \n", + "Static | \n", + "0.00 | \n", + "0.013 | \n", + "
| 9 | \n", + "0.36 | \n", + "15.5 | \n", + "0.090 | \n", + "A | \n", + "Static | \n", + "0.00 | \n", + "0.020 | \n", + "
| 10 | \n", + "0.33 | \n", + "14.1 | \n", + "0.016 | \n", + "A | \n", + "Stirring | \n", + "0.91 | \n", + "0.062 | \n", + "
+
+**Figure 1.** Framework structures of the most relevant zeolite topologies. Vertices represent T atoms (usually Si and Al) connected via O-atoms (not depicted)*.* Dashed lines demarcate the unit cells. All figures are sourced from the online IZA Database.[^5]
+
+Pore and channel dimensions, typically in the Ängström scale, are characterized by the number *n* of T-atoms in the limiting ring, denoted as *n*MR (*n*\-membered ring) (Figure 2). Ring sizes vary from 3MR to 24MR, and exhibit various shapes, from circular to elliptical.[^5][^6] This ring and channel system enables zeolites to function as molecular sieves, allowing only molecules smaller than the free diameter to traverse the rings, while larger molecules are excluded.[^1][^2] Zeolites are also known for their shape selectivity, wherein the distribution of products in a reaction system is influenced by how reactants, intermediates, and products fit within the cage and channel structure of the zeolites.[^7] Moreover, atoms beyond the first coordination sphere can impact the reactivity of active sites by guiding substrate approach and/or by (de)stabilizing the active site or transition state. These effects, known as 'second-sphere effects' play a crucial role in zeolite reactivity.[^8][^9]
+
+
+
+**Figure 2.** Different ring sizes in zeolites.[^9][^10]
+
+A specific zeolite topology is defined by the spatial arrangement of its lattice, regardless of the materialâs chemical composition. Consequently, even within materials sharing the same zeolite topology, numerous parameters remain adjustable. Among these, the most prevalent is the amount and distribution of aluminum (Al) atoms within the zeolite lattice. Each \[AlO4\]\- tetrahedron introduces one negative lattice charge, which is counterbalanced by exchangeable, extra-framework cations. These cations can be protons, imparting zeolites with their distinctive acidic properties,[^11] or metal cations such as transition metal ions (TMIs), employed in redox reactions.[^9] They can also be combined in bifunctional catalysis.[^12]
+
+Thanks to their thermal stability, low cost (particularly for commercially available bulk zeolites like depicted in Figure 1) and unique chemical properties (acidity, cation exchange capacity, molecular sieving capabilitiesâŠ), zeolites find extensive use across various industrial sectors. They are integral components in detergents (such as washing powders), they serve as adsorbents and desiccants,[^13] and they play critical roles in petrochemistry, notably as acid catalysts in the fluid catalytic cracking (FCC) process.[^14] Beyond these applications, zeolites demonstrate remarkable versatility, being employed in sensors,[^15] membranes[^16] and redox processes, such as the removal of nitrogen-containing exhaust gases where Cu-exchanged CHA zeolites are at the forefront of selective catalytic reduction (SCR) of NOx gasses in vehicles.[^17][^18] The vast applicability of zeolites is evidenced by their annual production of approximately 6 million tons, with a global market valued at around 13 billion USD.[^19]
+
+In addition to their extensive industrial applications, the favorable properties of zeolites guarantee their continued significance in fundamental research, both within academia and industry. Zeolites persist as one of the most extensively studied materials for various innovative applications, with numerous laboratories worldwide striving to optimize the synthesis process, customize the resulting properties of zeolites, and discover new topologies.[^20]
+
+## 2. Zeolite Synthesis â Ingredients and Equipment
+
+The zeolite synthesis research field boasts a rich history,[^21] reflected in the abundance of literature detailing various synthesis recipes for each zeolite topology (and different materials within the topology). These recipes span from straightforward methods with minimal ingredients, resilient to experimental errors, to intricate procedures demanding specialized equipment.[^22] Typically, hydrothermal synthesis methods are employed, involving the mixing of precursors in water followed by crystallization under elevated temperature and pressure conditions.
+
+A âtypicalâ hydrothermal zeolite synthesis procedure involves combining a silicon source, an aluminum source, an organic structure-directing agent (OSDA) serving as a template to fill the zeolite pores during synthesis (afterwards removed via calcination), inorganic cations (typically alkali metal hydroxides) and water.[^22] These ingredients are mixed within an inert Teflon liner and placed inside a stainless steel autoclave (Figure 3A), which is quite literally a black box for which it is hard to understand, let alone predict, the internal processes.[^23] The synthesis occurs at autogenous pressure in an oven maintained at a certain temperature (typically 100-200°C) for a specific time (typically 3-7 days, though durations may vary). The synthesis can occur under stirring conditions using a magnetic stirring bar within the Teflon liner, by tumbling the autoclave in the oven, or under static conditions without agitation. An optional step prior to hydrothermal synthesis involves stirring the synthesis solution or gel for a period (often a day) at room temperature, known as âagingâ.
+
+However, the term âtypical zeolite synthesisâ is somewhat contradictory since there are numerous variations on this process. While the aforementioned ingredients are common, they are not strictly required. Additionally, many other components could be introduced, such as fluoride as a mineralizer, alternative metals (e.g. B, Fe, Zn) for lattice incorporation alongside or in place of aluminum, a secondary OSDA, or the application of ultrasound or microwaves during synthesis.[^22] Even applying a voltage with electrodes recently emerged as a novel possibility.[^24] However, most of these unconventional conditions are utilized for niche materials with limited industrial significance, often due to the exotic requirements of such conditions.
+
+
+
+**Figure 3.** A) Teflon liners and corresponding autoclaves.[^25] B) Multi-autoclave system for high-throughput parallel experiments with a robotic arm.[^26]
+
+This brief overview of zeolite synthesis highlights the vast parameter space involved and the ongoing challenge of fully understanding the crystallization mechanisms. Consequently, despite the significant industrial importance of zeolites, their synthesis predominantly relies on heuristic approaches, practitioner experience, and an element of serendipity. An experienced zeolite synthesis practitioner can narrow down this parameter space by drawing from literature and past experiences. For instance, they might opt for a fixed set of ingredients and consider a limited range of temperatures and concentrations. Within this confined search space, typically, either a grid search or a random search is conducted. This process can be arduous, especially considering the sensitivity of certain zeolite synthesis processes to small changes in conditions. Fortunately, in many laboratories, parallel experiments using small multi-autoclave synthesis systems accelerate this process (Figure 3B).[^26] Figure 3B also offers a glimpse into a future with fully autonomous systems, where robots handle synthesis solution preparation, oven transfer, and synthesis product characterization. When integrated with âintelligentâ experimental design techniques like active learning, such as Bayesian optimization, this advancement could pave the way for the development of self-driving laboratories, drastically enhancing the pace of discovery and optimization in zeolite synthesis.[^27]
+
+## 3. Zeolite Synthesis â Optimization
+
+As discussed in the previous section, active learning has the potential to navigate the extensive parameter space of black box zeolite syntheses. This would prove beneficial in both manual and self-driving laboratories, saving time, reducing associated costs, and increasing the likelihood of discovering materials that are close to optimal.
+
+Nonetheless, conducting a search on Science Direct and Google Scholar using the terms "zeolite Bayesian optimization" and "zeolite active learning" yields only a limited number of relevant examples, excluding cases where active learning is employed to optimize machine learning model hyperparameters:
+
+1. Bayesian optimization was employed to experimentally find optimal Si/Al and Cu loading for Cu-CHA zeolites to optimize catalytic activity and selectivity in the partial oxidation of methane to methanol. As the CHA zeolites were commercially purchased, the Si/Al ratio was limited to 4 possibilities (categorical), whereas the in-house Cu ion exchange allowed for a continuous range of Cu loading.[^28]
+2. Bayesian optimization was employed to experimentally determine the optimal metal loading of Cu/Fe-CHA catalysts, aiming to maximize their performance in both fresh and hydrothermally aged conditions for the selective catalytic reduction of NOx in diesel engine exhaust.[^29]
+3. Bayesian optimization was used to find mechanically superior zeolite structures (in term of shear and bulk moduli calculated with DFT and machine learning) in an existing database containing approximately 590,000 hypothetical zeolites.[^30]
+
+However, none of these examples directly relate to zeolite synthesis. The first example effectively illustrates how an optimized in-house synthesis could lead to further improvement of the catalyst. This would involve expanding the feasible range of Si/Al values and considering additional variables such as synthesis temperature and duration, which can impact Al distribution. Thus, it appears that the zeolite synthesis field with its associated large parameter space could benefit from leveraging active learning methods. Therefore, the optimization characteristics are explored below, building upon the earlier section detailing zeolite synthesis ingredients and equipment.
+
+### 3.1 Parameter Space
+
+Zeolite synthesis involves numerous inherently **continuous** variables, such as the concentrations of the aforementioned ingredients. Typically, syntheses are presented in literature as molar ratio *xi* of ingredient *i* with respect to silicon, e.g., 1 Si: 0.1 Al: 0.5 OSDA: 0.2 Na+: 30 H2O. Given the known synthesis volume in the teflon liner, these ratios determine the required ingredient masses. Consequently, in Bayesian optimization, it is sufficient to optimize the molar ratio values. Other continuous variables include synthesis temperature, duration, and stirring/tumbling speed.
+
+The parameter bounds encompass reasonable molar ratio values (ranging from a lower bound of 0 to upper bounds determined by the ingredient and target, such as *xAl* < 0.5), along with constraints linked to equipment specifications. These constraints may include the maximum temperature of the synthesis oven and the predefined maximum speeds for stirring and tumbling systems. Depending on the equipment at hand, certain continuous variables may need to be discretized. For instance, in a setup where only one tumbling speed is available, the tumbling variable would become categorical: {Yes, No}.
+
+On the other hand, the selection of precursor materials is **categorical**. Even though the difference between certain sources seems minimal, they could have a small difference in solubility or basicity and lead to different crystallization kinetics.[^31]
+
+Below, we list several standard choices for the typical hydrothermal synthesis outlined above:
+
+| **Precursor** | **Options** |
+| --- | --- |
+| | |
+| Silicon | Colloidal (Ludox HS-40, Ludox AS-40âŠ), fumed (aerosil, Cab-O-SilâŠ), solid (sodium silicates, other zeolites in an interzeolite conversionâŠ), liquids (tetraethyl orthosilicateâŠ) ⊠|
+| Aluminum | Aluminum metal, Al(OH)3, Al(NO3)3, Al2(SO4)3, NaAlO2⊠|
+| OSDA | Typically one to a few OSDAs are known to effectively template the pores of a specific zeolite topology. For example, the typical OSDA for CHA synthesis is N,N,N-trimethyladamantylammonium hydroxide, but alternatives like tetraethylammonium hydroxide have also been documented.[^32] However, most often, a fixed OSDA is chosen a priori, and other variables are adjusted, as syntheses with different OSDAs may require entirely different conditions. |
+| Inorganic cation | LiOH, NaOH, KOH, CsOH, NaCl⊠|
+| Water | Demineralized water is preferred for reproducibility, and further filtration (e.g., Milli-Q purification) is desirable. Therefore, no choices are to be made in this regard. |
+
+### 3.2 Constraints
+
+In zeolite synthesis, additional constraints on top of the specified bounds for the parameter space are uncommon. For instance, one might envision rare scenarios where both Al and a small amount of B are desired in the synthesis. Then a constraint could be introduced, e.g.: 10*xB* < *xAl*. Another scenario might involve limiting the total synthesis cost by assigning a monetary value to each parameter.
+
+### 3.3 Objectives
+
+Various objectives might be pursued depending on the economic considerations of the process and the specific application requirements:
+
+**General objectives:**
+
+- Attaining high crystallinity of the resulting zeolite.
+- Ensuring high purity by minimizing the presence of side-products.
+
+**Economic objectives:**
+
+- Achieving a high synthesis yield.
+- Maintaining low concentrations of expensive ingredients such as the organic structure-directing agent (OSDA), or other potentially harmful ingredients like fluoride.
+- Operating at low synthesis temperatures.
+- Reducing the synthesis time.
+
+**Application objectives:**
+
+- Targeting specific Si/Al ratios, which might be important for acid-catalyzed applications and also influences the zeolite stability.[^33]
+- Controlling the distribution of aluminum within the structure, important for ion exchange and catalysis.[^34]
+- Adjusting crystal size or morphology to optimize catalysis, adsorption or separation applications.[^35]
+
+Most of these objectives are correlated in positive or negative ways. For instance, low crystal size is generally accomplished through lower synthesis temperatures, favoring nucleation over crystal growth. However, this also results in slow nucleation and crystal growth and â depending on the synthesis time â low yields.[^36]
+
+### 3.4 Zeolite Synthesis Datasets
+
+Most zeolite synthesis papers include tables, either in the main text or supplementary information, that contain the molar ratios of the ingredients used in the âgrid searchâ synthesis experiments. The exact procedure and details of the ingredients are often described in textual form in the method sections. Following the recent advancements in material informatics, some studies have systematically scraped and summarized zeolite synthesis data. The data can be combined with structural parameters available on the IZA website for the respective topologies.[^5]
+
+Jensen et al. gathered data for 1,200 distinct synthetic pathways for Ge-containing zeolites[^37] and 5,663 synthesis routes specifying OSDAs.[^38] Muraoka et al. compiled a dataset with 686 unique OSDA-free synthesis routes for 23 distinct frameworks, detailing gel compositions and reaction conditions.[^39] However, these datasets are rather small and do not fully encapsulate all parameters essential for zeolite synthesis: synthesis composition, OSDA and reaction conditions.
+
+In a recent impressive development, Pan et al. introduced ZeoSyn, a comprehensive dataset containing 23,961 zeolite synthesis pathways for 233 distinct zeolite topologies, representing over 80% of synthesized frameworks to date, using 921 unique OSDAs.[^40] Each synthesis route details gel composition (a combination of 51 possible ingredients, including Si, Al, P, Na+, K+, OHâ, Fâ, Ge, Ti, B, OSDA, H2O and additional solvents), reaction conditions and the resulting zeolite structure (or absence thereof, e.g. formation of dense or amorphous phases). In some cases, resulting zeolite properties such as Si/Al ratio, crystal size, crystallinity and BET surface area are reported. Each synthesis pathway is cross-referenced with the corresponding article from which it was sourced, alongside its publication year. The dataset is available on GitHub.[^41]
+
+## 4. Discussion
+
+Despite the significant industrial importance of zeolites, their synthesis primarily relies on heuristic approaches, domain knowledge, and a degree of serendipity. An experienced researcher can narrow down the extensive zeolite synthesis parameter space by drawing from literature and past experiences. Typically, within this constrained search space, practitioners resort to either a grid search or a random search. However, this procedure can prove time-consuming and costly. Active learning techniques, such as Bayesian optimization, offer the potential to more efficiently navigate the parameter space of black box zeolite syntheses and increase the likelihood of discovering materials that are close to optimal.
+
+Zeolite synthesis involves numerous continuous and categorical variables (see the Zeolite Synthesis sections). Depending on the considered search space, **mixed variable type** Bayesian optimization becomes necessary.[^42][^43] Additionally, many syntheses are **multi-objective**;[^44] for example, researchers may seek high synthesis yield coupled with a small crystal size. As mentioned earlier, laboratories often conduct parallel experiments using small multi-autoclave synthesis systems. Such setups could benefit from **batch** Bayesian optimization, which involves evaluating multiple candidate solutions simultaneously in each iteration of the optimization process. This can potentially reduce the total number of iterations required to find the desired solution.[^45][^46]
+
+One advantage of the extensive history of zeolite synthesis is the abundance of recipes available for most interesting topologies. Furthermore, unlike many other fields, failed synthesis attempts are frequently documented in the literature.[^40] Optimization campaigns enriched by **knowledge transfer** from past campaigns could identify desired materials with fewer experiments.[^47]
+
+Nowadays, various packages, such as Ax[^48], BayBE[^49] and scikit-optimize[^50]âŠ, readily provide access to these different aspects of Bayesian optimization. In the [accompanying Jupyter notebook](./demo_zeolite_synthesis_bo.ipynb), we illustrate these features using real-world literature data.[^51][^52] One of the objectives of these studies was to maximize the amount of proximal Al pairs in synthesized CHA zeolites, which is required for stabilizing Fe2+ sites. Upon activation, these sites can selectively oxidize methane to methanol. The importance of these proximal Al pairs is underscored by the fact that in Cu-CHA zeolites, conversely, a low concentration of proximal Al pairs is preferred. In this context, these proximal Al pairs stabilize redox-inactive Cu2+ species that do not participate in the oxidation of methane to methanol.[^53][^54]
+
+Two approaches are followed in the notebook:
+1) Optimization of a simulated analytical target function based on synthesis parameters.
+2) Existing literature data is fed as prior experimental data to the Bayesian optimization algorithm, which then recommends subsequent experimental conditions to try out.
+
+This [code demonstration](./demo_zeolite_synthesis_bo.ipynb) showcases the effectiveness of Bayesian optimization in zeolite synthesis, hopefully encouraging the exploration of active learning approaches in similar endeavors.
+
+## References
+
+[^1]: Davis, M. E. Zeolites and Molecular Sieves: Not Just Ordinary Catalysts. [*Ind. Eng. Chem. Res.* **1991**, *30*, 1675â1683.](https://doi.org/10.1021/ie00056a001)
+
+[^2]: Flanigen, E. M. Chapter 2 - Zeolites and Molecular Sieves an Historical Perspective. In *Studies in Surface Science and Catalysis*; Van Bekkum, H., Flanigen, E. M., Jacobs, P. A., Jansen, J. C., Eds.; [Elsevier Science B.V., 2001; Vol. 137, pp 11â35.](https://doi.org/10.1016/S0167-2991(01)80243-3)
+
+[^3]: Dusselier, M.; Davis, M. E. Small-Pore Zeolites: Synthesis and Catalysis. [*Chem. Rev.* **2018**, *118*, 5265â5329.](https://doi.org/10.1021/acs.chemrev.7b00738)
+
+[^4]: Pophale, R.; Cheeseman, P. A.; Deem, M. W. A Database of New Zeolite-like Materials. [*Phys. Chem. Chem. Phys.* **2011**, *13*, 12407â12412.](https://doi.org/10.1039/C0CP02255A)
+
+[^5]: Structure Commission of the International Zeolite Association. Database of Zeolite Structures.