diff --git a/LLM/Mixtral/Mixtral_fine_tuning.ipynb b/LLM/Mixtral/Mixtral_fine_tuning.ipynb new file mode 100644 index 0000000..8bda720 --- /dev/null +++ b/LLM/Mixtral/Mixtral_fine_tuning.ipynb @@ -0,0 +1,3106 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "iEkm_Q9AGgTX" + }, + "source": [ + "# MIXTRAL 8x7B - Mixture of Experts\n", + "\n", + "This will not run on the free T4 GPU from Google Colab. You will need A100 to run this." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "AlVGJXwsyVQO" + }, + "source": [ + "### Install Required Packages" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "3IovaXt4ZJQ_" + }, + "outputs": [], + "source": [ + "!pip install -q -U bitsandbytes\n", + "!pip install -q -U git+https://github.com/huggingface/transformers.git\n", + "!pip install -q -U git+https://github.com/huggingface/peft.git\n", + "!pip install -q -U git+https://github.com/huggingface/accelerate.git\n", + "!pip install -q datasets scipy\n", + "!pip install -q trl\n", + "!pip install flash-attn --no-build-isolation" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4qjy9PYUaY3W" + }, + "source": [ + "### Loading the Base Model\n", + "\n", + "Load the model in `4bit`, with double quantization, with `bfloat16` as the compute dtype.\n", + "\n", + "In this case we are using the instruct-tuned model - instead of the base model. For fine-tuning a base model will need a lot more data!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load dataset for finetuning" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Lets Load the Dataset\n", + "\n", + "For this tutorial, we will fine-tune Mistral 7B Instruct for code generation.\n", + "\n", + "We will be using this [dataset](https://huggingface.co/datasets/TokenBender/code_instructions_122k_alpaca_style) which is curated by [TokenBender (e/xperiments)](https://twitter.com/4evaBehindSOTA) and is an excellent data source for fine-tuning models for code generation. It follows the alpaca style of instructions, which is an excellent starting point for this task. The dataset structure should resemble the following:\n", + "\n", + "```json\n", + "{\n", + " \"instruction\": \"Create a function to calculate the sum of a sequence of integers.\",\n", + " \"input\": \"[1, 2, 3, 4, 5]\",\n", + " \"output\": \"# Python code def sum_sequence(sequence): sum = 0 for num in sequence: sum += num return sum\"\n", + "}\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "model_id = \"mistralai/Mixtral-8x7B-v0.1\"" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "import torch\n", + "from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig\n", + "\n", + "nf4_config = BitsAndBytesConfig(\n", + " load_in_4bit=True,\n", + " bnb_4bit_quant_type=\"nf4\",\n", + " bnb_4bit_use_double_quant=True,\n", + " bnb_4bit_compute_dtype=torch.bfloat16\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "model = AutoModelForCausalLM.from_pretrained(\n", + " model_id,\n", + " device_map='auto',\n", + " quantization_config=nf4_config,\n", + " use_cache=False,\n", + " attn_implementation=\"flash_attention_2\"\n", + "\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "tokenizer = AutoTokenizer.from_pretrained(model_id)\n", + "\n", + "tokenizer.pad_token = tokenizer.eos_token\n", + "tokenizer.padding_side = \"right\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's example how well the model does at this task currently:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [], + "source": [ + "def generate_response(prompt, model):\n", + " encoded_input = tokenizer(prompt, return_tensors=\"pt\", add_special_tokens=True)\n", + " model_inputs = encoded_input.to('cuda')\n", + "\n", + " generated_ids = model.generate(**model_inputs,\n", + " max_new_tokens=512,\n", + " do_sample=True,\n", + " pad_token_id=tokenizer.eos_token_id)\n", + "\n", + " decoded_output = tokenizer.batch_decode(generated_ids)\n", + "\n", + " return decoded_output[0].replace(prompt, \"\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "prompt=\"\"\"[INST]Use the provided input to create an instruction that could have been used to generate the response with an LLM. \\nThere are more than 12,000 species of grass. The most common is Kentucky Bluegrass, because it grows quickly, easily, and is soft to the touch. Rygrass is shiny and bright green colored. Fescues are dark green and shiny. Bermuda grass is harder but can grow in drier soil.[\\INST]\"\"\"\n", + "\n", + "generate_response(prompt, model)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(model)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from datasets import load_dataset\n", + "\n", + "dataset = load_dataset(\"TokenBender/code_instructions_122k_alpaca_style\", split=\"train\")\n", + "dataset" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "df = dataset.to_pandas()\n", + "df.head(10)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Instruction Fintuning - Prepare the dataset under the format of \"prompt\" so the model can better understand :\n", + "1. the function generate_prompt : take the instruction and output and generate a prompt\n", + "2. shuffle the dataset\n", + "3. tokenizer the dataset" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Formatting the Dataset\n", + "\n", + "Now, let's format the dataset in the required [Mistral-7B-Instruct-v0.1 format](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1).\n", + "\n", + "> Many tutorials and blogs skip over this part, but I feel this is a really important step.\n", + "\n", + "We'll put each instruction and input pair between `[INST]` and `[/INST]` output after that, like this:\n", + "\n", + "```\n", + "[INST] What is your favorite condiment? [/INST]\n", + "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavor to whatever I'm cooking up in the kitchen!\n", + "```\n", + "\n", + "You can use the following code to process your dataset and create a JSONL file in the correct format:" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [], + "source": [ + "def generate_prompt(data_point):\n", + " \"\"\"Gen. input text based on a prompt, task instruction, (context info.), and answer\n", + "\n", + " :param data_point: dict: Data point\n", + " :return: dict: tokenzed prompt\n", + " \"\"\"\n", + " prefix_text = 'Below is an instruction that describes a task. Write a response that ' \\\n", + " 'appropriately completes the request.\\n\\n'\n", + " # Samples with additional context into.\n", + " if data_point['input']:\n", + " text = f\"\"\"[INST]{prefix_text} {data_point[\"instruction\"]} here are the inputs {data_point[\"input\"]} [/INST]{data_point[\"output\"]}\"\"\"\n", + " # Without\n", + " else:\n", + " text = f\"\"\"[INST]{prefix_text} {data_point[\"instruction\"]} [/INST]{data_point[\"output\"]} \"\"\"\n", + " return text\n", + "\n", + "# add the \"prompt\" column in the dataset\n", + "text_column = [generate_prompt(data_point) for data_point in dataset]\n", + "dataset = dataset.add_column(\"prompt\", text_column)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "dataset = dataset.shuffle(seed=1234) # Shuffle dataset here\n", + "dataset = dataset.map(lambda samples: tokenizer(samples[\"prompt\"]), batched=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [], + "source": [ + "dataset = dataset.train_test_split(test_size=0.2)\n", + "train_data = dataset[\"train\"]\n", + "test_data = dataset[\"test\"]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "train_data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "train_data[\"input_ids\"][:10]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### After Formatting, We should get something like this\n", + "\n", + "```json\n", + "{\n", + "\"text\":\"[INST] Create a function to calculate the sum of a sequence of integers. here are the inputs [1, 2, 3, 4, 5] [/INST]\n", + "# Python code def sum_sequence(sequence): sum = 0 for num in sequence: sum += num return sum\",\n", + "\"instruction\":\"Create a function to calculate the sum of a sequence of integers\",\n", + "\"input\":\"[1, 2, 3, 4, 5]\",\n", + "\"output\":\"# Python code def sum_sequence(sequence): sum = 0 for num in,\n", + " sequence: sum += num return sum\"\n", + "\"prompt\":\"[INST] Create a function to calculate the sum of a sequence of integers. here are the inputs [1, 2, 3, 4, 5] [/INST]\n", + "# Python code def sum_sequence(sequence): sum = 0 for num in sequence: sum += num return sum\"\n", + "\n", + "}\n", + "```\n", + "\n", + "While using SFT (**[Supervised Fine-tuning Trainer](https://huggingface.co/docs/trl/main/en/sft_trainer)**) for fine-tuning, we will be only passing in the “text” column of the dataset for fine-tuning." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(test_data)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "g-Vs5Dc3DdY_" + }, + "source": [ + "### Setting up the Training\n", + "we will be using the `huggingface` and the `peft` library!" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "id": "SQulqDzjd0gD" + }, + "outputs": [], + "source": [ + "from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training\n", + "\n", + "peft_config = LoraConfig(\n", + " lora_alpha=16,\n", + " lora_dropout=0.1,\n", + " r=64,\n", + " bias=\"none\",\n", + " target_modules=[\n", + " \"q_proj\",\n", + " \"k_proj\",\n", + " \"v_proj\",\n", + " \"o_proj\",\n", + " \"gate_proj\",\n", + " \"up_proj\",\n", + " \"down_proj\",\n", + " \"lm_head\",\n", + " ],\n", + " task_type=\"CAUSAL_LM\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "1SWl1MgjDrXX" + }, + "source": [ + "we need to prepare the model to be trained in 4bit so we will use the `prepare_model_for_kbit_training` function from peft\n", + "\n", + "> Indented block\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": { + "id": "bcco3ITVd486" + }, + "outputs": [], + "source": [ + "model = prepare_model_for_kbit_training(model)\n", + "model = get_peft_model(model, peft_config)" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": { + "id": "BJzV4BRhyVQU" + }, + "outputs": [], + "source": [ + "def print_trainable_parameters(model):\n", + " \"\"\"\n", + " Prints the number of trainable parameters in the model.\n", + " \"\"\"\n", + " trainable_params = 0\n", + " all_param = 0\n", + " for _, param in model.named_parameters():\n", + " all_param += param.numel()\n", + " if param.requires_grad:\n", + " trainable_params += param.numel()\n", + " print(\n", + " f\"trainable params: {trainable_params} || all params: {all_param} || trainable%: {100 * trainable_params / all_param}\"\n", + " )" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "aWqhs0W-yVQk", + "outputId": "29093942-dc74-43e8-fa29-e5177f7c75cb" + }, + "outputs": [], + "source": [ + "print_trainable_parameters(model)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Model after Adding Lora Config" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "lxN2-hyHyVQk", + "outputId": "342ea747-938a-44c3-cad5-b94b79ebcf0b" + }, + "outputs": [], + "source": [ + "print(model)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "go8_nVq7jO9d" + }, + "source": [ + "### Hyper-paramters for training\n", + "These parameters will depend on how long you want to run training for.\n", + "Most important to consider:\n", + "\n", + "`num_train_epochs/max_steps`: How many iterations over the data you want to do, BE CAREFUL, don't try too many, you will over-fit!!!!!\n", + "\n", + "`learning_rate`: Controls the speed of convergence\n" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": { + "id": "hdzkKPXvyVQk", + "outputId": "2b645f4e-673b-49ec-d463-94a0f806adfe" + }, + "outputs": [], + "source": [ + "if torch.cuda.device_count() > 1: # If more than 1 GPU\n", + " print(torch.cuda.device_count())\n", + " model.is_parallelizable = True\n", + " model.model_parallel = True" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": { + "id": "YhoCjs9md8pB" + }, + "outputs": [], + "source": [ + "from transformers import TrainingArguments\n", + "\n", + "args = TrainingArguments(\n", + " output_dir = \"Mixtral_Alpace_v3\",\n", + " #num_train_epochs=5,\n", + " max_steps = 100, # comment out this line if you want to train in epochs\n", + " per_device_train_batch_size = 32,\n", + " warmup_steps = 0.03,\n", + " logging_steps=10,\n", + " save_strategy=\"epoch\",\n", + " #evaluation_strategy=\"epoch\",\n", + " evaluation_strategy=\"steps\",\n", + " eval_steps=10, # comment out this line if you want to evaluate at the end of each epoch\n", + " learning_rate=2.5e-5,\n", + " bf16=True,\n", + " # lr_scheduler_type='constant',\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Gz6Vhh4WFpMM" + }, + "source": [ + "Setting up the trainer.\n", + "\n", + "`max_seq_length`: Context window size\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "UyyNtDrmeAkc", + "outputId": "4e992c2f-38eb-40fd-e9cf-d4f8ec301b76" + }, + "outputs": [], + "source": [ + "from trl import SFTTrainer\n", + "\n", + "max_seq_length = 1024\n", + "\n", + "trainer = SFTTrainer(\n", + " model=model,\n", + " peft_config=peft_config,\n", + " max_seq_length=max_seq_length,\n", + " tokenizer=tokenizer,\n", + " packing=True,\n", + " args=args,\n", + " dataset_text_field=\"prompt\",\n", + " train_dataset=train_data,\n", + " eval_dataset=test_data,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 373 + }, + "id": "bsOO4bR9fQBb", + "outputId": "cb946242-b113-4a57-a70b-8fbe05bbac07" + }, + "outputs": [], + "source": [ + "trainer.train()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "ysHgWnwbfRSt" + }, + "outputs": [], + "source": [ + "trainer.save_model(\"Mixtral_Alpace_v2\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "na6xC6f-mGGm" + }, + "source": [ + "# Save Model and Push to Hub" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "SBPVlNe0j-Nk" + }, + "outputs": [], + "source": [ + "# !pip install huggingface-hub -qU" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "uCDmkWCXnmuv" + }, + "outputs": [], + "source": [ + "# from huggingface_hub import notebook_login\n", + "\n", + "# notebook_login()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "HJTZaSXqnqPa" + }, + "outputs": [], + "source": [ + "# trainer.push_to_hub(\"Promptengineering/mistral-instruct-generation\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "9zYrr6sfkA6M" + }, + "outputs": [], + "source": [ + "merged_model = model.merge_and_unload()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "c1Eo6D2mnOgN" + }, + "outputs": [], + "source": [ + "def generate_response(prompt, model):\n", + " encoded_input = tokenizer(prompt, return_tensors=\"pt\", add_special_tokens=True)\n", + " model_inputs = encoded_input.to('cuda')\n", + "\n", + " generated_ids = model.generate(**model_inputs,\n", + " max_new_tokens=150,\n", + " do_sample=True,\n", + " pad_token_id=tokenizer.eos_token_id)\n", + "\n", + " decoded_output = tokenizer.batch_decode(generated_ids)\n", + "\n", + " return decoded_output[0]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "fVhc6eD-yVQl" + }, + "outputs": [], + "source": [ + "prompt = \"[INST]Use the provided input to create an instruction that could have been used to generate the response with an LLM.\\nThere are more than 12,000 species of grass. The most common is Kentucky Bluegrass, because it grows quickly, easily, and is soft to the touch. Rygrass is shiny and bright green colored. Fescues are dark green and shiny. Bermuda grass is harder but can grow in drier soil.[/INST]\"\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "NMFDDMwly9L4" + }, + "outputs": [], + "source": [ + "generate_response(prompt, merged_model)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "ldSnbui8yVQl", + "outputId": "48427c76-a18c-47b8-b92b-d54e8cf67a60" + }, + "outputs": [], + "source": [ + "250*32" + ] + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "gpuType": "V100", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.13" + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "0055eb4d0d054684878bba6da9cae910": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "084223b1556a420da5964ee086c117e0": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "09f69e88c4834d25be2be790022384b0": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "0b9648d0eccf4516a8c0a7e3843f76b8": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "118879d64cae46f5b3e744f86404922c": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "164bc349194f4e6ab1229f3d2d03b3db": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "1c03379c0bf84b62a671021810a5caee": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "2073bd16554341df9ea62874f4907e18": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "21ac78e940c84842bdde1b64bd1395ad": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "240f5b4175e4469381a53eacdf657419": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "290face1ed4647d7b172109e162c3769": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_773040f77dae4115aefdd958a658a3fc", + "max": 116, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_876619b7e19441a096225f72e58b5544", + "value": 116 + } + }, + "296ee6c082da4c76be2b099484662d26": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "2998cf3d58354f6f8465a6e2a10ebab2": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "2de61bee8b56442887b6f42592423064": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "2efa53a79dd54d86b72284f3a87c7ca9": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_164bc349194f4e6ab1229f3d2d03b3db", + "placeholder": "​", + "style": "IPY_MODEL_732e5508dfb1422c97c0fbc27b888f91", + "value": " 571/571 [00:00<00:00, 36.4kB/s]" + } + }, + "3454ad0be8554f39bea55eb8d9c3db1b": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "35f2eadd1bca48bdbc6504fcd693dab7": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "4245f82dafc0478ca7ac1f5b372e5a12": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_82c83b75261c4f1db9b7c3d9ad0d097e", + "placeholder": "​", + "style": "IPY_MODEL_fc38a40ba92d46c49b3679a84c8ef879", + "value": "pytorch_model-00002-of-00002.bin: 100%" + } + }, + "42becf797d9a42bc9984d8e30b307cff": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "444b9f99a79045b9a5c779a3d18bedd4": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_240f5b4175e4469381a53eacdf657419", + "placeholder": "​", + "style": "IPY_MODEL_7af0d44687f142288e3e58f15338dedd", + "value": "generation_config.json: 100%" + } + }, + "48648a0d46104a0db77eabf6d8fdf0f0": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "494347c07d684876ba64b39d698250d0": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_fb5aaa2a96e047a0aa47d07a07b183fe", + "max": 5064823659, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_f7a940cb871f41c7af7e79c32c3156e7", + "value": 5064823659 + } + }, + "4fbd859668ac43c884d39f675f1158f7": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_fbe68ed8ac6243f180408175284b98f5", + "placeholder": "​", + "style": "IPY_MODEL_86712973ec7e498da66f29dbcea3f24f", + "value": "config.json: 100%" + } + }, + "50b0bd7f60ea41c59d6cbd055a10d765": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_85f25707d0a04084aaa1ada15616f415", + "max": 2, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_553955d94d844f05bc19116611761127", + "value": 2 + } + }, + "50d301095b594acdb2d754377b029245": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "553955d94d844f05bc19116611761127": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "6223c94b2b814f2ab7abe976402551b1": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_f56251c2b16c4dae99f80db60e5f6330", + "placeholder": "​", + "style": "IPY_MODEL_9e98f0e4ef964bbeaa63ca278313bb19", + "value": " 5.06G/5.06G [00:17<00:00, 438MB/s]" + } + }, + "643f193c688d49a0a204ea70019f2cc5": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "68b12f6f28fa452ea302d83e37084c43": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_f369a81a80d84934881c8e9a0bc6a14f", + "max": 571, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_72588505d177495b88c39917ae9f743c", + "value": 571 + } + }, + "6950d59bdf254dd896c9f30a220c0533": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_0b9648d0eccf4516a8c0a7e3843f76b8", + "max": 23950, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_a2a870bddb694a8ea69bdac062ef6d46", + "value": 23950 + } + }, + "72588505d177495b88c39917ae9f743c": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "72a4b9321abf4d58a093a3d09263c1ad": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "732e5508dfb1422c97c0fbc27b888f91": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "74b57be114494edda79768ff04961193": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "773040f77dae4115aefdd958a658a3fc": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "79d82f5e0881485897a8e977269079c8": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "7af0d44687f142288e3e58f15338dedd": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "7d8f03389fc64ddfa128be53ea2eb1e8": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_21ac78e940c84842bdde1b64bd1395ad", + "max": 2, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_ddfc0a4121b8447b84abab68482e46a3", + "value": 2 + } + }, + "7e5645b33aa64e7ebbd60cb76e1ecd11": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_4245f82dafc0478ca7ac1f5b372e5a12", + "IPY_MODEL_494347c07d684876ba64b39d698250d0", + "IPY_MODEL_6223c94b2b814f2ab7abe976402551b1" + ], + "layout": "IPY_MODEL_48648a0d46104a0db77eabf6d8fdf0f0" + } + }, + "82c83b75261c4f1db9b7c3d9ad0d097e": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "82cafea491ca4350bea831165926d9fb": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_f3d9d131f5934a048930bc8983437983", + "IPY_MODEL_50b0bd7f60ea41c59d6cbd055a10d765", + "IPY_MODEL_f27e89855a374c6d93e292dce8e146c5" + ], + "layout": "IPY_MODEL_72a4b9321abf4d58a093a3d09263c1ad" + } + }, + "838cd2c0701c4b0b82fa8ddce51215db": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "85f25707d0a04084aaa1ada15616f415": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "86712973ec7e498da66f29dbcea3f24f": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "876619b7e19441a096225f72e58b5544": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "8aec8536c28f4c75b2728936f6843a2c": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "90de7d165e804fbf8f3782ac5b16f8fb": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "95863a2d3c074f45b79a894cb8e67d27": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "963c8c0504554e57bb9a6a8e2c2060fe": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_118879d64cae46f5b3e744f86404922c", + "placeholder": "​", + "style": "IPY_MODEL_0055eb4d0d054684878bba6da9cae910", + "value": "pytorch_model.bin.index.json: 100%" + } + }, + "9e98f0e4ef964bbeaa63ca278313bb19": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "a2a870bddb694a8ea69bdac062ef6d46": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "adcfd8c2daa14f81aea6c85bf98cb735": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_42becf797d9a42bc9984d8e30b307cff", + "placeholder": "​", + "style": "IPY_MODEL_1c03379c0bf84b62a671021810a5caee", + "value": "Downloading shards: 100%" + } + }, + "ae012e86315f41e7ae12b84312d2455b": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_444b9f99a79045b9a5c779a3d18bedd4", + "IPY_MODEL_290face1ed4647d7b172109e162c3769", + "IPY_MODEL_fbeb4393ccb34413bea869355235d0fa" + ], + "layout": "IPY_MODEL_838cd2c0701c4b0b82fa8ddce51215db" + } + }, + "b245d2f284d7450b9d99e820f6193739": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "b3a9937a7f5642e1bdddd2cb1c723490": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_2de61bee8b56442887b6f42592423064", + "max": 9943028044, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_79d82f5e0881485897a8e977269079c8", + "value": 9943028044 + } + }, + "b414aed96db744e99cbe4bf052426fa9": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "becf709cc6a4490481f6cbebbc6bad49": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_df28571a454d47d698d56d7cf4d90009", + "placeholder": "​", + "style": "IPY_MODEL_b414aed96db744e99cbe4bf052426fa9", + "value": " 23.9k/23.9k [00:00<00:00, 1.83MB/s]" + } + }, + "c07ef3e83cec46ef8a9fc9d7d3fdb6fe": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_b245d2f284d7450b9d99e820f6193739", + "placeholder": "​", + "style": "IPY_MODEL_09f69e88c4834d25be2be790022384b0", + "value": " 9.94G/9.94G [00:32<00:00, 324MB/s]" + } + }, + "c577ab91428c4f5aac8299ba85104dc6": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_4fbd859668ac43c884d39f675f1158f7", + "IPY_MODEL_68b12f6f28fa452ea302d83e37084c43", + "IPY_MODEL_2efa53a79dd54d86b72284f3a87c7ca9" + ], + "layout": "IPY_MODEL_643f193c688d49a0a204ea70019f2cc5" + } + }, + "c65ab63b72d143c7bca446276d0f1aa5": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_963c8c0504554e57bb9a6a8e2c2060fe", + "IPY_MODEL_6950d59bdf254dd896c9f30a220c0533", + "IPY_MODEL_becf709cc6a4490481f6cbebbc6bad49" + ], + "layout": "IPY_MODEL_90de7d165e804fbf8f3782ac5b16f8fb" + } + }, + "ccbc22e41b754682bea5e80531498470": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "d429cb7700aa40099e28fa9b6565be26": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_db518d1bca324c98911ad35e5df91c3a", + "IPY_MODEL_b3a9937a7f5642e1bdddd2cb1c723490", + "IPY_MODEL_c07ef3e83cec46ef8a9fc9d7d3fdb6fe" + ], + "layout": "IPY_MODEL_74b57be114494edda79768ff04961193" + } + }, + "dae5a71abe874e69b3972d5fc861fd1a": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "db518d1bca324c98911ad35e5df91c3a": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_ccbc22e41b754682bea5e80531498470", + "placeholder": "​", + "style": "IPY_MODEL_3454ad0be8554f39bea55eb8d9c3db1b", + "value": "pytorch_model-00001-of-00002.bin: 100%" + } + }, + "ddfc0a4121b8447b84abab68482e46a3": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "df28571a454d47d698d56d7cf4d90009": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "ed774deaa35441578397fa1241118263": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_adcfd8c2daa14f81aea6c85bf98cb735", + "IPY_MODEL_7d8f03389fc64ddfa128be53ea2eb1e8", + "IPY_MODEL_fd169008a90840f69ca3ff57ad7f84ed" + ], + "layout": "IPY_MODEL_8aec8536c28f4c75b2728936f6843a2c" + } + }, + "f27e89855a374c6d93e292dce8e146c5": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_296ee6c082da4c76be2b099484662d26", + "placeholder": "​", + "style": "IPY_MODEL_2073bd16554341df9ea62874f4907e18", + "value": " 2/2 [00:14<00:00, 6.73s/it]" + } + }, + "f369a81a80d84934881c8e9a0bc6a14f": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "f3d9d131f5934a048930bc8983437983": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_35f2eadd1bca48bdbc6504fcd693dab7", + "placeholder": "​", + "style": "IPY_MODEL_084223b1556a420da5964ee086c117e0", + "value": "Loading checkpoint shards: 100%" + } + }, + "f56251c2b16c4dae99f80db60e5f6330": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "f7a940cb871f41c7af7e79c32c3156e7": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "fb5aaa2a96e047a0aa47d07a07b183fe": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "fbe68ed8ac6243f180408175284b98f5": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "fbeb4393ccb34413bea869355235d0fa": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_95863a2d3c074f45b79a894cb8e67d27", + "placeholder": "​", + "style": "IPY_MODEL_2998cf3d58354f6f8465a6e2a10ebab2", + "value": " 116/116 [00:00<00:00, 9.67kB/s]" + } + }, + "fc38a40ba92d46c49b3679a84c8ef879": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "fd169008a90840f69ca3ff57ad7f84ed": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_50d301095b594acdb2d754377b029245", + "placeholder": "​", + "style": "IPY_MODEL_dae5a71abe874e69b3972d5fc861fd1a", + "value": " 2/2 [00:51<00:00, 24.32s/it]" + } + } + } + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/mkdocs.yml b/mkdocs.yml index 7692294..62bb01f 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,160 +1,3 @@ -# site_name: AI Engineering.academy -# docs_dir: . -# site_dir: ../site -# plugins: -# - search -# - same-dir -# - exclude: -# glob: -# - archives/** -# - mkdocs-jupyter: -# include_source: true -# ignore_h1_titles: true -# execute: false -# allow_errors: true -# ignore_patterns: -# - "archives/*" -# - mkdocstrings: -# default_handler: python -# handlers: -# python: -# rendering: -# show_source: true -# theme: -# name: material -# features: -# - navigation.tabs -# - navigation.sections -# - toc.integrate -# - navigation.top -# - search.suggest -# - search.highlight -# - content.tabs.link -# - content.code.annotation -# - content.code.copy -# language: en -# palette: -# - scheme: default -# toggle: -# icon: material/toggle-switch-off-outline -# name: Switch to dark mode -# primary: teal -# accent: purple -# - scheme: slate -# toggle: -# icon: material/toggle-switch -# name: Switch to light mode -# primary: teal -# accent: lime -# markdown_extensions: -# - pymdownx.highlight: -# anchor_linenums: true -# - pymdownx.inlinehilite -# - pymdownx.snippets -# - admonition -# - pymdownx.arithmatex: -# generic: true -# - footnotes -# - pymdownx.details -# - pymdownx.superfences: -# custom_fences: -# - name: mermaid -# class: mermaid -# format: !!python/name:pymdownx.superfences.fence_code_format -# - pymdownx.mark -# - attr_list - -# nav: -# - Home: README.md -# - RAG: -# - Introduction: RAG/README.md -# - RAG from Scratch: -# - Overview: RAG/00_RAG_from_Scratch/README.md -# - Implementation: RAG/00_RAG_from_Scratch/RAG_in_10_lines.ipynb -# - Basic RAG: -# - Overview: RAG/01_Basic_RAG/README.md -# - Implementation: RAG/01_Basic_RAG/notebook.ipynb -# - Evaluation: RAG/01_Basic_RAG/notebook_eval.ipynb -# - BM25 RAG: -# - Overview: RAG/01_BM25_RAG/README.md -# - Implementation: RAG/01_BM25_RAG/notebook.ipynb -# - Data Ingestion: -# - Overview: RAG/01_Data_Ingestion/README.md -# - Data Parsing: RAG/01_Data_Ingestion/data_parsing.ipynb -# - Data Chunking: RAG/01_Data_Ingestion/data_chunking.ipynb -# - Data Embedding: RAG/01_Data_Ingestion/data_embedding.ipynb -# - Data Ingestion: RAG/01_Data_Ingestion/data_ingestion.ipynb -# - RAG Evaluation: -# - Overview: RAG/01_RAG_Evaluation/README.md -# - RAGAS: RAG/01_RAG_Evaluation/RAGAS.ipynb -# - DeepEval: RAG/01_RAG_Evaluation/deepeval.ipynb -# - TruLens: RAG/01_RAG_Evaluation/trulens.ipynb -# - Notebook: RAG/01_RAG_Evaluation/notebook.ipynb -# - RAG Observability: -# - Overview: RAG/01_RAG_Observability/README.md -# - Implementation: RAG/01_RAG_Observability/notebook.ipynb -# - ReRanker RAG: -# - Overview: RAG/02_ReRanker_RAG/README.md -# - Implementation: RAG/02_ReRanker_RAG/notebook.ipynb -# - Evaluation: RAG/02_ReRanker_RAG/notebook_eval.ipynb -# - Hybrid RAG: -# - Overview: RAG/03_Hybrid_RAG/README.md -# - Qdrant Hybrid Search: RAG/03_Hybrid_RAG/_Qdrant_Hybrid_Search.ipynb -# - Implementation: RAG/03_Hybrid_RAG/qdrant_hybrid.ipynb -# - Sentence Window RAG: -# - Overview: RAG/04_Sentence_Window_RAG/README.md -# - Implementation: RAG/04_Sentence_Window_RAG/Sentence_window_retrieval.ipynb -# - Auto Merging RAG: -# - Overview: RAG/05_Auto_Merging_RAG/README.md -# - Implementation: RAG/05_Auto_Merging_RAG/Auto-merging_Retrieval.ipynb -# - HyDE RAG: -# - Overview: RAG/06_HyDE_RAG/README.md -# - Implementation: RAG/06_HyDE_RAG/HyDEQueryTransformDemo.ipynb -# - Query Transformation RAG: -# - Overview: RAG/06_Query_Transformation_RAG/README.md -# - Implementation: RAG/06_Query_Transformation_RAG/query_transform_cookbook.ipynb -# - Self Query RAG: -# - Overview: RAG/07_Self_Query_RAG/README.md -# - Implementation: RAG/07_Self_Query_RAG/Self_Query_RAG.ipynb -# - RAG Fusion: -# - Overview: RAG/08_RAG_Fusion/README.md -# - Implementation: RAG/08_RAG_Fusion/ragfusion.ipynb -# - RAPTOR: -# - Overview: RAG/09_RAPTOR/README.md -# - Implementation: RAG/09_RAPTOR/raptor.ipynb -# - ColBERT RAG: -# - Overview: RAG/10_ColBERT_RAG/README.md -# - Implementation: RAG/10_ColBERT_RAG/ColBert_RAG.ipynb -# - Ragatouille Retriever: RAG/10_ColBERT_RAG/ragatouille_retriever.ipynb -# - Graph RAG: -# - Overview: RAG/11_Graph_RAG/README.md -# - Implementation: RAG/11_Graph_RAG/GraphRAG_v1.ipynb -# - Agentic RAG: -# - Overview: RAG/12_Agnetic_RAG/README.md -# - Implementation: RAG/12_Agnetic_RAG/multi_document_agents.ipynb -# - Vision RAG: -# - Implementation: RAG/13_Vision_RAG/gpt4v_multi_modal_retrieval.ipynb -# - Finetuning: -# - Introduction: Finetuning/README.md -# - LLM: -# - Gemma: Finetuning/LLM/Gemma_finetuning_notebook.ipynb -# - Llama2: Finetuning/LLM/Llama2_finetuning_notebook.ipynb -# - Llama3: Finetuning/LLM/Llama3_finetuning_notebook.ipynb -# - Mistral: Finetuning/LLM/Mistral_finetuning_notebook.ipynb -# - Deployment: -# - Introduction: Deployment/README.md -# - Projects: -# - Introduction: Projects/README.md - -# extra: -# exclude_files: -# - archives/**/* - -# extra_javascript: -# - https://unpkg.com/mermaid@8.11.2/dist/mermaid.min.js - -# extra_css: -# - css/extra.css site_name: AI Engineering Academy site_url: https://aiengineering.academy repo_url: https://github.com/adithya-s-k/AI-Engineering.academy @@ -297,17 +140,47 @@ nav: - Implementation: RAG/12_Agnetic_RAG/multi_document_agents.ipynb - Vision RAG: - Implementation: RAG/13_Vision_RAG/gpt4v_multi_modal_retrieval.ipynb - - Finetuning: + - LLM: - Introduction: Finetuning/README.md - LLM: - - Gemma: Finetuning/LLM/Gemma_finetuning_notebook.ipynb - - Llama2: Finetuning/LLM/Llama2_finetuning_notebook.ipynb - - Llama3: Finetuning/LLM/Llama3_finetuning_notebook.ipynb - - Mistral: Finetuning/LLM/Mistral_finetuning_notebook.ipynb + - Gemma: + - Overview: LLM/Gemma/README.md + - Implementation: LLM/Gemma/Gemma_finetuning_notebook.ipynb + - Llama2: + - Overview: LLM/LLama2/README.md + - Implementation: LLM/LLama2/Llama2_finetuning_notebook.ipynb + - QLora: LLM/LLama2/Llama_2_Fine_Tuning_using_QLora.ipynb + - Llama3: LLM/Llama3_finetuning_notebook.ipynb + - Mistral: + - Overview: LLM/Mistral-7b/README.md + - Implementation: LLM/Mistral-7b/Mistral_finetuning_notebook.ipynb + - Evaluation: LLM/Mistral-7b/LLM_evaluation_harness_for_Arc_Easy_and_SST.ipynb + - DPO Fine-tuning: LLM/Mistral-7b/notebooks_DPO_fine_tuning.ipynb + - SFT Trainer: LLM/Mistral-7b/notebooks_SFTTrainer TRL.ipynb + - ChatML Inference: LLM/Mistral-7b/notebooks_chatml_inference.ipynb + - Mixtral: LLM/Mixtral/Mixtral_fine_tuning.ipynb + - VLM: + - Florence2: LLM/VLM/Florence2_finetuning_notebook.ipynb + - PaliGemma: LLM/VLM/PaliGemma_finetuning_notebook.ipynb + - LLM Architecture: + - Lora Explained: LLM/LLMArchitecture/LoraExplained/README.md + - Parameter Count: LLM/LLMArchitecture/ParameterCount.ipynb - Deployment: - Introduction: Deployment/README.md + - Deploy LLM: + - Overview: Deployment/DeployLLM/README.md + - Quantization: + - AWQ: Deployment/Quantization/AWQ_Quantization.ipynb + - GGUF: Deployment/Quantization/GGUF_Quantization.ipynb - Projects: - Introduction: Projects/README.md + - YouTube Clones: + - Overview: Projects/YT_Clones/README.md + - Fireship Clone: Projects/YT_Clones/Fireship_clone.ipynb + - Dataset Preparation: Projects/YT_Clones/dataset_prep.ipynb + - Agents: + - Overview: Agents/README.md + - Multi-document Agents: Agents/multi_document_agents.ipynb extra: social: